Intentionally introducing failures into Juju

[Stuart Bishop]

On 14 August 2014 07:31, Menno Smits <menno.smits at canonical.com> wrote:
> I like the idea being able to trigger failures using the juju command line.
>
> I'm undecided about how the need to fail should be stored. An obvious
> location would be in a new collection managed by state, or even as a field
> on existing state objects and documents. The downside of this approach is
> that a connection to state will then need to be available from where-ever we
> would like failures to be triggered - this isn't always possible or
> convenient.
>
> Another approach would be to have "juju inject-failure" drop files in some
> location (along the lines of what I've already implemented) using SSH. This
> has the advantage of making the failure checks easy to perform from anywhere
> with the disadvantage of making it more difficult to manage existing
> failures. There would also be some added complexity when creating failure
> files for about-to-be-created entities (e.g. the "juju deploy
> --inject-failure" case).
>
> Do you have any thoughts on this?


Further to just injecting failures, I'm interested in controlling when
and the order hooks can run. A sort of manual mode, which could be
driven by a test harness such as Amulet. Perhaps all hooks in the
queue are initially held, and I can unhold them one at a time.

This would let me test the odd edge cases, such as peers departing
peer relations during handshaking, or what happens when a new client
unit is added and its relation-changed hooks manages to run before the
relation-joined hooks at the server end.

If you could do this, you could inject your failures by actually
breaking your units using juju run or juju ssh. Deploy your units, run
the install hooks, juju ssh in breaking one of the units (rm -rf /,
whatever), run the peer relation hooks, confirm that the service is
still usable despite the failed unit.

-- 
Stuart Bishop <stuart.bishop at canonical.com>