Recent uniter changes

[Andrew Wilkins]

Hi everybody!

Last month, a small group of us sprinted in Malta to address several issues
with the uniter. We've been meaning to send out a summary for a few weeks,
but there has been some fallout; some bugs introduced, some existing ones
uncovered while shaking code. Sorry about that.

The primary goals we set out to achieve in the sprint were to:
 - simplify the uniter code, isolating concurrency concerns from logic;
 - make the uniter unit-testable (i.e. don’t require JujuConnSuite);
 - make the uniter more robust against failures (e.g. server connection
loss); and
 - allow some uniter operations to run concurrently.

There were three main sets of changes to achieve these goals. First, we
refactored the "modes" and "filter" code of the uniter into a new
"resolver" structure, which I will describe below. Second, we introduced
use of the dependency engine into the uniter. Finally, the metrics
collection code was moved out of worker/uniter altogether, and into its own
worker (worker/metrics/collect).

Moving to the dependency engine infrastructure will enable us to make more
intelligent decisions about what to do in the face of unavailable
resources. For now, we behave the same as before, but we will later enhance
the uniter to gracefully degrade when, for example, the Juju server is
unreachable. In the short term, moving the metrics code out of the uniter
enables the immediate goal of enabling the continued collection of workload
metrics while the server is unavailable; and allowing metrics collection to
occur concurrently with other hooks, so long-running hooks do not impact on
metric collection.

For me, the more interesting outcome is the new "resolver" structure.
Everything below refers to this only.

Previously the uniter's primary logic was interwoven with concurrency
concerns, which you can find in the old "worker/uniter/modes.go"; this code
represented a state machine whose stimuli were the "filter", leadership
trackers, and status timers. The old mode provided an efficient way of
selecting which events to react to at the cost of complicating the code. In
order to make the code simpler and easier to test, we have split the
concurrency concerns out entirely at the cost of slightly less efficient
code (which is not that important here anyway).

We introduced two new packages for this:
 - worker/uniter/remotestate
 - worker/uniter/resolver
 - worker/uniter/actions
 - worker/uniter/leadership
and removed
 - worker/uniter/filter

The remotestate package defines an interface, Watcher, that contains two
main methods: one that returns a channel that is signalled when a change
has occurred to the goal state, and one that fetches the latest snapshot of
the goal state. Note that we don't use a channel of changes because we want
to coalesce them while the client is busy. If you intend to update the
uniter with a new source of changes, this is the place to encapsulate that.

The resolver package defines two main things: the Resolver interface and
the Loop function. Resolver compares the local state with the remote state,
and returns operations that will resolve the differences; each resolver
returns operations in a deterministic order, to ensure predictable hook
ordering, and to simplify testing. The Loop function drives a Resolver by
waiting for changes from a remotestate.Watcher, and executes operations as
they are returned.

The new actions and leadership packages define actions-specific and
leadership-specific resolvers. The existing relation and storage packages
have been updated similarly. To bring them all together, there is
worker/uniter/resolver.go, which delegates to the other resolvers at the
appropriate times.

The existing tests for the uniter have not been rewritten, and I don't
think they will be replaced, but rather we intend to move them to
juju/featuretests as they are and introduce new unit tests that test the
resolvers in isolation. There are some tests written specifically for the
resolvers, but they are not as comprehensive, e.g. in the actions and
relation packages.

Let us know if you have any questions.

Cheers,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20150921/20bfd813/attachment.html>