Juju core tasks for GUI project

[Gary Poster]

Thank you very much for the replies, William and Roger.  Comments inline
below.

On 06/11/2013 01:29 PM, roger peppe wrote:
> On 5 June 2013 12:34, William Reade <william.reade at canonical.com> wrote:
>> On Wed, May 29, 2013 at 11:51 PM, Gary Poster <gary.poster at canonical.com>
>> wrote:
>>>
>>> Hi all.  I've talked several times to many of you about four things that
>>> the GUI project needs from Juju core soon, and we have verbal approval
>>> from Roger, William, Mark Ramm, and others.  I'd like to list these
>>> tasks here and see if I can get written agreement, visibility on
>>> scheduling these, and possibly clarity on what else I need to provide to
>>> move things along.
>>>
>>> The GUI team can also provide devs for these goals if sufficient
>>> direction is provided and we all agree that it is a good use of resources.
>>>
>>> == (1) SetUnit API ==
>>>
>>> Goal: Provide an API that lets users set the desired number of units for
>>> a service, and have Juju increase or decrease the units to match.
>>>
>>> Rationale: Concurrency.  Using AddUnit, two people both trying to go
>>> from 5 to 10 units will quite possibly act concurrently and move from 5
>>> to 15 (add 5 and add 5).  Similarly, if two users want to decrease from
>>> 15 units to 10, and they happen to choose different units in RemoveUnit,
>>> they may decrease to 5.
>>>
>>> Looked at another way, AddUnit is not idempotent, and, effectively,
>>> RemoveUnit isn't either if all you want to do is reduce unit count.
>>>
>>> Previously identified design challenge: how does Juju decide which units
>>> should be removed?  One solution bandied about: user can choose to
>>> remove from newest to oldest or from oldest to newest, defaulting to the
>>> latter.  Perhaps user can also optionally provide a list of specific
>>> unit ids to iterate through and remove first, before switching to policy.
>>
>>
>> This will take some work in state, involving either a DB schema change or
>> some *very* fancy footwork, on top of the obvious API work.
>>
>> It would probably be wiser to defer this work until we have either
>> implemented major-version upgrades, or have entirely obscured state behind
>> the API; both of these are very high priorities for us. I don't think core
>> has the bandwidth to take this on in the next ~6 weeks, because we're
>> chasing those goals already, but I'd be very happy to make myself available
>> for a pre-discussion with anyone who felt up to a significant challenge (and
>> to discuss or comment on proposals that fleshed the idea out, particularly
>> wrt removal strategies -- or, potentially, that did an end-run around the
>> removal-strategy problem by implementing SetMinimumUnits [0]).
> 
> I like SetMinimumUnits (I'd suggested the idea before but that name
> makes it nicer still). It means that removal strategy is entirely in the
> hands of the client, whether it's a GUI user clicking "delete" or an auto-scaler
> removing the least performant node.

Yes, we're hoping this will work out.  I'll have more details at the
bottom of the email, where William has a footnote and Roger talks about
this again.

>>>  - Implement a restricted level of authorization.  A user with this
>>> authorization cannot use API calls that mutate Juju, cannot access Juju
>>> debug log output, and cannot see the values of "restricted" service
>>> configuration options, as identified by the previously discussed flag.
>>
>>
>> (...and cannot see sensitive environment config values, like credentials, as
>> identified using the existing SecretAttrs mechanism.)

Ah, ok, thanks for that heads up.

>>
>> This is gated on the user model changes that we have planned, but which we
>> won't get to imminently. It's explicitly planned for core, but I'd be happy
>> to discuss handing over a chunk that unblocks you; I don't currently have a
>> solid feel for how realistic it is.
>>
>>>
>>>  - Implement optional anonymous restricted API access.  This is
>>> initially turned off, and can be turned on by an administrator.
>>
>>
>> This, though, doesn't *necessarily* depend on the user model, and seems like
>> it's potentially quite high-value. I'd be happy to get some help here.
>>
>>>
>>> Status: Roger had a very quick prototype of everything except the charm
>>> config option "restricted" flag working at some point.
>>
>>
>> Roger, I'd like to hear your thoughts here.
> 
> If we assume that we can nicely partition API calls into "restricted"
> and "non-restricted" calls (and I *think* we can) then making
> a restricted API is as simple as providing an object in the API
> with only the restricted methods. The object implementing the non-restricted API
> embed the restricted API and add restricted methods.
> 
> The implementation is trivial and can fit nicely with our recent
> directions in the API server implementation, I think.

Yes, that was my understanding.  This approach, combined with handling
the restricted values, was something Roger felt could be done without
much work.

I've proposed a call with William, Roger, and some GUI folks for later
this week.  The agenda will be to verify that William finds the approach
acceptable, and if so, hand off the implementation plans from Roger to
GUI (or, even better, discover that Roger built the whole thing in his
lunch hour ;-) ).  We'll report back here.

> 
>> [0] hmm, this approach might be possible without breaking compatibility,
>> even today. Deserves further discussion.
> 
> I think so.
> 
> The main difficulty with SetMinimumUnits that I see is that the planned
> GUI interface is still SetNumUnits, and you have to be quite careful
> how you implement unit removal, otherwise two clients might both
> decide to remove different sets of units and we might end up with fewer than
> desired. I *think* that if client choice of units to remove is deterministic
> (e.g. lowest number), then things should be ok.

You are absolutely right that SetMinimumUnits does not quite align with
the existing and currently planned GUI interface.  I have a call with
our UX designer tomorrow to see how we can reconcile the differences.

Francesco and I also had a call with William yesterday about this.  Here
is the outcome of that conversation, thanks to a write-up from
Francesco.  I see a couple of issues in retrospect that I'll mention
after the write-up.

====================================================

Yesterday I had a chat with Gary and William about possible
solutions to a goal we have for the Juju GUI: let users set
the desired number of units for a given service.

Currently the GUI uses the AddServiceUnits and
DestroyServiceUnits API calls, which are not idempotent and
can lead to race conditions if two users add/remove units
at the same time.

We introduced the concept of MinimumUnits, a value set by
the user representing the minimum number of units for a
specific service that Juju pledges to be always alive.

Use cases
---------

Assume we have an initial number of MinimumUnits of one, and
one unit actually deployed.
User A wants the units to be five at least, and changes
MinimumUnits to five. As a consequence, four additional
units are deployed by Juju.
User B removes a unit in an error state, the actual number
of units decreases to four. Juju reacts deploying another
unit so that the MinimumUnits are correctly restored.

API Changes
-----------

*ServiceDeploy* needs to change to include MinimumUnits.
Current parameters:
    ServiceName, CharmUrl, NumUnits, Config, ConfigYAML,
    Constraints, ForceMachineId.
Proposed:
    ServiceName, CharmUrl, NumUnits, MinimumUnits,
    Config, ConfigYAML, Constraints, ForceMachineId./

Also note that [0], currently in review, slightly
changes how the current parameters above are handled
by juju-core:
- ServiceDeploy now validates CharmUrl and returns
  an error if the provided one is not a fully qualified
  charm URL, i.e. cs:series/name-revision;
- serviceName is no longer optional;
- NumUnits must be sane (>0 for principals,
  0 for subordinates).

*ServiceSet* needs to be improved so that it can be used
to configure every aspect of a service.
Current parameters:
    ServiceName, Options (Settings)
Proposed:
    ServiceName  MinimumUnits, Config, ConfigYAML,
    Constraints.

Both the API changes above require modifications
to the current Juju GUI "go.js" environment.

Pre-implementation details
--------------------------

Add the MinimumUnits field to the service document.

Implement a watcher like the following:
- watch all services;
- for each service, track the MinimumUnits value and generate
  events if that value is increased;
- for each service, watch all the units, and generate
  events based on changes on the number of units alive.

Implement a worker listening to events raised by
the watcher above. The worker reacts to changes
adding units if required.


[0] https://codereview.appspot.com/10166044/

====================================================

My first concern is that the use case given implies that the "minimum
units" value is not enforced by refusing to remove units, but only by
starting new units.  This would mean that concurrent users could
mistakenly remove all running units.  They would eventually be replaced,
but that does not sound like the sort of guarantee we are looking for.

If you try to remove a unit with DestroyServiceUnits, but this would
take you beneath the minimum units, I now think that you should get an
error.  Juju should refuse.  Therefore, in the case of replacing a
broken unit in the CLI when you are at your minimum units, you would try
to remove the unit but get an error; add a unit; and then be able to
remove the original unit.  The GUI could make this more automated.

It's also worth noting that I agree that our watcher still needs to
generate events based on number of units alive.  Units might die for
reasons other than calls to DestroyServiceUnits.

Thinking about MinimumUnits further, the simplest way of calculating
existing units is to ignore their state--that is, if you have three
units, and two are running but one has a start-error, you still have
three units, and thus satisfy a "MinimumUnits = 3" scenario.  Does this
simple approach handle our use cases in the desired way?  What if all
three are in an error state?  I hope so, because otherwise the semantics
get complicated, but I think it is worth raising the question.

My other concerns are basically that this story is not quite as simple
and concurrency-friendly as I'd like.  However, I know why we've gotten
here, instead of SetUnit.  Maybe my discussion with UX tomorrow will
point out a way to resolve some or all of my concerns.

Thanks again,

Gary