API updates

[William Reade]

On Wed, May 29, 2013 at 1:24 PM, John Arbash Meinel
<john at arbash-meinel.com>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Ian, William, and I discussed the changes to how we implement API
> requests. And I think we sorted out a few points. Hopefully they agree
> that this is what we agreed on. :)
>

I agree with the bits I don't mention.


> 3) We aren't going to put versions into the API today. Instead, we
> expect that when we have to update (MVU) we'll insert them to the
> names. So something like Units.Get() will probably become Units.GetV2()
>

FWIW, I kinda hate this, and I'd like to talk more about it. I see some
advantage to per-method versioning, but I really think I'd rather version
the "services" themselves. (I'd kinda like to version the whole API, even,
because then clients could just pick a damn endpoint, like most non-hateful
APIs... what are the forces that work against this? We're still wide open,
here, because we're still on "version <nil>" (ie, we will be able to
distinguish this version by its lack of version), but as soon as we release
a non-backward-compatible API version it becomes *much* harder to switch
schemes later.)


> 5) It is reasonable for the client side to update fields on the
> in-memory structure that it knows are modified as a side-effect of the
> API it called (eg Machine.AddUnit knows to set dirty=true locally,
> even though it isn't a Machine.SetDirty call). However, it is expected
> to *not* modify fields that aren't obviously related (after calling
> AddUnit, it shouldn't have the side effect of seeing a new
> Machine.Name value.)
>

Just to expand on this: this is the model used by the state package, and
we've got a bunch of code written with these assumptions in mind, so fixing
this will be very costly; certainly, I think, more so than duplicating the
update-a-few-fields style of the existing state code.


> 6) *Most* API calls are going to be field-level changes, with
> field-level transactional guarantees, rather than object-level
> guarantees.
>
> eg, Machine.ChangeName(old, new) should probably take the old value
> and the new value, and do a DB transaction that asserts name is still
> old before setting it to new. However, Machine.AddUnit doesn't need to
> lock the whole Machine object in order to add a unit. Thus
> Machine.ChangeName from client A will conflict with a similar request
> from client B, but will *not* conflict with a call to Machine.AddUnit
> from client C.
>

If the old value matters, yes. I should point out that there's no
Machine.ChangeName method, and I'd be interested to hear which methods
you're expecting to be an issue here; this is another case where the style
is dictated by the assumptions of the existing code that will be changing
to talk to the API rather than to state.

+1 on the Machines.AssignUnits footnote (there's no AddUnits there ;p)

7) There will be some logic server side to handle concurrency, and
> conflict resolution occurs in the API server as much as possible.
>
> eg: Client A calls Machine.ChangeName("foo", "bar") and Client B calls
> Machine.ChangeName("foo", "bar") at the same time.
> API Server A wins, and the change goes into the DB in a simple
> transaction.
> API server B tries, but sees that the DB transaction fails because the
> precondition "foo" is not met. In then inspects the DB to see that
> "bar" is already set. In which case it can return Success to Client B,
> because the *effect* of the request has been satisfied, even if it
> wasn't done *for* Client B.
>
> eg: Unit.LifeCycle.SetDying() can see that the lifecycle is already
> Dead, so it can return Success.
>

Again, this is a restatement of how state already works. It's not really an
API concern except inasmuch as that it's likely to be very painful to alter
these assumptions across the agent codebase.


> 9) Clients can be optimistic, but should be defensive. They can assume
> that their local information is correct (Machine.name is still the
> same), but should be prepared to get failures and retry with correct
> information.
> This is somewhat similar to ETAFFTP vs LYBL (easier to ask for
> forgiveness than permission, vs look-before-you-leap).
>
> Structurally, this means that stuff should generally not do a Refresh
> before doing something, but should be prepared to get a failure and
> Refresh and then try again. *Except* in the case of a Watcher
> notification, where it is expected that the first step would be to do
> a Refresh and then handle the new information.
>

It actually means we should *never* refresh on the client side except in
response to a watcher event. This is another restatement of how state works
internally.

If (when) we start doing server-side caching (rather than hitting the db
directly for every operation on every entity) it means the same thing; that
server-side entity caches can be implemented completely optimistically,
updated only on watcher events, and calls will always Just Work (or Just
Fail, as appropriate). Once we've got all this stuff behind the API, we can
start being more aggressive about refreshing full objects inside state
calls (when indicated), because the only direct client (the API) will
actually be helped by this behaviour. (er, I think... server-side caching
is some way in the future, our MMV in practice)


> 10) If there are transactional requirements, these are taken care of
> in the state code (by doing assertions in the DB transaction
> generally). So if something needs to handle multiple-field
> correctness, then it should happen at this level.
>
> Any other points I might have glossed over?
>

I think that's pretty solid. Thanks John.

Cheers
William
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20130529/f5b7163e/attachment.html>