API updates

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ian, William, and I discussed the changes to how we implement API
requests. And I think we sorted out a few points. Hopefully they agree
that this is what we agreed on. :)

1) The server side of the APIs should all support batch requests. (eg,
we shouldn't have Unit.Get(one), but Units.Get([]many).)

2) Client side, we'll still write a lot of stuff as
   myunit = client.Unit.Get(one)

As most clients will be concerned with only 1 thing
(Unit/Machine/Agent, etc).

3) We aren't going to put versions into the API today. Instead, we
expect that when we have to update (MVU) we'll insert them to the
names. So something like Units.Get() will probably become Units.GetV2()

This will likely be versioned per-request, rather than versioning the
API as a whole.

Lots of stuff not discussed here (how do we deprecate old things,
eventually removing them, etc.)

4) API requests will generally *not* return full objects as responses.
(eg Machine.AddUnit()[0] won't return a full representation of the
machine with the new unit added, and the dirty flag changed, and
whatever other records that might have been changed by a 3rd party)

5) It is reasonable for the client side to update fields on the
in-memory structure that it knows are modified as a side-effect of the
API it called (eg Machine.AddUnit knows to set dirty=true locally,
even though it isn't a Machine.SetDirty call). However, it is expected
to *not* modify fields that aren't obviously related (after calling
AddUnit, it shouldn't have the side effect of seeing a new
Machine.Name value.)

6) *Most* API calls are going to be field-level changes, with
field-level transactional guarantees, rather than object-level
guarantees.

eg, Machine.ChangeName(old, new) should probably take the old value
and the new value, and do a DB transaction that asserts name is still
old before setting it to new. However, Machine.AddUnit doesn't need to
lock the whole Machine object in order to add a unit. Thus
Machine.ChangeName from client A will conflict with a similar request
from client B, but will *not* conflict with a call to Machine.AddUnit
from client C.

7) There will be some logic server side to handle concurrency, and
conflict resolution occurs in the API server as much as possible.

eg: Client A calls Machine.ChangeName("foo", "bar") and Client B calls
Machine.ChangeName("foo", "bar") at the same time.
API Server A wins, and the change goes into the DB in a simple
transaction.
API server B tries, but sees that the DB transaction fails because the
precondition "foo" is not met. In then inspects the DB to see that
"bar" is already set. In which case it can return Success to Client B,
because the *effect* of the request has been satisfied, even if it
wasn't done *for* Client B.

eg: Unit.LifeCycle.SetDying() can see that the lifecycle is already
Dead, so it can return Success.

8) It is expected that the primary mechanism for keeping clients up to
date is actually Watchers.

eg: Machine.AddUnit won't notice that the Name changed. Instead, a
client interested in Machine-0 needs to have a watcher on Machine-0,
when something (such as name or units) changes on Machine-0 the
watcher will get a notification "You need to refresh your information
about Machine-0", and then the client can use Machine.Refresh() to
actually get that information.

9) Clients can be optimistic, but should be defensive. They can assume
that their local information is correct (Machine.name is still the
same), but should be prepared to get failures and retry with correct
information.
This is somewhat similar to ETAFFTP vs LYBL (easier to ask for
forgiveness than permission, vs look-before-you-leap).

Structurally, this means that stuff should generally not do a Refresh
before doing something, but should be prepared to get a failure and
Refresh and then try again. *Except* in the case of a Watcher
notification, where it is expected that the first step would be to do
a Refresh and then handle the new information.

10) If there are transactional requirements, these are taken care of
in the state code (by doing assertions in the DB transaction
generally). So if something needs to handle multiple-field
correctness, then it should happen at this level.

Any other points I might have glossed over?

John
=:->


[0] Note that server-side because of (1) this is probably actually
Machines.AddUnits(many => many).
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGl5VMACgkQJdeBCYSNAAPmqACeJV1Ss4FdO7yF1mhq/ekSvlVx
5jsAoKZhmDU4bsERL9LBUKRz7RFpd01k
=Km0t
-----END PGP SIGNATURE-----