API Bulk Operations

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 2013-05-29 20:12, Dimiter Naydenov wrote:
> Hi guys,
> 
> I decided to summarize what we discussed lately about API bulk 
> operations support.
> 
> 1. Why do we need this?
> 
> We'll need bulk operations mostly for clients like the GUI or CLI 
> commands. Good example is "juju status" - having to issue one
> multiple requests per entity is a bad design and won't be usable
> with large environments. Now we get all machines, for each one we
> get the status, agent alive, constraints, etc. Then all services,
> each unit, along with their status, etc. These are a lot of
> round-trips and considering a large environment (100 or more
> machines, services, units, relations) will take a very long time.

Just to put some numbers here. For "very long time", with 70ms latency
and a unit size of about 700 units, around 500 "services", it took
2min to issue 'juju status', which could be dropped to around 6s by
taking out the round trip querying. (I did it with concurrent
requests, it would be better to do it with batch requests.)
...

> 5. Can we have both bulk and single operations in the API?
> 
> I don't see why all the operations *have* to be bulk. There are
> lots of examples of APIs like that out there, used a lot and their
> users are happy. Openstack comes to mind, also AWS. But having bulk
> operations in the first place brings up all the aforementioned
> things, which need to be resolved.

I don't know that every operation has to be bulk. But I also know that
when we've encountered APIs in Openstack that aren't in bulk, it often
is a problem (Delete from swift, for example).

> 
> Sorry it came a lot longer that I expected, but I think I covered
> all the important bits.
> 
> Let's have the discussion and decide which implementation to use
> now, before the API is used everywhere and it's hard to go back.
> 
> Cheers, Dimiter
> 

I'm not sure what the final decision here is. I would suggest that we
default to bulk unless it is clear there isn't a point.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGmvJgACgkQJdeBCYSNAAOeewCgvxso6zGuNGh0jIocwmTabTcV
AIgAnR23wa1U8ZFpG/ePFHmRu0MrQ5Vw
=/n94
-----END PGP SIGNATURE-----