Charm store API proposal, new version

[roger peppe]

On 16 July 2014 13:32, William Reade <william.reade at canonical.com> wrote:
> On Wed, Jul 16, 2014 at 12:10 PM, Richard Harding
> <rick.harding at canonical.com> wrote:
>>
>> I'm not against having any bulk api calls but we've got a handful of
>> clients and the one use case we've found for them is Aaron's work which I
>> think we can better address with our current scheme anyway as he only
>> needed the bulk api call to get data he'd already have on hand.
>
>
> I'd suggest at least one other use case -- namely, that juju-core wants to
> know which deployed charms have updates available, and it's annoying to
> issue N requests for N charms.
>
> But the major drivers are:
>
> 1) consistency, so that clients across the board can just *always know* that
> they can issue a given request in N-plicate and never have to mess around
> hunting for which variant of a given api call offers what they're looking
> for;

The flip side of this coin, in an HTTP-based API at any rate, is that
bulk requests are not consistent with the way that HTTP requests
are usually phrased.

> 2) expressivity, because you can always express a single request in a bulk
> call but can't go the other way;

Patently untrue, I'm afraid. Single calls and bulk calls are
exactly as expressive as each other. In fact in the juju-core API, it can easily
be faster to issue several concurrent non-bulk calls
than to make the equivalent bulk call, because
the database calls happen concurrently.

> 3) agility(?), in that bulk calls have scope for server-side optimisations
> that would be very very hard to apply across a list of single calls, and
> we'd rather address performance issues via implementation tweaks than API
> churn; and

I know this  (that there is scope for server-side optimisations
of bulk calls) is true in theory, but I have yet to find a decent example.
Every single Juju bulk API call is currently implemented with
a simple for loop AFAIK.

> 4) past experience, both others' (launchpad) and our own, leading us to
> believe that it's generally nicer to work with chunky APIs than chatty ones,
> despite the slightly steeper initial learning curve.
>
> ...not to mention that IMO most of the arguments for single calls represent
> a failure of imagination. I certainly agree that it's not always obvious why
> someone might want to discover the latest revisions of N charms at once, or
> the metadata for M other charms that may or may not be directly related to
> one another, or reporting info for a bunch of others (or, in juju-core
> terms, why we might want to find out the life statuses of a bunch of
> entities, or why the set of state server addresses might vary according to
> what agent needs to connect, or why a client might want provisioning info
> for a whole bunch of new machines at a time).
>
> But simple prudence dictates that we pay the minimal costs of enabling such
> behaviours early, in the interest of providing a rich experience for all our
> clients, rather than scrambling to fix them up later as we discover the use
> cases that always somehow seem terribly predictable in hindsight -- and in
> doing so take on the long-term costs of uglified and non-orthogonal APIs
> (or, perhaps even worse, failing to do so and leaving our clients hanging
> because we're focused on other things).

Again I ask: what do you think of my general bulk-call endpoint
approach that can be used to turn any single id-based query into
a multiple-id query?

  cheers,
    rog.