Charm store API proposal, new version

[Richard Harding]

On Tue, 15 Jul 2014, Aaron Bentley wrote:

> I am surprised that juju-reports was not considered a known client.  I
> certainly made many comments on the first draft of the original proposal.

It is listed under known clients in the spec, and we mentioned your request
down below.  What we lack is your specific use cases as no one working on
the spec is knowledgeable about how you're using the api.

> Our use case is we want to be able to synchronize public data from the
> charm store into our DB.
>
> We need to provide the statistics that show adoption rates to Mark
> Ramm and Mark Shuttleworth.  We cannot forsee exactly which questions
> will be asked.  We need to be able to add new statistics without
> friction, so we need our own copy of this data.
>
> One way to do this would be to have an API that accepts a timestamp
> and returns the metadata for all charm and bundle revisions created on
> or after that timestamp.
>
> Right now we do this instead:
> 1. Retrieve a list of all charms from charmworld
> 2. Determine the tip revisions of all charms from the charm store
> 3. Determine which revisions exist but we haven't seen (including
> non-tip revisions)
> 4. Determine the revision ids of all the new revisions
> 5. Use the revision-ids to determine the publication dates of all the
> new revisions
>
> Steps 2, 4, 5 use multiple-id queries.
>
> Right now, we're showing number of charms published here:
> http://reports.vapour.ws/scorecard
>
> We'll need to do the same for bundles.
>
> We've also been asked to display the number of running environments.

What specific metadata do you need? I know we need the list of older
revisions of charms for the juju-gui. I'm going to make sure that's listed
as a potential metadata request in the api. (It's missing currently)

Given that data, when you get the latest version of a charm, you'd get a
list of previously published versions to facilitate the upgrade/downgrade
UI. It seems this would help you with the publish counts?

Most of the other data around revisions and bugs we'll do our best at, but
will be sacrificed information so that users can publish a charm or bundle
from any vcs. What other metadata should be make sure is addressed?


> > You had requested an all charms/bundles api call in the last
> > revision of the doc and we're looking at addresssing that through
> > search results.
>
> The doc doesn't say that search can be used to list all charms and
> bundles.  If you meant it to be used that way, please say so in the
> doc ;-)

Sure thing, thanks for pointing it out.


> Search provides results for only the most recent revisions.  It can
> stand in for step 1 & 2 of our algorithm, but in order to implement 4
> and 5 efficiently, we'd need to do multi-id queries.

With the previous rev metadata we're missing but also required by the
juju-gui I think we can resolve this with the current api implementation.
Let me know if there's more we need to address.

Thanks for reviewing the doc and providing feedback. We're very much hoping
to build out a strong consistent api we can use for a long time going
forward.


--

Rick Harding

Juju UI Engineering
https://launchpad.net/~rharding
@mitechie