Charm Quality Ratings updates

[Marco Ceppi]

Just jumping in to throw down some thoughts I have on this before the
discussion skews too far one way or the other.

I've been assigned the task of figuring out how to collect and process
interface/relation data automagically. Like Kapil mentioned, there's
currently a mechanism in Amulet that records relation data via a proxy
service. While it only currently records the end results of an interface
sequence, the next release of Amulet will document each step of the
communication protocol between the two services (as to capture the actual
flow of the protocol). At that point I plan to write automated amulet
deployments from the test-graph service that Kapil created to spin up and
relate as many services as relatable, record their communication, and
publish the results of those communication steps to an API service that I
will write.

This service can then compare how each service talks, the steps it took,
the keys sent, etc. From there we can find the "generally agreed upon"
protocol for each interface, document it, and identify charms that are
outliers to this protocol specification (ones that either don't provide
enough or too much data over the wire). The results of this can/will be
displayed on a webpage describing how the protocol works and what each
service that provides/requires the protocol does. For the most part I
believe this can be nearly 100% auto-generated with no intervention from
the Charm Author. I think this auto-generation is the route to go as
protocol's can change all the time, whether we want them to or not. If we
document what the majority of charms providing/consuming offer at any
single point of time over an interface and the outline the charms that
don't fit that spec we can tackle changes and old charms sooner as well as
alerting new charm authors as to what the "protocol dance" is for any
giving interface.

Since this API service that collects all this data knows the communication
sequence for an interface we can go so far as to even suggest with pseudo
code how to implement a given interface/relation based on what the other
charms that implement that interface do.

Now that Amulet is released and I'm a little more free'd up I hope to
collect my thoughts on this and submit to the list for review.

Thanks,
Marco Ceppi


On Fri, Oct 4, 2013 at 12:05 PM, Kapil Thangavelu <
kapil.thangavelu at canonical.com> wrote:

> So charm interface docs can be had two ways. Fundamentally a doc here
> needs to capture the data and sequence flow between the two related
> services and then some descriptive narrative. The data flow as gustavo
> postulated could be captured via intermediary proxy charm, and i believe
> marco ceppi has done some work on such. An alternative that's a bit tied to
> the low level implementation but is also trivial, is simply a
> capture/report and annotation against the transaction log in juju against
> the two services related unit communication.
>
> cheers,
>
> Kapil
>
>
> On Fri, Oct 4, 2013 at 6:52 AM, Nick Veitch <nick.veitch at canonical.com>wrote:
>
>> On Fri, Oct 4, 2013 at 12:28 PM, Mark Shuttleworth <mark at ubuntu.com>
>> wrote:
>> >
>> > I think we'll need the ability both to document the interface in a
>> particular charm, and to link to other charms
>> > interface docs from inside such documentation.
>>
>> It is easy enough to do that by using Markdown, since we already
>> render that. Do we want it as a separate file? In many ways, it
>> *could* just be part of a well-written README, but then it isn't so
>> obvious where the information actually is.
>>
>> --
>> Juju mailing list
>> Juju at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>
>
> --
> Juju mailing list
> Juju at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20131004/a0cee399/attachment-0001.html>