Aggregate all bundle.yaml | API endpoint

[Marco Ceppi]

Data flow in juju is bi-directional the order does not image or change
anything in the relation

On Thu, Jul 5, 2018, 10:49 Ilias Tsoumas <itsoum at unipi.gr> wrote:

> Ok, it is clear.
> Also, how I identify the "direction" of relationship? It is denoted with
> order of the dashes?
> *- - <application-name1>:<endpoint-name1>*
> *  - <application-name2>:<endpoint-name2>*
> imply that the data-flow from  *<application-name1>* is directed in
> *<application-name2>*
> and
> *- - <application-name2>:<endpoint-name2>*
> *  - <application-name1>:<endpoint-name1>*
> imply that the data-flow from  *<application-name2>* is directed in
> *<application-name1>*
>
> On Thu, Jul 5, 2018 at 5:10 PM Merlijn Sebrechts <
> merlijn.sebrechts at gmail.com> wrote:
>
>> There is nothing preventing directed cycles in the graph, although I'm
>> not sure how common they are in real bundles.
>>
>> On Thu, 5 Jul 2018, 05:57 Ilias Tsoumas, <itsoum at unipi.gr> wrote:
>>
>>> Merlijn thank you for your reply. It was enlightening about the
>>> relations of charms.
>>> Only a comment till now, is each bundle a directed acyclic graph (DAG)?
>>>
>>>
>>> On Tue, Jun 26, 2018 at 7:10 PM Merlijn Sebrechts <
>>> merlijn.sebrechts at gmail.com> wrote:
>>>
>>>> The format is
>>>>
>>>> - - <application-name1>:<endpoint-name1>
>>>>   - <application-name2>:<endpoint-name2>
>>>>
>>>> Where application-name is the name of the _instance_ of the charm (as
>>>> defined in the applications section of bundle.yaml) and endpoint-name is
>>>> the name of the endpoint (requires/provides from metadata.yaml) that is
>>>> used to connect the two nodes.
>>>>
>>>> Endpoint-name is optional if only one type of relation is possible
>>>> between the nodes (a relation is established between a required endpoint of
>>>> one node and a provides endpoint of the same interface type of the other
>>>> node). The : is not present if the endpoint name is not specified.
>>>>
>>>> So the relations section is a 2 dimensional list, the first dimension
>>>> is an array of all the relations and the second dimension specifies which
>>>> endpoints of which nodes that relation connects.
>>>>
>>>> And finally, the applications section is called "services" in some
>>>> bundles.
>>>>
>>>> I think that covers most of the strange quirks, let me know if you have
>>>> more questions.
>>>>
>>>> On Tue, 26 Jun 2018, 05:37 Ilias Tsoumas, <itsoum at unipi.gr> wrote:
>>>>
>>>>> Hello Merlijn and all,
>>>>>
>>>>> I developed a python script which fetching all the bundle.yml.
>>>>> Here is the repo: https://github.com/itsoum/BundlesFetcher
>>>>> Merlijn special thanks for the help about the API.
>>>>>
>>>>> Now, there rise a new "problem". As I had mentioned in the first-post
>>>>> above I fetch bundles for the relations between nodes(charms). But there
>>>>> isn't an obvious annotation technique about how relations are described
>>>>> inside each bundle.yaml.
>>>>> Check some different examples below
>>>>>
>>>>> https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml
>>>>>
>>>>>
>>>>> https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
>>>>> <https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
>>>>>
>>>>>
>>>>> https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
>>>>> <https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>
>>>>>
>>>>> For example in the first:
>>>>> relations:
>>>>> - - haproxy:reverseproxy
>>>>> - wordpress:website
>>>>> - - wordpress:cache
>>>>> - memcached:cache
>>>>> - - wordpress:db
>>>>> - mariadb:db
>>>>> - - mariadb-slave:slave
>>>>> - mariadb:master
>>>>> - - nagios:nagios
>>>>> - wordpress:juju-info which is the topology of application graph. The
>>>>> ':' isn't denotes the edge between two charms? If yes, why - wordpress:db
>>>>> and - mariadb:db?
>>>>> There us someone who knows the annotation method of relations? Could
>>>>> you explain?
>>>>>
>>>>> BR,
>>>>> Ilias
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 5, 2018 at 2:11 PM Ilias Tsoumas <itsoum at unipi.gr> wrote:
>>>>>
>>>>>> Hi Merlijn,
>>>>>>
>>>>>> First of all thank you for your quick and essential answer!
>>>>>>
>>>>>> Reply inline
>>>>>>
>>>>>> Great to hear you're doing cool stuff with Juju! I recently asked a
>>>>>>> similar question for a paper I've written for CLOUD 2018
>>>>>>> https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
>>>>>>> you want, I can send it to you in private (it's accepted but not published
>>>>>>> yet).
>>>>>>>
>>>>>>
>>>>>> Of course. I would like to read your paper. Also me and a colleague
>>>>>> work on performance estimations of app components (=charms). If I have
>>>>>> suspected properly your paper is related with performance
>>>>>> analysis/profiling thus it will be inspiration for us.
>>>>>>
>>>>>> Here is the documentation for the charm store api:
>>>>>>> https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
>>>>>>>
>>>>>>
>>>>>> I will check it.
>>>>>>
>>>>>> Here is an Jupyter notebook I use to get metrics of charm and layer
>>>>>>> usage:
>>>>>>> https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
>>>>>>> It should be easy to modify that code to pull info about the bundles. As an
>>>>>>> example; this is the URL to get all bundles in the Juju store:
>>>>>>> https://api.jujucharms.com/charmstore/v5/list?type=bundle
>>>>>>
>>>>>>
>>>>>> Thank you! I hope to be feasible to pull all bundle.yaml with one
>>>>>> query/request.
>>>>>>
>>>>>> BR,
>>>>>> Ilias
>>>>>>
>>>>>> On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <
>>>>>> merlijn.sebrechts at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Ilias
>>>>>>>
>>>>>>>
>>>>>>> Great to hear you're doing cool stuff with Juju! I recently asked a
>>>>>>> similar question for a paper I've written for CLOUD 2018
>>>>>>> https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If
>>>>>>> you want, I can send it to you in private (it's accepted but not published
>>>>>>> yet).
>>>>>>>
>>>>>>> Here is the documentation for the charm store api:
>>>>>>> https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
>>>>>>>
>>>>>>>    - download the entire zip for a bundle:
>>>>>>>    https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
>>>>>>>    - download a specific file of a bundle:
>>>>>>>    https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
>>>>>>>
>>>>>>> Here is an Jupyter notebook I use to get metrics of charm and layer
>>>>>>> usage:
>>>>>>> https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
>>>>>>> It should be easy to modify that code to pull info about the bundles. As an
>>>>>>> example; this is the URL to get all bundles in the Juju store:
>>>>>>> https://api.jujucharms.com/charmstore/v5/list?type=bundle
>>>>>>>
>>>>>>> Cleaning up the data will be the hard part probably, since there are
>>>>>>> a lot of unused/broken bundles in the store. I've used the downloads/month
>>>>>>> metric to figure out what charms are actually still used.
>>>>>>>
>>>>>>> Op ma 4 jun. 2018 om 14:05 schreef Ilias Tsoumas <itsoum at unipi.gr>:
>>>>>>>
>>>>>>>> Hello all,
>>>>>>>>
>>>>>>>> I have designed and I am developing a novel recommender system
>>>>>>>> which will provide recommendations at application graph (=bundle)
>>>>>>>> composition time. Now I need a real graphs dataset. I think that all juju
>>>>>>>> bundles of components is a good real dataset. Are there some how to
>>>>>>>> aggregate all these "bundle.yaml" files? Are there some api endpoint?
>>>>>>>> This work is funding from the European Union’s Horizon 2020
>>>>>>>> research and innovation program under grant agreement No 761898 project
>>>>>>>> Matilda.
>>>>>>>>
>>>>>>>> Kind regards,
>>>>>>>> Ilias Tsoumas
>>>>>>>> --
>>>>>>>> Juju mailing list
>>>>>>>> Juju at lists.ubuntu.com
>>>>>>>> Modify settings or unsubscribe at:
>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>>>>>>
>>>>>>> --
> Juju mailing list
> Juju at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20180705/05aae9fb/attachment.html>