Aggregate all bundle.yaml | API endpoint

[Ilias Tsoumas]

Hello Merlijn and all,

I developed a python script which fetching all the bundle.yml.
Here is the repo: https://github.com/itsoum/BundlesFetcher
Merlijn special thanks for the help about the API.

Now, there rise a new "problem". As I had mentioned in the first-post above
I fetch bundles for the relations between nodes(charms). But there isn't an
obvious annotation technique about how relations are described inside each
bundle.yaml.
Check some different examples below
https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml

https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/web-in-a-box-9.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>

https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/hadoop-spark-1.yml
<https://github.com/itsoum/BundlesFetcher/blob/master/bundles_yaml/wordpress-site-3.yml>

For example in the first:
relations:
- - haproxy:reverseproxy
- wordpress:website
- - wordpress:cache
- memcached:cache
- - wordpress:db
- mariadb:db
- - mariadb-slave:slave
- mariadb:master
- - nagios:nagios
- wordpress:juju-info which is the topology of application graph. The ':'
isn't denotes the edge between two charms? If yes, why - wordpress:db and -
mariadb:db?
There us someone who knows the annotation method of relations? Could you
explain?

BR,
Ilias




On Tue, Jun 5, 2018 at 2:11 PM Ilias Tsoumas <itsoum at unipi.gr> wrote:

> Hi Merlijn,
>
> First of all thank you for your quick and essential answer!
>
> Reply inline
>
> Great to hear you're doing cool stuff with Juju! I recently asked a
>> similar question for a paper I've written for CLOUD 2018
>> https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
>> want, I can send it to you in private (it's accepted but not published yet).
>>
>
> Of course. I would like to read your paper. Also me and a colleague work
> on performance estimations of app components (=charms). If I have suspected
> properly your paper is related with performance analysis/profiling thus it
> will be inspiration for us.
>
> Here is the documentation for the charm store api:
>> https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
>>
>
> I will check it.
>
> Here is an Jupyter notebook I use to get metrics of charm and layer usage:
>> https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
>> It should be easy to modify that code to pull info about the bundles. As an
>> example; this is the URL to get all bundles in the Juju store:
>> https://api.jujucharms.com/charmstore/v5/list?type=bundle
>
>
> Thank you! I hope to be feasible to pull all bundle.yaml with one
> query/request.
>
> BR,
> Ilias
>
> On Mon, Jun 4, 2018 at 3:22 PM Merlijn Sebrechts <
> merlijn.sebrechts at gmail.com> wrote:
>
>> Hi Ilias
>>
>>
>> Great to hear you're doing cool stuff with Juju! I recently asked a
>> similar question for a paper I've written for CLOUD 2018
>> https://lists.ubuntu.com/archives/juju/2018-January/009845.html. If you
>> want, I can send it to you in private (it's accepted but not published yet).
>>
>> Here is the documentation for the charm store api:
>> https://github.com/juju/charmstore/blob/v5-unstable/docs/API.md
>>
>>    - download the entire zip for a bundle:
>>    https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive
>>    - download a specific file of a bundle:
>>    https://api.jujucharms.com/charmstore/v5/bundle/<bundle-name>/archive/bundle.yaml
>>
>> Here is an Jupyter notebook I use to get metrics of charm and layer
>> usage:
>> https://github.com/IBCNServices/reactive-pattern-results/blob/master/reactive-usage-stats.ipynb.
>> It should be easy to modify that code to pull info about the bundles. As an
>> example; this is the URL to get all bundles in the Juju store:
>> https://api.jujucharms.com/charmstore/v5/list?type=bundle
>>
>> Cleaning up the data will be the hard part probably, since there are a
>> lot of unused/broken bundles in the store. I've used the downloads/month
>> metric to figure out what charms are actually still used.
>>
>> Op ma 4 jun. 2018 om 14:05 schreef Ilias Tsoumas <itsoum at unipi.gr>:
>>
>>> Hello all,
>>>
>>> I have designed and I am developing a novel recommender system which
>>> will provide recommendations at application graph (=bundle) composition
>>> time. Now I need a real graphs dataset. I think that all juju bundles of
>>> components is a good real dataset. Are there some how to aggregate all
>>> these "bundle.yaml" files? Are there some api endpoint?
>>> This work is funding from the European Union’s Horizon 2020 research and
>>> innovation program under grant agreement No 761898 project Matilda.
>>>
>>> Kind regards,
>>> Ilias Tsoumas
>>> --
>>> Juju mailing list
>>> Juju at lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20180626/54903f9b/attachment.html>