Juju Compose

Tue Jul 14 20:45:55 UTC 2015

Hi Ben,

I've spent some time looking through this and I think it's a great start! I
know we've talked on and off about an idea like this in the past so it's
great to see some public code around it.

On Tue, Jul 14, 2015 at 3:54 PM Nate Finch <nate.finch at canonical.com> wrote:

> I'm not sure I understand the problems this is solving that aren't
> adequately solved by branching from a source charm.  It seems like the hard
> part of branching - maintaining changes to non-trivial code - is not helped
> by this tool.  For charms that just need to add hook files or modify charm
> metadata, this seems fine, but those don't seem like the difficult parts of
> branching from a source charm either.  I guess my point is, this seems like
> it's doing a lot of what version control already does.
>

I think you're on the right train of thought as to what this is, but miss a
few important issues this helps address. It's is trivial to copy code
around, and we talk about it all the time in charm schools "find a charm
similar, fork, edit". What I think compose is helping solve, and correct me
if I'm wrong Ben & co, is that those operations are a one time event, where
compose is an operation that can be done multiple times.

You fork the tomcat charm, you munge in your bits to deploy your tomcat
application and now you have a charm. That's it. If the maintainer of the
tomcat charm updates that charm, there's no way you - who now has old code,
that possibly could contain bugs or security issues - would know to update
it, or how to even how to update it. By having layers of charms - in the
sense of a tomcat layer, then your bits in another layer, that get composed
to create one final charm it would address those issues. Now we have a
mechanism to not only know how and what files were copied from where, we
also have the ability to potentially notify users when a base is updated
(prompting them to re-compose their charm and test it).

With this approach we no longer need to shoe-horning huge
complex ecosystems to use the "framework charms" concept which is really
just an over-extension of the subordinate charm feature in Juju.Instead, we
could build framework layers (tomcat, docker, django, wordpress, magento,
hadoop) which themselves could be charms or just fragments of charms. Using
compose users could start to mix and match so that you could theoretically
combine a tomcat layer with a docker layer and other layers then add your
own custom layer on top and now you've got a charm which you can update and
manage when layers are updated.

Another thing that this approach opens up to help solve, and it's been a
problem for Juju since essentially day one, is relationship management.
Potentially, instead of having to read other code to figure out how to
reproduce a relation there could just be interface layers that model the
relation exchange and allow you to write your layer to just respond to
those events. While this seems complex, and would be a bit down the road,
this pattern starts to solidify a whole new way to create, manage, and
maintain charms.

At the end of the day, this is just as you said. Detached version control
of files being moved around, and we can do that already. This legitimizes
that and makes it so that managing that code going forward isn't so
tedious, opening up new avenues for charm authors when it comes to creating
charms.

Marco

>
> On Fri, Jul 10, 2015 at 2:22 PM Benjamin Saller <
> benjamin.saller at canonical.com> wrote:
>
>> Charm Composition is a small tool facilitating a new pattern for
>> developing and maintain Juju Charms. In its simplest form it lets you
>> combine directories, called Layers, of files to create a charm. Rather than
>> just copying files around however it keeps track of which files came from
>> which Layer so that at a later time the process can be run again. This
>> means that if any of the Layers have changed those bits will be reflected
>> in the newly regenerated charm.
>>
>> Imaging that you have a charm you’ve forked to add some simple tweak or
>> customization. You might be able to maintain your changes in a new Layer
>> allowing the base you depend on to continue to evolve.
>>
>> Suppose we have layer ‘a’.
>>
>> trusty/a/
>>
>> ├── hooks/install
>>
>> ├── metadata.yaml
>>
>> └── README.md
>>
>> And a layer ‘b’
>>
>> trusty/b/
>>
>> ├── hooks/db-relation-changed
>>
>> ├── composer.yaml
>>
>> ├── metadata.yaml
>>
>> └── README.md
>>
>> If ‘b’ included ‘a’ (specified in the composer.yaml file) then composer
>> might combine the layers in the following way.
>>
>> trusty/b/
>>
>> ├── hooks/install
>>
>> ├── hooks/db-relation-changed
>>
>> ├── composer.yaml
>>
>> ├── metadata.yaml
>>
>> └── README.md
>>
>> That is just a ‘cp -r’ you’re saying, and it would be except the compose
>> process is able to support a variety of methods for combining layers. By
>> default compose knows about Charm specific files like metadata.yaml and
>> config.yaml it is able to combine them in sensible ways. For files like
>> this the default is a merge with the ability to remove add/replace keys
>> within the data of the file in the lower layer (in this case ‘a’ would be
>> overridden if it had a key in common with ‘b’ when composed).
>>
>> Composer itself is extensible so if you’re layers need special rules
>> associated with how certain types of files should be combined there is an
>> interface for doing that.
>>
>> Composer has code for doing much more interesting things than just
>> combining layers, but we can save that for another post. If you’re
>> interested in this now and want to start thinking about what nice base
>> layers might look like, check it out at
>> https://github.com/bcsaller/juju-compose
>>
>>
>>
>> Thanks,
>> Ben
>>
>> There is more that should be said about how this relates to the work with
>> Relation Stubs and so on but that can be another thread.
>> --
>> Juju mailing list
>> Juju at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
> --
> Juju mailing list
> Juju at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20150714/a71e4b8b/attachment.html>