Juju Compose

[Cory Johns]

On Tue, Jul 14, 2015 at 4:45 PM, Marco Ceppi <marco.ceppi at canonical.com> wrote:
> I think you're on the right train of thought as to what this is, but miss a
> few important issues this helps address. It's is trivial to copy code
> around, and we talk about it all the time in charm schools "find a charm
> similar, fork, edit". What I think compose is helping solve, and correct me
> if I'm wrong Ben & co, is that those operations are a one time event, where
> compose is an operation that can be done multiple times.
>
> You fork the tomcat charm, you munge in your bits to deploy your tomcat
> application and now you have a charm. That's it. If the maintainer of the
> tomcat charm updates that charm, there's no way you - who now has old code,
> that possibly could contain bugs or security issues - would know to update
> it, or how to even how to update it. By having layers of charms - in the
> sense of a tomcat layer, then your bits in another layer, that get composed
> to create one final charm it would address those issues. Now we have a
> mechanism to not only know how and what files were copied from where, we
> also have the ability to potentially notify users when a base is updated
> (prompting them to re-compose their charm and test it).

I don't think this fully highlights why compose is different from, and
I think better than (for the use-case for which it is intended),
existing VCS solutions.

> With this approach we no longer need to shoe-horning huge complex ecosystems
> to use the "framework charms" concept which is really just an over-extension
> of the subordinate charm feature in Juju.Instead, we could build framework
> layers (tomcat, docker, django, wordpress, magento, hadoop) which themselves
> could be charms or just fragments of charms. Using compose users could start
> to mix and match so that you could theoretically combine a tomcat layer with
> a docker layer and other layers then add your own custom layer on top and
> now you've got a charm which you can update and manage when layers are
> updated.
>
> Another thing that this approach opens up to help solve, and it's been a
> problem for Juju since essentially day one, is relationship management.
> Potentially, instead of having to read other code to figure out how to
> reproduce a relation there could just be interface layers that model the
> relation exchange and allow you to write your layer to just respond to those
> events. While this seems complex, and would be a bit down the road, this
> pattern starts to solidify a whole new way to create, manage, and maintain
> charms.
>
> At the end of the day, this is just as you said. Detached version control of
> files being moved around, and we can do that already. This legitimizes that
> and makes it so that managing that code going forward isn't so tedious,
> opening up new avenues for charm authors when it comes to creating charms.

This is what it's really about, but I want to clarify exactly why we
want to do this instead of VCS forking.

Let's say you take an existing charm and fork it using your favorite
VCS, and then make some changes to extend it.  You now have a complete
copy of the original charm plus your changes.  The VCS can then help
you track and merge in upstream changes, and they do this quite well,
so this is obviously not what we're trying to accomplish, though it is
of course necessary.

The key point is that your fork is a *complete* copy of the original
charm.  While this is exactly what you want for deploying, what you're
losing (or at least what is much harder to see on its own) is what
*you* changed.  It is much more difficult to see just the things that
you have added or removed from the charm.  This makes it harder to
review the differences, and in many cases it makes it harder to
understand.

What compose is all about is allowing charm authors to maintain *just*
the layer that is different from the base.  Just their logic.  And the
base can itself be made up of multiple layers, each building on the
one below it.  Yet at the end of the day, you can tell the composer to
put all those layers together into a single, deployable, auditable,
complete charm.

Another area where VCS falls down is bringing together *multiple*
separate sources into a single resulting charm.  With compose, charm
layers become building blocks for building more complex charms.  And,
as Marco pointed out, relations can be decomposed into layers that
allow us to reuse the code so that each charm is not reimplementing
the logic (possibly incompletely).

Of course, some of this can be accomplished using libraries, but then
the end-result charm is not complete.  Library / dependency management
becomes a significant issue, as it has been for some time with charms.
And browsing the charm on jujucharms.com is more difficult because it
is harder to follow the logic into external libraries (or libraries
bundled in as compressed archives, etc.).

Compose gives us a very descriptive way to say, "These are all the
common, shared bits that make up my charm.  And these are the bits
that are unique to my charm," while still being able to turn that into
a complete, deployable charm.  I think that is a completely new
ability that will make working with charms much nicer going forward.