Topology and Orchestration Specification for Cloud Applications (TOSCA) and Juju

[Clint Byrum]

Excerpts from Johannes Wettinger's message of 2012-08-23 08:56:48 -0700:
> Hi Gustavo,
> 
> thanks for your reply.
> 
> On Thu, Aug 23, 2012 at 10:30 AM, Gustavo Niemeyer
> <gustavo.niemeyer at canonical.com> wrote:
> >> I'm currently doing some research and prototype implementation in the
> >> field of Cloud standards. In particular, my work is strongly related
> >> to an upcoming standard called "Topology and Orchestration
> >> Specification for Cloud Applications" (TOSCA):
> >> http://www.tosca-open.org
> >
> > Nice, thanks for reaching out. What's your involvement there, out of curiosity?
> 
> Currently, I'm doing my Master's thesis. The topic of my thesis is
> about integrating DevOps methodologies into model-driven Cloud
> management. TOSCA defines itself as being a means to enable
> model-driven management of Cloud application whereas Juju is more
> focused on the concerns of the DevOps movement. So, TOSCA and Juju
> have a different background, but in the end they own similar meta
> models. This is why I think integrating the two of them could make
> sense.
> 

You're seeking to bring a development methodology (modeling) to
operations. Seeing as DevOps is a movement to bring development and
operations closer, it would seem that you could also say "TOSCA is
focused on the concerns of the DevOps movement".

> >> Of course, TOSCA is much more complex
> >> because of its highly generic approach to describe service models.
> >> Just wondering if anyone here is familiar with TOSCA, too?
> >
> > I'd argue that the complexity doesn't come from the generality.
> 
> What do you think is the reason for the complexity of TOSCA's meta
> model? As an example, let's take a look at the structure of node
> types:
> 
> - NodeType "MySQL"
>    - Interface "Lifecycle"
>       - Operation "install"
>          - ImplementationArtifact (type = "ScriptArtifact for Windows")
>          - ImplementationArtifact (type = "ScriptArtifact for Linux")
>       - Operation "configure"
>       - Operation ...
>    - Interface "Maintenance"
>       - Operation "backup-database"
>       - ...
> 
> First, there can be several implementation artifacts assigned to a
> single operation, e.g. offering installation scripts for Windows,
> Linux, etc. This makes a node type portable. How would you implement
> this in Juju? You may have to implement two separate charms such as
> "mysql-server-windows" and "mysql-server-linux", right? Thus, the
> portability of a single charm is quite limited or did I miss
> something?
> 

Charms are tied to a "series". Each series is developed and
tested together as a whole to ensure that the various charms work
together. Currently we are targetting releases of Ubuntu with each
series of charms because we have not been presented with a compelling
reason to do otherwise. (actually I haven't even been presented with
any non-compelling reasons. I'm sure users are interested, but nobody
is campaigning for other platforms.)

However, there is no reason we could not create a "portable" series of
juju charms which do offer conditional logic based on platform. Because
charms are only declarative at a very high level, they are quite flexible.

> Second, you can define arbitrary interfaces and operations in a node
> type beside the default lifecycle operations (install, start, stop,
> etc.). Hooks in Juju are basically limited to those lifecycle
> operations. But what if you would like to implement maintenance
> operations such as doing a backup of the database?
> 

https://bugs.launchpad.net/juju/+bug/809070

It has been on the list for a while. There is a simple workaround which
is to do arbitrary operations in the config-changed hook.

> I think most of TOSCA's complexity is there because of its goal to be
> general and highly portable.
> 

Agreed, the more aspects you add, the more complex it will become,
as that is basically the definition of complexity.

> >> What is your opinion of bringing these two worlds (TOSCA and Juju)
> >> together? I mean wouldn't it make a lot of sense to support TOSCA
> >> service templates in Juju? As an example, node types in TOSCA are
> >> basically very similar to Juju's charms and lifecycle operations
> >> including their implementation artifacts in TOSCA are quite similar to
> >> hooks in Juju. :-)
> >
> > What an amazing coincidence. :-)
> >
> > I'd love to see people collaborating, but considering that there's a
> > committee writing down a 40+ page specification that arbitrates how a
> > system that looks a lot like juju might work in theory, I don't see a
> > good path to do that. We can't simply stop producing working software
> > to make it speak a language that nothing else does.
> 
> I completely agree! The goal cannot be to stop the development of
> Juju. ;-) But wouldn't it make sense for Juju to be able to import
> TOSCA node types as charms? Another use case could be to load a TOSCA
> topology template as "Juju environment snapshot" or export a running
> Juju environment as TOSCA topology template.
> 
> Of course there have to be made some assumptions regarding the TOSCA
> node types and topology templates that are supported by Juju (e.g.,
> only default lifecycle operations are supported). But I think this
> would be completely valid, because a TOSCA engine doesn't have to
> support any arbitrary node type. As a result, Juju would become a
> TOSCA engine (supporting several TOSCA features) without changing its
> internal meta model.

It does seem like our two groups will do well to stay in contact with
one another, as we'll probably both have great ideas for how multi-host
cloud systems can be managed. Just like we have charms which use existing
puppet modules to implement system configuration, I could see us have
charms that effectively just import a TOSCA node description and run the
actions defined within as part of the hooks of the charm. That might be
an interesting exercise as the library of TOSCA node descriptions grows.