Multiple package installs with subordinates?

[Clint Byrum]

Excerpts from Thomas Leonard's message of 2012-12-04 01:20:39 -0800:
> On 2012-12-03 18:43, Gustavo Niemeyer wrote:
> > On Mon, Dec 3, 2012 at 3:42 PM, Bruno Girin <brunogirin at gmail.com> wrote:
> >> Installed via package manager (whether that be apt, yum, etc),
> >> Via download from know URL + install,
> >> Via VCS (git, bzr, etc),
> >> Via language specific installers: pip, gems, etc.
> >> And of course the ability to define new installers.
> >
> > Interestingly, we already support all of these options today, and the
> > syntax to support all of them is well-known and non-special. It
> > supports variations at runtime, such as which release of Ubuntu is
> > being run, or even which distribution, or how the software is
> > configured etc. There are also issues like upgrades to consider.
> >
> > This seems like one of these cases where less is more.
> 
> How should we support multiple units, though? If the charm just does 'git 
> clone ...' in its install hook, then the second unit may get a different 
> version of the software.
> 
> On the other hand, doing this on the client is hard, because the client 
> machine might not be the same OS/version as the unit's machine.
> 
> So maybe the first unit of a service should get the software and then upload 
> it to S3 for the other units?
> 

A long time ago we discussed another program for charms to make use of
called 'file-get'. It would simply cache URLs in the object storage for
the environment. It would use a key something like

${hash_of_url}-${charm_namespace_and_rev}

So on first unit deploy you'd say

file-get https://github.com/fooproject/download/baz-3.4.4.tar.gz

This would look in S3/swift/etc for:

feed046474e2f6e8e550f70e1e49d1dd-mycharm:4

And if not there, download it, and then push it back up to there.

For things that can't be simply fetched:

if ! [ -d fooproject ] ; then
    key=github-fooproject
    path=`file-get $key`
    if [ -z "$path" ] ; then
        git clone https://github.com/fooproject/fooproject.git fooproject
        tar zcf fooproject.tar.gz fooproject
        file-put $key fooproject.tar.gz
    else
        tar zxf $path
    fi
fi

If we just do the bits possible today, its quite simple:

[ -d fooproject] || git clone ...

But when we start to try and solve this problem (which exists in basically
any charm that does not encode stable releases or use archive-only
packages) we'll have to address it in *every* charm.

Thats where a charm helping tool will help to hide the implementation
details of juju and also charm policy from the charms... as at some point
juju may grow git URL support, but until it does, users can write parts
of charms like this:


packages: [ 'libbaz1' ]
vcs:
    fooproject:
        type: git
        url: https://github.com/fooproject/fooproject.git
        dir: fooproject

And then as git support and vcs support improve in juju or the charm
helper, so do the charms.