Deploying Juju charms from Github

Robert Day Robert.Day at metaswitch.com
Wed Apr 15 09:51:44 UTC 2015


Hi Kapil,

I agree that not having juju-deployer handle the Git side of things, and just working locally from a Git checkout, would be simpler – but I don’t think that sort of bundle would work in the Juju GUI, because the extra steps needed mean that you can’t drag-and-drop it. Have I understood correctly?

On the syntax for bundle files, one thing I might want to do is take my five charms (all hosted in one Git repository) and bundle them together with several charms from someone else (which might all be hosted in the same repository). I can’t quite see how the syntax you suggest (one Git repository per bundle) would allow that. Something like:

> clearwater:
>  services:
>    "clearwater-bono":
>      charm: "clearwater-bono"
>      branch: "https://github.com/Metaswitch/clearwater-juju@46785ea8368a47c4351a516bb9a76763f6d4a952"
>      charms_subdirectory: "charms/precise/clearwater-bono"
>    "clearwater-sprout":
>      charm: "clearwater-sprout"
>      branch: "https://github.com/Metaswitch/clearwater-juju@46785ea8368a47c4351a516bb9a76763f6d4a952"
>      charms_subdirectory: "charms/precise/clearwater-sprout"
>    "somethingElse":
>      charm: "somethingElse"
>      branch: "https://github.com/someoneElse/somethingElse@abcde"
>      charms_subdirectory: "contrib/juju-charms"
Does that seem OK to you, or do you still prefer the single charm_repository option?

Thanks,
Rob

From: Kapil Thangavelu [mailto:kapilt at gmail.com]
Sent: 14 April 2015 13:26
To: Robert Day
Cc: juju at lists.ubuntu.com
Subject: Re: Deploying Juju charms from Github

On Tue, Apr 14, 2015 at 6:09 AM, Robert Day <Robert.Day at metaswitch.com<mailto:Robert.Day at metaswitch.com>> wrote:
Thanks for the response! I’ve:
* found https://code.launchpad.net/~whitmo/juju-deployer/git-clone-fix and applied those changes to my local copy of juju-deployer
* removed the “--depth 1” from vcs.py
* corrected my bundle to the below

>  services:
>    "clearwater-bono":
>      charm: "clearwater-bono"
>      branch: "https://github.com/Metaswitch/clearwater-juju@46785ea8368a47c4351a516bb9a76763f6d4a952"

I then get this error:

2015-04-14 09:33:21 Using deployment clearwater
2015-04-14 09:33:21 Starting deployment of clearwater
2015-04-14 09:33:21 Invalid config charm clearwater-bono zone=clearwater.local
2015-04-14 09:33:21 Deployment stopped. run time: 0.52



I’m pretty sure that this is because juju-deployer expects the charm’s metadata.yaml file (or config.yaml, etc.) to be in precise/clearwater-bono/metadata.yaml, whereas due to the way my Git repository is laid out, it’s actually in precise/clearwater-bono/charms/precise/clearwater-bono/metadata.yaml (the Git repository is checked out correctly to precise/clearwater-bono, but my charms are all in the charms/precise subdirectory of that repository – it’s at https://github.com/Metaswitch/clearwater-juju if you want to see what I mean).

specifically the error is referencing the key 'zone' is not found in the charms's config.yaml. if you have a single directory of charms in vcs, its a bit easier to not have deployer do vcs, ie. checkout, set JUJU_REPOSITORY env var to root of checkout dir, omit vcs info from the bundle config, and then just juju-deployer. Granted that has the downside of requiring additional instructions when sharing.


I think I could correct this by having one repository per charm, but I’d like to keep all my charms in the same repository if possible – there are six or seven of them and they’re quite closely related. It feels like having an option in the bundle to say “the charm files are in this subdirectory of the Git repository” would be useful – I might try and put that patch together, unless there are reasons why that wouldn’t be a good idea (or unless something similar already exists). Something like:

> clearwater:
>  services:
>    "clearwater-bono":
>      charm: "clearwater-bono"
>      branch: "https://github.com/Metaswitch/clearwater-juju@46785ea8368a47c4351a516bb9a76763f6d4a952"
>      charms_subdirectory: "charms/precise/clearwater-bono"
>    "clearwater-sprout":
>      charm: "clearwater-sprout"
>      branch: "https://github.com/Metaswitch/clearwater-juju@46785ea8368a47c4351a516bb9a76763f6d4a952"
>      charms_subdirectory: "charms/precise/clearwater-sprout"

actually i think a different syntax would be nice.

clearwater:
   charm_repository: https://github.com/Metaswitch/clearwater@987132asdf
   services:
      charm: clearwater-bono

that repo directory would get checked out, and set as the JUJU_REPOSITORY for the rest of the services, ie. propagate for all local charms automatically, but they could override as needed.  patches welcome.

cheers,

Kapil


Thanks,
Rob


From: Kapil Thangavelu [mailto:kapilt at gmail.com<mailto:kapilt at gmail.com>]
Sent: 14 April 2015 05:57
To: Robert Day
Cc: juju at lists.ubuntu.com<mailto:juju at lists.ubuntu.com>
Subject: Re: Deploying Juju charms from Github


There's a few things going on. The charm key in the bundle should just be the charm name. The git support has a bug (there's a pending merge proposal for this) re parent dir/checkout dir which is the cause of those tracebacks. The @ syntax works for revision and tags but not branches. Given its common for branch co-location with git unlike bzr that seems reasonable, the @rev specification will likely need some additional syntax to support branches. Another issue is deployer is optimizing for the deployment case so its taking shallow copies of repos (--depth 1 on git clone). With git that seems to preclude fetching remote branch info. The shallow clone is probably of marginal utility given git's speed, with bzr its a significant speed up. Its going to take a few days to sort out changes to support this.

-kapil


On Mon, Apr 13, 2015 at 11:11 AM, Robert Day <Robert.Day at metaswitch.com<mailto:Robert.Day at metaswitch.com>> wrote:
Hi all,

I'm trying to deploy a Juju bundle and set of charms hosted on Github rather than Launchpad, but I'm running into some problems. Although juju-deployer does appear to support this, I haven't found many examples, so I'm not certain I'm doing the right thing - if anyone has this working, and could point me at their Github repository, that'd be great.
.

I'm using juju-deployer version 0.4.3-0ubuntu1~ubuntu12.04.1~ppa1, juju-core version 1.22.0-0ubuntu1~12.04.2~juju1 and Git version 2.3.3. The bundle I'm using is at https://raw.githubusercontent.com/Metaswitch/clearwater-juju/github_everywhere/charms/bundles/clearwater/bundle/bundles.yaml. The Github repository/branch is https://github.com/Metaswitch/clearwater-juju/, which contains several charms and the bundle.

What I want is:
- to check out the dnsaas branch of https://github.com/Metaswitch/clearwater-juju.git
- to deploy the charm in the subdirectory 'charms/precise/clearwater-bono'

so I have the following in my bundle:

>   services:
>     "clearwater-bono":
>       charm: "charms/precise/clearwater-bono"
>       branch: "https://github.com/Metaswitch/clearwater-juju@dnsaas"

When I run juju-deployer, it fails with "No such file or directory" when trying to deploy the charm at https://github.com/Metaswitch/clearwater-juju/tree/dnsaas/charms/precise/clearwater-bono (though there's nothing special about this charm, it's just the first one in the bundle):

> $ juju-deployer -c https://raw.githubusercontent.com/Metaswitch/clearwater-juju/github_everywhere/charms/bundles/clearwater/bundle/bundles.yaml
> 2015-04-13 13:50:20 Using deployment clearwater
> 2015-04-13 13:50:20 Starting deployment of clearwater
> Traceback (most recent call last):
>   File "/usr/bin/juju-deployer", line 9, in <module>
>     load_entry_point('juju-deployer==0.4.3', 'console_scripts', 'juju-deployer')()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/cli.py", line 130, in main
>     run()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/cli.py", line 228, in run
>     importer.Importer(env, deployment, options).run()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/action/importer.py", line 188, in run
>     self.get_charms()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/action/importer.py", line 63, in get_charms
>     no_local_mods=self.options.no_local_mods)
>   File "/usr/local/lib/python2.7/dist-packages/deployer/deployment.py", line 139, in fetch_charms
>     os.mkdir(charm.series_path)
> OSError: [Errno 2] No such file or directory: 'precise/charms/precise'

If I create that directory (with mkdir -p precise/charms/precise) I get a different error:

> $ juju-deployer -c https://raw.githubusercontent.com/Metaswitch/clearwater-juju/github_everywhere/charms/bundles/clearwater/bundle/bundles.yaml
> 2015-04-13 13:50:58 Using deployment clearwater
> 2015-04-13 13:50:58 Starting deployment of clearwater
> Traceback (most recent call last):
>   File "/usr/bin/juju-deployer", line 9, in <module>
>     load_entry_point('juju-deployer==0.4.3', 'console_scripts', 'juju-deployer')()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/cli.py", line 130, in main
>     run()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/cli.py", line 228, in run
>     importer.Importer(env, deployment, options).run()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/action/importer.py", line 188, in run
>     self.get_charms()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/action/importer.py", line 63, in get_charms
>     no_local_mods=self.options.no_local_mods)
>   File "/usr/local/lib/python2.7/dist-packages/deployer/deployment.py", line 140, in fetch_charms
>     charm.fetch()
>   File "/usr/local/lib/python2.7/dist-packages/deployer/charm.py", line 132, in fetch
>     self.vcs.update(self.rev)
>   File "/usr/local/lib/python2.7/dist-packages/deployer/vcs.py", line 99, in update
>     self._call(params, self.err_update)
>   File "/usr/local/lib/python2.7/dist-packages/deployer/vcs.py", line 30, in _call
>     args, cwd=cwd or self.path, stderr=stderr)
>   File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
>     process = Popen(stdout=PIPE, *popenargs, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
>     errread, errwrite)
>   File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
>     raise child_exception
> OSError: [Errno 2] No such file or directory: 'precise/charms/precise/clearwater-bono'

It looks like something odd has happened with the paths - juju-deployer is looking in 'precise/charms/precise/clearwater-bono' but my charm actually seems to have been checked out to 'precise/charms/precise/precise/charms/precise/clearwater-bono/charms/precise/clearwater-bono/'.

> $ ls precise/charms/precise/precise/charms/precise/clearwater-bono/charms/precise/clearwater-bono/
> config.yaml  copyright  hooks  icon.svg  lib  metadata.yaml  README.md  revision

I get the same error with juju-quickstart - a notification in the juju-gui tells me:

> Updated status for deployment id: 0
>
> An error occurred while deploying the bundle: [Errno 2] No such file or directory: 'precise/charms/precise'
> less than a minute ago

I tried creating a new charm repository holding only a single charm (as opposed to my current model, where six or seven closely related charms are all in the same repository), but hit similar errors - I didn't pursue this any further, because it's much easier for me to keep all these charms in the same repository.

Thanks,
Rob

--
Juju mailing list
Juju at lists.ubuntu.com<mailto:Juju at lists.ubuntu.com>
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20150415/ead371db/attachment.html>


More information about the Juju mailing list