Charm series in bundles (Was Re: Juju 2.0 and local charm deployment)

Antonio Rosales antonio.rosales at canonical.com
Tue Mar 29 01:04:09 UTC 2016 

+ Juju list for others awareness


On Thu, Mar 10, 2016 at 1:53 PM, Ian Booth <ian.booth at canonical.com> wrote:
> Thanks Rick. Trivial change to make. This work should be in beta3 due next week.
> The work includes dropping support for local repositories in favour of path
> based local charm and bundle deployment.

Ian,
First thanks for working on this feature. Second, I tried this for a
local ppc64el deploy which is behind a firewall, and thus local charms
are good way forward. I may have got the syntax incorrect and thus
wanted to confirm here. What I did was is at:
http://paste.ubuntu.com/15547725/
Specifically, I set the the charm path to something like:
    charm: /home/ubuntu/charms/trusty/apache-hadoop-compute-slave
However, I got the following error:
ERROR cannot deploy bundle: cannot resolve URL
"/home/ubuntu/charms/trusty/apache-hadoop-compute-slave": charm or
bundle URL has invalid form:
"/home/ubuntu/charms/trusty/apache-hadoop-compute-slave"

This is on the latest beta3:
2.0-beta3-xenial-ppc64el

Any suggestions?

-thanks,
Antonio


>
> On 10/03/16 23:37, Rick Harding wrote:
>> Thanks Ian, after thinking about it I think what we want to do is really
>> #2. The reasoning I think is:
>>
>> 1) we want to make things consistent. The CLI experience is present a charm
>> and override series with --series=
>> 2) more consistent, if you do it with local charms you can always do it
>> 3) we want to encourage folks to drop series from the charmstore urls and
>> worry less about series over time. Just deploy X and let the charm author
>> pick the default best series. I think we should encourage this in the error
>> message for #2. "Please remove the series section of the charm url" or the
>> like when we error on the conflict, pushing users to use the series
>> override.
>>
>> Uros, Francesco, this brings up a point that I think for multi-series
>> charms we want the deploy cli snippets to start to drop the series part of
>> the url as often as we can. If the url doesn't have the series specified,
>> e.g. jujucharms.com/mysql then the cli command should not either. Right now
>> I know we add the series/revision info and such. Over time we want to try
>> to get to as simple a command as possible.
>>
>> On Thu, Mar 10, 2016 at 7:23 AM Ian Booth <ian.booth at canonical.com> wrote:
>>
>>> I've implemented option 1:
>>>
>>>  error if Series attribute is used at all with a store charm URL
>>>
>>> Trivial to change if needed.
>>>
>>> On 10/03/16 12:58, Ian Booth wrote:
>>>> Yeah, agreed having 2 ways to specify store series can be suboptimal.
>>>> So we have 2 choices:
>>>>
>>>> 1. error if Series attribute is used at all with a store charm URL
>>>> 2. error if the Series attribute is used and conflicts
>>>>
>>>> Case 1
>>>> ------
>>>>
>>>> Errors:
>>>>
>>>> Series: trusty
>>>> Charm: cs:mysql
>>>>
>>>> Series: trusty
>>>> Charm: cs:trusty/mysql
>>>>
>>>> Ok:
>>>>
>>>> Series: trusty
>>>> Charm: ./mysql
>>>>
>>>>
>>>> Case 2
>>>> ------
>>>>
>>>> Ok:
>>>>
>>>> Series: trusty
>>>> Charm: cs:mysql
>>>>
>>>> Series: trusty
>>>> Charm: cs:trusty/mysql
>>>>
>>>> Series: trusty
>>>> Charm: ./mysql
>>>>
>>>> Errors:
>>>>
>>>> Series: xenial
>>>> Charm: cs:trusty/mysql
>>>>
>>>>
>>>> On 10/03/16 12:51, Rick Harding wrote:
>>>>> Bah maybe you're right. I want to sleep on it. It's kind of ugh either
>>> way.
>>>>>
>>>>> On Wed, Mar 9, 2016, 9:50 PM Rick Harding <rick.harding at canonical.com>
>>>>> wrote:
>>>>>
>>>>>> I think there's already rules for charmstore charms. it uses the
>>> default
>>>>>> if not specified. I totally agree that for local charms we have to have
>>>>>> this. For remote charms though this is providing the user two ways to
>>> do
>>>>>> the same thing
>>>>>>
>>>>>> On Wed, Mar 9, 2016, 9:46 PM Ian Booth <ian.booth at canonical.com>
>>> wrote:
>>>>>>
>>>>>>> If the charm store charm defines a series in the URL, then we will
>>>>>>> consider it
>>>>>>> an error to specify a different series using the attribute. But charm
>>>>>>> store URLs
>>>>>>> are not required to have a series, so we can use the attribute in that
>>>>>>> case. It
>>>>>>> also allows users to easily switch between store and local charms
>>> during
>>>>>>> development just by replacing "./" with "cs:"
>>>>>>>
>>>>>>>      nova-compute:
>>>>>>>        series: xenial
>>>>>>>        charm: ./nova-compute
>>>>>>>
>>>>>>>      nova-compute:
>>>>>>>        series: xenial
>>>>>>>        charm: cs:nova-compute
>>>>>>>
>>>>>>>
>>>>>>> On 10/03/16 12:21, Rick Harding wrote:
>>>>>>>> I'm not sure we want to make this attribute apply to charmstore
>>> charms.
>>>>>>>> We've an established practice of the charmstore url being the series
>>>>>>>> information. It gives the user a chance to have conflicting
>>> information
>>>>>>> if
>>>>>>>> the charmstore url is cs:trusty/nova-compute and the series
>>> attribute is
>>>>>>>> set to xenial. I think we should toss an error to a bundle that has
>>>>>>> series:
>>>>>>>> specified for a charmstore based charm value (or non-local value
>>>>>>> whichever
>>>>>>>> way you want to think about it)
>>>>>>>>
>>>>>>>> On Wed, Mar 9, 2016 at 6:29 PM Ian Booth <ian.booth at canonical.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> One additional enhancement we need for bundles concerns specifying
>>>>>>> series
>>>>>>>>> for
>>>>>>>>> multi-series charms, in particular local charms now that the local
>>> repo
>>>>>>>>> will be
>>>>>>>>> going away.
>>>>>>>>>
>>>>>>>>> Consider:
>>>>>>>>>
>>>>>>>>> A new multi-series charm may have a URL which does not specify the
>>>>>>> series.
>>>>>>>>> In
>>>>>>>>> that case, the series used will be the default specified in the
>>> charm
>>>>>>>>> metadata
>>>>>>>>> or the latest LTS. But we want to allow people to choose their own
>>>>>>> series
>>>>>>>>> also.
>>>>>>>>>
>>>>>>>>> So we need a new (optional) Series attribute in the bundle metadata.
>>>>>>>>>
>>>>>>>>> bundle.yaml
>>>>>>>>>   series: trusty
>>>>>>>>>   services:
>>>>>>>>>     nova-compute:
>>>>>>>>>       series: xenial <------ new
>>>>>>>>>       charm: ./nova-compute
>>>>>>>>>       num_units: 2
>>>>>>>>>
>>>>>>>>> or with a charm store charm
>>>>>>>>>
>>>>>>>>> bundle.yaml
>>>>>>>>>   series: trusty
>>>>>>>>>   services:
>>>>>>>>>     nova-compute:
>>>>>>>>>       series: xenial    <------ new
>>>>>>>>>       charm: cs:nova-compute
>>>>>>>>>       num_units: 2
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Note: the global series in the bundle still applies if series is not
>>>>>>>>> otherwise
>>>>>>>>> known.
>>>>>>>>> The new series attribute is per charm.
>>>>>>>>>
>>>>>>>>> So in the case above, cs:nova-compute may ordinarily be deployed on
>>>>>>> trusty
>>>>>>>>> (the
>>>>>>>>> default series in that charm's metadata). But the bundle requires
>>> the
>>>>>>>>> xenial
>>>>>>>>> version. With the charm store URL, we can currently use
>>>>>>>>> cs:xenial/nova-compute
>>>>>>>>> but that's not the case for local charms deployed out of a
>>> directory.
>>>>>>> We
>>>>>>>>> need a
>>>>>>>>> way to allow the series to be specified in that latter case.
>>>>>>>>>
>>>>>>>>> We'll look to make the changes in core initially and can followup
>>> later
>>>>>>>>> with the
>>>>>>>>> GUI etc. The attribute is optional and only really affects bundles
>>> with
>>>>>>>>> local
>>>>>>>>> charms.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 09/03/16 09:53, Ian Booth wrote:
>>>>>>>>>> So to clarify what we'll do. We'll support the same syntax in
>>> bundle
>>>>>>>>> files as we
>>>>>>>>>> do for deploy.
>>>>>>>>>>
>>>>>>>>>> Deploys charm store charms:
>>>>>>>>>>
>>>>>>>>>> $ juju deploy cs:wordpress
>>>>>>>>>> $ juju deploy wordpress
>>>>>>>>>>
>>>>>>>>>> Deploys a local charm from a directory:
>>>>>>>>>>
>>>>>>>>>> $ juju deploy ./charms/wordpress
>>>>>>>>>> $ juju deploy ./wordpress
>>>>>>>>>>
>>>>>>>>>> So below deploys a local nova-compute charm in a directory
>>> co-located
>>>>>>>>> with the
>>>>>>>>>> bundle.yaml file.
>>>>>>>>>>
>>>>>>>>>>  series: trusty
>>>>>>>>>>  services:
>>>>>>>>>>    nova-compute:
>>>>>>>>>>      charm: ./nova-compute
>>>>>>>>>>      num_units: 2
>>>>>>>>>>
>>>>>>>>>> This one deploys a charm store charm:
>>>>>>>>>>
>>>>>>>>>>  series: trusty
>>>>>>>>>>  services:
>>>>>>>>>>    nova-compute:
>>>>>>>>>>    charm: nova-compute
>>>>>>>>>>    num_units: 2
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 09/03/16 03:59, Rick Harding wrote:
>>>>>>>>>>> Long term we want to have a pattern when the bundle is a directory
>>>>>>> with
>>>>>>>>>>> local charms in a directory next to the bundles.yaml file. We
>>> could
>>>>>>> not
>>>>>>>>> do
>>>>>>>>>>> this cleanly before the multi-series charms that are just getting
>>> out
>>>>>>>>> the
>>>>>>>>>>> door. I think that bundles with local charms will be suboptimal
>>>>>>> until we
>>>>>>>>>>> can get those bits to line up.
>>>>>>>>>>>
>>>>>>>>>>> I don't think we want to be doing the file based urls, but to
>>> build a
>>>>>>>>>>> pattern that's reusable and makes sense across systems. Creating a
>>>>>>>>> standard
>>>>>>>>>>> pattern I think is the best path forward.
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 8, 2016 at 12:26 PM Martin Packman <
>>>>>>>>> martin.packman at canonical.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On 05/03/2016, Ian Booth <ian.booth at canonical.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> How will bundles work which reference local charms? Will this
>>>>>>> work as
>>>>>>>>>>>>>> expected where nova-compute is a directory at the same level
>>> as a
>>>>>>>>> bundle
>>>>>>>>>>>>>> file?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ```
>>>>>>>>>>>>>> series: trusty
>>>>>>>>>>>>>> services:
>>>>>>>>>>>>>>   nova-compute:
>>>>>>>>>>>>>>     charm: ./nova-compute
>>>>>>>>>>>>>>     num_units: 2
>>>>>>>>>>>>>> ```
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The above will work but not until a tweak is made to bundle
>>>>>>>>> deployment to
>>>>>>>>>>>>> interpret a path on disk rather than a url. It's a small change.
>>>>>>> This
>>>>>>>>>>>> would
>>>>>>>>>>>>> be done as part of the work to remove the local repo support.
>>>>>>>>>>>>
>>>>>>>>>>>> Can we keep interpreting the the reference in the bundle as a
>>> url,
>>>>>>> but
>>>>>>>>>>>> start supporting file urls? That seems neater than treating the
>>> cs:
>>>>>>>>>>>> prefix as magic not-a-filename.
>>>>>>>>>>>>
>>>>>>>>>>>> The catch is that there's no sane way of referencing locations
>>>>>>> outside
>>>>>>>>>>>> a base url.
>>>>>>>>>>>>
>>>>>>>>>>>>     charm: file:nova-compute
>>>>>>>>>>>>
>>>>>>>>>>>> Works as a reference to a dir inside the base location, but:
>>>>>>>>>>>>
>>>>>>>>>>>>     charm: file:../nova-compute
>>>>>>>>>>>>
>>>>>>>>>>>> Will not work as a reference to a sibling directory. And absolute
>>>>>>> file
>>>>>>>>>>>> paths are pretty useless across machines.
>>>>>>>>>>>>
>>>>>>>>>>>> Martin
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Juju-dev mailing list
>>>>>>>>>>>> Juju-dev at lists.ubuntu.com
>>>>>>>>>>>> Modify settings or unsubscribe at:
>>>>>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev



-- 
Antonio Rosales
Ecosystem Engineering
Canonical

More information about the Juju mailing list