Charm derivation

[Clint Byrum]

Excerpts from Gustavo Niemeyer's message of Tue Jan 17 06:44:41 -0800 2012:
> > The same 15 - 20 lines of shell over and over gave rise to a need to stop
> > cargo culting this and have a place for snippets of well tested, well
> > written shell script that can be shared between charms and improved upon
> > as a whole. Its easier to suggest to people that they use things like:
> >
> > ch_get_file "http://foo/bar.tgz" "aasdfasdfasdfasd8fasfd8as9d"
> 
> If you find this logic so important, lets have that in Ubuntu itself as a tool!
> 

Sometimes this makes sense, in others, its just about abstracting away
a task that is just common in charms themselves.

> Maybe even wget itself:
> 
>     wget --check-sha1 $SUM $URL
> 

Getting a change like that into wget is going to take time, though I agree
we can and should move to make all these charm helper abstractions simply
call the best upstream tool with the right arguments and deprecate them
recommending their use once such changes are upstream and available in
all supported releases.

> > These get cargo culted like mad, and sometimes we spot something that
> > needs fixing in one charm, but then have to go search all the other
> > charms for the same mistake.  So using a library for this makes sense.
> 
> I agree. I just don't think these problems should all be solved within
> juju itself.
> 

*agreed*

Hence making charm-helpers-sh for the little shell bits that are needed.

> > Formal leader election is, in fact, quite hard. However, given that we
> 
> s/Formal/Correct/ :-)
> 
> > have some guarantees in the way unit ids are numbered, we can actually
> > do it very effectively in peer hooks. Because an id is never reused.. we
> > can always say that we are, or are not, the leader, though pinpointing
> > who is the leader, when you are not, is actually not reliable because
> > we can't guarantee the atomicity of relation-list throughout a hook's
> > execution. So this is not pure leader election of the usual kind, but
> > it does allow us to say "if I am the leader, do X".
> 
> Huh? You just described why, actually, it doesn't allow you to say that.
> 
> What you are really saying is "It is broken, but I'm doing it
> anyway.". That's fine for proof of concepts, but please be aware that
> this will eat your lunch.
> 
> > This has proven useful in a few instances, such as with ceph, in picking a
> > node to initialize the filesystem.  Because this only ever happens once,
> > we can just say "if I am the leader, do the initialization", and then
> > even if another node later on thinks it is the leader, it will not do
> 
> That's fine, but please let's not call that leader election then. By
> definition, if you have two leaders working in parallel, there's no
> leader election.
> 
> > anything because the filesystem is already initialized. Another example
> > is when a shared key must be generated and disseminated to all units,
> > this works to just pick one to be the generator and the others to wait
> > for a key to arrive. It works because even if the generator dies between
> > generating and setting, the others will receive a departed event and
> > have another chance to check if they are the leader and generate the key.
> 
> You can also end up with multiple keys.
> 

Right, I think we should probably change the term from "leader" to
"actor", as what we're really trying to choose is not the authoritative
leader of an action, but just the most efficient choice to perform an
action right now. Its entirely possible that between the time your hook
has started doing said action, and the time it is finished, juju may
think you have left the cluster and told all the other units about that.
In the CEPH case, we're still relying on CEPH to broker this action. In
the shared key case, we're accepting that the key might change. This is
not so much a leader, but just the current best choice.

This makes me wonder if we shouldn't just spin up a service inside
the units and implement our own leader election within these charm
helpers. Sure this is an inefficient thing to do given juju's abilities
to do it at a low level using ZK, but something we could do as a PoC,
and when a few charms are written relying on this inefficiency, we can
move the priority a bit higher in juju.

> > I'm open to suggestions on how to get things into juju faster, but
> 
> Me too. :-)
> 
> > I figure doing them in charm-tools first has the added benefit of us
> > charmers actually having tried something before asking for yet another
> > feature. We've talked about *all* of these things in the past, and agreed
> > they should be in juju (like a file downloader, and leader election). But
> > time is limited, and this will suffice for most needs in the short term.
> 
> I understand, and appreciate a lot the fact you've been pushing things
> forward. The point I'm raising is that you're recommending an approach
> that actually isn't so great for what the name implies.
> 

Agreed, I'll take a look at the two approaches I laid out above. This
process is iterative, so the feedback is also appreciated.