my first charm

[Nick Veitch]

Wow! "Docs not rubbish" shock...

Thanks for this though, I know there are places where the docs can be
improved a lot, mainly in the areas of explaining things better and I
will be saving this mail to refer to when I get a chance to iterate
over these pages again.

Nick

On Wed, Sep 3, 2014 at 8:46 PM, Eric Snow <eric.snow at canonical.com> wrote:
> Hi all,
>
> Here's a write-up on my experience writing a charm for the first time.
> In summary, it wasn't nearly as bad as I expected it to be.  The docs
> were super useful (both the getting-started side of things and later
> for reference).  There really wasn't a time that I got stuck for more
> than a minute or two (other than with bugs in the reviewboard charm).
>
> Keep in mind that I'm still rather unfamiliar with charming and likely
> have missed stuff, so take any criticisms with a grain of salt (though
> they may still represent common misconceptions).  Also note that I
> have a few months under my belt with the juju code base and community,
> so I had some advantages that other first-time charmers might not
> enjoy.  Also note that nearly all my experience was using local
> provider.  I expect that if I had been using some remote provider
> (like EC2) the whole time, the experience would have been much more
> frustrating.
>
> I'll break down my experience below (doing my best from memory).  The
> repo history [1] gives a vague look at how things progressed.  Feel
> free to ask me all about it, particularly if anything is unclear or it
> feels like anything is missing from my story.
>
> -eric
>
> [1] starting at
> https://bitbucket.org/ericsnowcurrently/rb_oauth_extension/commits/91c32dc82d7318634e9428eec48e68a687089eba
>
> =================================
>
> Summary
> ------------------
>
> The reviewboard-oauth charm provides the rb_oauth reviewboard
> extension, both installing it and registering it.  At first it was
> stand-alone, but I could tell something wasn't right with that
> approach and soon discovered about subordinate charms.  The tricky
> part of the charm is thus when it's related to a reviewboard service
> (that's when we directly modify reviewboard's settings).
>
> Writing the hooks was pretty straight-forward.  I took advantage of
> several existing charms (apache2, reviewboard, postgresql, django),
> frequently referring to them for examples of how to do stuff.  Between
> those and the docs, it was relatively easy to figure out what to do
> and where to do it.
>
> The charm helpers (especially hookenv.py) were indispensable.  Not
> only were they directly useful but the code there helped me understand
> how things fit together.
>
> While it's not the end of the world (and probably an improvement over
> yesteryear), writing the hooks involved what seemed to me like a lot
> of unnecessary boilerplate.  In the end I took at stab at factoring
> out a lot of the boilerplate into "external" modules so that my actual
> top-level code was much more focused.
>
> The most frustrating part (in the mild sense of the word) was in
> managing the env/service lifecycle from the commandline, particularly
> where one of the charms is somewhat fragile.  For example, it wasn't
> obvious how to remove a service that was in an error state.  There is
> no --force option on juju remove-service and I didn't realize you may
> have to call juju resolved more than once.  It wasn't until someone
> pointed that out that I was able to progress much (I was having to
> destroy-env and then bootstrap/deploy/relate/etc.).
>
> Overall, I found the experience of writing a charm to be pleasant and
> mostly snag-free.  My charm is non-trivial, but still not all that
> complex, so my experience may or may not be all that representative.
> Furthermore, there is functionality that I did not add in new/existing
> hooks which I probably should at some point, so perhaps I would have
> had more trouble.  However, I don't think that would have been the
> case.  So again, writing my first charm was a good experience.
>
> My Charm
> ------------------
>
> https://jujucharms.com/~ericsnowcurrently/trusty/reviewboard-oauth/
> https://jujucharms.com/~ericsnowcurrently/precise/reviewboard-oauth/
>
> Repo
> ------------------
>
> https://bitbucket.org/ericsnowcurrently/rb_oauth_extension/src/default/charm/reviewboard-oauth/
> (mercurial)
> (for the charm store:
> https://code.launchpad.net/~ericsnowcurrently/charms/trusty/reviewboard-oauth/trunk)
>
> Helpful
> ------------------
>
> * the docs were great
> * the charm helpers (e.g. hookenv.py) made things much easier
>
> Could Be Better (mostly nitpicks)
> ------------------
>
> * it wasn't clear at first how config.yaml related to my deployed
> charm (though it became clear quickly)
> * I had to rely on a blog post (thanks Nate!) for an explanation on
> personal namespaces
> * regarding hooks, there's a lot of boilerplate that could probably be
> eliminated (see my final Charm abstraction)
> * the relationship between hooks and juju commands (e.g. deploy, set)
> could be clearer
>   - the execution path from command to hook
>   - the role of jujuc
>   - which hooks are triggered by each command and in which order
> * interacting with related charms still isn't super clear to me
> * the lifecycle status of a charm isn't super clear (and certainly not
> well defined) from a related charm (e.g. to know if the reviewboard
> charm is ready for business I have to manually check by looking for
> its settings file, which feels like a hack)
> * the directory structure required for local charm repos is annoying
> * juju remove-service should have a --force option
>
> Suprises
> ------------------
>
> * no hooks are involved in exposing a service
> * there is no unexpose command
> * there aren't any standard (optional) "required" interfaces for
> common cross-cutting services (like logging and backups and
> redundancy[1])
>
> [1] I haven't had a chance to look at haproxy, but I'd expect that
> interface to be dependent on services that support multiple units.
>
> Here's what I did (roughly)
> ------------------
>
> My memory for this sort of detail isn't super, so I've pieced this
> section together the best I can.  The sequence here is most assuredly
> not accurate, but it's close enough and captures the bulk of what took
> place.
>
> prep:
> 1. looked up reviewboard charm in charm store
> 2. figured out how to deploy charms
> 3. deployed on local provider (working through several blockers)
> 4. figured out about setting configs and removing charms
> 5. discovered about the log files under .juju*
> 6. discovered that charms may not be robust under all permutations of
> deploy/relate/set
>
> * I later discovered juju debug-log, but didn't find it to be an
> improvement, and never tried debug-hooks.
>
> starting:
> 1. copied the reviewboard charm code into charm/reviewboard-oauth/ in
> the rb-oauth repo
> 2. removed/replaced unapplicable files
> 3. set basic info in config.yaml, metadata.yaml, and charm-helpers.yaml
> 4. set up symlinks for applicable hooks to hooks.py
> 5. added code for directly interacting with the installed reviewboard
> 6. add the hooks in hooks.py
>
> trying it out:
> 1. figured out how to use a local charm repo
> 2. deployed the charm
> 3. it didn't work
> 4. took at look at the unit's log under .juju and app logs on the
> machine (via juju ssh)
> 5. manually tweaked things on the machine (also via juju ssh) until it worked
>
> cleaning up:
> 1. ran juju charm proof
> 2. added a README, copyright and revision file
> 3. added tests (and fixed things in response)
> 4. added Makefile to simplify testing
>
> stabilizing:
> 1. turned it into a subordinate charm
> 2. deployed the charm (postgresql/reviewboard already set up)
> 3. it didn't work
> 4. took at look at the unit's log under .juju and app logs on the
> machine (via juju ssh)
> 5. tweaked and refactored and added tests
> 6. removed the charm
> 7. go to 1
> 8. it finally mostly worked without manually tweaking things
>
> wrapping up:
> 1. restructured code for better encapsulation
> 2. added a charm config that will result in modifying the reviewboard install
> 3. stabilized again
>
> publishing:
> 1. set up a local bzr repo
> 2. pushed an LP branch to launchpad
> 3. looked up the charm in the charm store
> 4. deployed to local provider from the charm store (without a hitch)
> 5. deployed to an EC2 environment from the charm store
> 6. ...
> 7. profit!
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev



-- 
Nick Veitch
nick.veitch at canonical.com