my first charm

[Eric Snow]

Hi all,

Here's a write-up on my experience writing a charm for the first time.
In summary, it wasn't nearly as bad as I expected it to be.  The docs
were super useful (both the getting-started side of things and later
for reference).  There really wasn't a time that I got stuck for more
than a minute or two (other than with bugs in the reviewboard charm).

Keep in mind that I'm still rather unfamiliar with charming and likely
have missed stuff, so take any criticisms with a grain of salt (though
they may still represent common misconceptions).  Also note that I
have a few months under my belt with the juju code base and community,
so I had some advantages that other first-time charmers might not
enjoy.  Also note that nearly all my experience was using local
provider.  I expect that if I had been using some remote provider
(like EC2) the whole time, the experience would have been much more
frustrating.

I'll break down my experience below (doing my best from memory).  The
repo history [1] gives a vague look at how things progressed.  Feel
free to ask me all about it, particularly if anything is unclear or it
feels like anything is missing from my story.

-eric

[1] starting at
https://bitbucket.org/ericsnowcurrently/rb_oauth_extension/commits/91c32dc82d7318634e9428eec48e68a687089eba

=================================

Summary
------------------

The reviewboard-oauth charm provides the rb_oauth reviewboard
extension, both installing it and registering it.  At first it was
stand-alone, but I could tell something wasn't right with that
approach and soon discovered about subordinate charms.  The tricky
part of the charm is thus when it's related to a reviewboard service
(that's when we directly modify reviewboard's settings).

Writing the hooks was pretty straight-forward.  I took advantage of
several existing charms (apache2, reviewboard, postgresql, django),
frequently referring to them for examples of how to do stuff.  Between
those and the docs, it was relatively easy to figure out what to do
and where to do it.

The charm helpers (especially hookenv.py) were indispensable.  Not
only were they directly useful but the code there helped me understand
how things fit together.

While it's not the end of the world (and probably an improvement over
yesteryear), writing the hooks involved what seemed to me like a lot
of unnecessary boilerplate.  In the end I took at stab at factoring
out a lot of the boilerplate into "external" modules so that my actual
top-level code was much more focused.

The most frustrating part (in the mild sense of the word) was in
managing the env/service lifecycle from the commandline, particularly
where one of the charms is somewhat fragile.  For example, it wasn't
obvious how to remove a service that was in an error state.  There is
no --force option on juju remove-service and I didn't realize you may
have to call juju resolved more than once.  It wasn't until someone
pointed that out that I was able to progress much (I was having to
destroy-env and then bootstrap/deploy/relate/etc.).

Overall, I found the experience of writing a charm to be pleasant and
mostly snag-free.  My charm is non-trivial, but still not all that
complex, so my experience may or may not be all that representative.
Furthermore, there is functionality that I did not add in new/existing
hooks which I probably should at some point, so perhaps I would have
had more trouble.  However, I don't think that would have been the
case.  So again, writing my first charm was a good experience.

My Charm
------------------

https://jujucharms.com/~ericsnowcurrently/trusty/reviewboard-oauth/
https://jujucharms.com/~ericsnowcurrently/precise/reviewboard-oauth/

Repo
------------------

https://bitbucket.org/ericsnowcurrently/rb_oauth_extension/src/default/charm/reviewboard-oauth/
(mercurial)
(for the charm store:
https://code.launchpad.net/~ericsnowcurrently/charms/trusty/reviewboard-oauth/trunk)

Helpful
------------------

* the docs were great
* the charm helpers (e.g. hookenv.py) made things much easier

Could Be Better (mostly nitpicks)
------------------

* it wasn't clear at first how config.yaml related to my deployed
charm (though it became clear quickly)
* I had to rely on a blog post (thanks Nate!) for an explanation on
personal namespaces
* regarding hooks, there's a lot of boilerplate that could probably be
eliminated (see my final Charm abstraction)
* the relationship between hooks and juju commands (e.g. deploy, set)
could be clearer
  - the execution path from command to hook
  - the role of jujuc
  - which hooks are triggered by each command and in which order
* interacting with related charms still isn't super clear to me
* the lifecycle status of a charm isn't super clear (and certainly not
well defined) from a related charm (e.g. to know if the reviewboard
charm is ready for business I have to manually check by looking for
its settings file, which feels like a hack)
* the directory structure required for local charm repos is annoying
* juju remove-service should have a --force option

Suprises
------------------

* no hooks are involved in exposing a service
* there is no unexpose command
* there aren't any standard (optional) "required" interfaces for
common cross-cutting services (like logging and backups and
redundancy[1])

[1] I haven't had a chance to look at haproxy, but I'd expect that
interface to be dependent on services that support multiple units.

Here's what I did (roughly)
------------------

My memory for this sort of detail isn't super, so I've pieced this
section together the best I can.  The sequence here is most assuredly
not accurate, but it's close enough and captures the bulk of what took
place.

prep:
1. looked up reviewboard charm in charm store
2. figured out how to deploy charms
3. deployed on local provider (working through several blockers)
4. figured out about setting configs and removing charms
5. discovered about the log files under .juju*
6. discovered that charms may not be robust under all permutations of
deploy/relate/set

* I later discovered juju debug-log, but didn't find it to be an
improvement, and never tried debug-hooks.

starting:
1. copied the reviewboard charm code into charm/reviewboard-oauth/ in
the rb-oauth repo
2. removed/replaced unapplicable files
3. set basic info in config.yaml, metadata.yaml, and charm-helpers.yaml
4. set up symlinks for applicable hooks to hooks.py
5. added code for directly interacting with the installed reviewboard
6. add the hooks in hooks.py

trying it out:
1. figured out how to use a local charm repo
2. deployed the charm
3. it didn't work
4. took at look at the unit's log under .juju and app logs on the
machine (via juju ssh)
5. manually tweaked things on the machine (also via juju ssh) until it worked

cleaning up:
1. ran juju charm proof
2. added a README, copyright and revision file
3. added tests (and fixed things in response)
4. added Makefile to simplify testing

stabilizing:
1. turned it into a subordinate charm
2. deployed the charm (postgresql/reviewboard already set up)
3. it didn't work
4. took at look at the unit's log under .juju and app logs on the
machine (via juju ssh)
5. tweaked and refactored and added tests
6. removed the charm
7. go to 1
8. it finally mostly worked without manually tweaking things

wrapping up:
1. restructured code for better encapsulation
2. added a charm config that will result in modifying the reviewboard install
3. stabilized again

publishing:
1. set up a local bzr repo
2. pushed an LP branch to launchpad
3. looked up the charm in the charm store
4. deployed to local provider from the charm store (without a hitch)
5. deployed to an EC2 environment from the charm store
6. ...
7. profit!