Getting charms in to trusty

[Marco Ceppi]

Hello,

tl;dr: We've built very *very* simple tests to spur charm authors in to
getting their charms updated and validated for trusty.

Hot on the heels of the previous email, a few of us have been working on
getting every charm from precise in to trusty. This has been no easy task
by any means and is something we originally started with the notion that it
would simply take a little bit of time. Part of doing so was to create
simple, barebones tests which could be executed in our automated testing
environment. Due to the simplicity of these tests only one real point of
data can be determined: does the charm fail?

We've since completed this task of creating simple tests for all charms,
charm-tools has been updated to include the test generation method we used
(`charm add tests`) this will create two files a 00-setup file and a
99-autogen file in the tests directory of a charm. The tests will attempt
to stand up the charm and create relations to the service and validate that
all services move to started. These are not acceptable comprehensive tests,
as such we've applied the unmaintained charm policy[0] to these charms. If
they fall out of the charm store they'll be removed from trusty but remain
in precise under legacy policy.

We began to realize a lot of these charms weren't trusty safe, especially
ones which relied on Apache2, which in trusty (apache2.4) changed the way
configuration files were handled. This, and other incompatibilities between
trusty and precise (and the volume of charms) caused delays in the
review-queue for the past several weeks. We've since elected to press
forward with merging these failing tests under the basis that: the tests
are a good starting point for authors and contributors, the charm will be
in trusty, and we apply the unmaintained policy to the charm as mentioned
above.

A result of this, and the general low grade of tests generated, we will be
checking for 99-autogen test files in `charm proof` output over the next
several weeks. This is so authors and contributors working on these charm
will be aware, that, while this is a good base to start from it's not a
satisfactory nor complete test and if action isn't taken the charm may fall
out of the trusty namespace.

In hindsight, and after much reflection, a lot of this work should have
been discussed sooner. A task which we underestimated ended up effecting a
lot of the review queue and caused quite a large amount of congestion with
not a lot of planning or dissemination of information. Going forward we
plan to announce and discuss harebrained schemes much sooner to not only
get feedback from the community at large, but also recruit help in our
efforts.

I want to extend my thanks for the patience to those authors who have had
to deal with longer than desired wait times in the queue and my apologies
on behalf of the Juju Solutions team for not communicating our goals and
plans sooner.

[0]: https://lists.ubuntu.com/archives/juju/2014-August/004098.html

Thanks,
Marco Ceppi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20150114/4ddcef04/attachment.html>