Charms Failing Auto Testing

Antonio Rosales antonio.rosales at canonical.com
Sat Dec 20 00:32:21 UTC 2014


Over this last cycle we’ve taken a good chunk of time to really scope
out what’s going on in the charm store. A by-product of this was our
new Continuous Integration system[0][1] that you might have seen
commenting on new Charms and Merge Proposals. After our first set of
reviews, we discovered some charms need attention as they no longer
deploy properly. In response we have created bugs against the charms
that fail the tests.

The test consists of 4 basic things:
  1. Does the charm pass static analysis (charm proof)?
  2. If the charm has a Makefile with a “make lint” target it runs
`make lint` fail or pass is based on the return code.
  3. If the charm has a Makefile with a “make test” target it runs
`make test`, fail or pass is based on return code.
  4. Runs the executable files in the charm-name/tests directory.

The bug list is at: https://bugs.launchpad.net/bugs/+bugs?field.tag=auto-test
Charm Maintainers you have been specifically added to the bcc list.

To ensure charms are deploying as the author intended unmaintained
charms[2] are to be moved from the recommended namespace into the
unmaintained namespace[3]. What this means to you; you can still
deploy these charms, the charm remains in the Juju Charm Store, and
this will not affect any existing deployments.

We do not want to move any charms out of the recommended namespace.
Therefore, if you author, maintain, care about a specific charm, or
looking to contribute to the Juju Community; this is your opportunity
to fix the charm failure or reply to the bug that you are working on
it.

This email is to notify charm authors of the failure and the needed
action to bring the charm back to a working state. Action needed:
  -Comment in the bug stating a resolution is in progress.
  -Merge Proposal with a fix addressing the above test failures.

If the charm is not fixed or does not have a comment in the bug
stating work is in progress it will be moved into the unmaintained
name space as of January 15, 2015. Some folks have already started
work to bring charms back into good test standing and that is much
appreciated.

We’re taking due diligence in making sure the recommended charms are
100% in alignment with policy[4]. The main intent is to ensure any
user deploying a recommended charm has the positive experience the
author intended. Thus, the Charm Testing infrastructure lets authors
know how their charm is working (based on charm tests) in all
supported clouds (lxc, HPCloud, Joyent, Azure, MAAS, and AWS). This
provides Charm Authors an easy automated method to keep their charm in
an optimal state across multiple substrates.


All the best and happy holidays.


[0] http://reports.vapour.ws/charm-tests-by-charm
[1] http://review.juju.solutions/
[2] https://jujucharms.com/docs/charm-unmaintained-process
[3] https://launchpad.net/~unmaintained-charms
[4]https://jujucharms.com/docs/authors-charm-policy

thanks,
-- 
Antonio Rosales
Juju Ecosystem
Canonical



More information about the Juju mailing list