Charm Testing Spec Proposal

Gustavo Niemeyer gustavo.niemeyer at canonical.com
Wed Feb 1 21:40:17 UTC 2012


On Wed, Feb 1, 2012 at 18:03, Kapil Thangavelu
<kapil.thangavelu at canonical.com> wrote:
> The current ftests lp:juju/ftests are basically shell scripts that correspond
> to what's being specified here for a test case, albeit they don't bother with
> teardown or charm retrieval abstractions.

They actually have setup and teardown bits, and that's abstracted away
from the test itself, in the same way we should do for the testing of
charms themselves.

> http://bazaar.launchpad.net/~juju/juju/ftests/view/head:/tests/ec2-wordpress/run.sh
>
> I think given this spec, we could just incorporate those tests directly into the
> example charms for functional test runs.

Agreed.

> This looks good to me. I think the teardown at least wrt to the environment can
> be automated, the charm tests just need to clean out any local state. A

It shouldn't even have to clean up local state. Just put it in a
temporary directory that is wiped out after the test runs.

> useful automation for the tests would be running a verification script directly
> on a given unit, rather than remotely poking it from the testrunner.

The charm itself may contain such logic. Are you proposing something
else in addition to this?

> It might be outside of the scope, but capturing the unit log files on failure
> would be helpful for debugging against automated test runs.

Very true.

> One additional concern is that this piece.
>
> """
> There's a special sub-command of juju, ``deploy-previous``, will deploy the
> last successfully tested charm instead of the one from the current
>  delta. This will allow testing upgrade-charm.
> """
>
> implies some additional infrastructure, at least a test database recording test
> runs against versions.

This should be left out for the moment, IMO, and should definitely not
be shaped as a command that doesn't exist.

-- 
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog

-- I'm not absolutely sure of anything.



More information about the Juju mailing list