[ANNOUNCE] New version of "jitsu test", juju-test

[Marco Ceppi]

To avoid creating one giant email, I'll send the second half of my testing
update as a follow up. So now you have two long emails to read! The
juju-test is one half of the two pronged functional testing plan. Much like
hooks, tests can be written in whatever language using whichever testing
frameworks you choose. What I also plan on producing this cycle is a
testing harness (think charm-helpers but for functional testing) that will
help simplify testing for those who choose to use it. It will be a
completely optional, stand alone piece of software (much like the
charm-helpers project), that can be used with any language*. I'll have a
working prototype of that posted to the list in the up coming week. The aim
is to lower the barrier for those wishing to write charms by abstracting
some of the more tedious and burdensome functions to a few commands that
can be executed by tests, while not stopping people from using their own
methods of testing.

* Any language that can "shell" out commands

Thanks,
Marco Ceppi


On Thu, May 23, 2013 at 6:08 PM, Marco Ceppi <marco.ceppi at canonical.com>wrote:

> Hi everyone,
>
> Hot on the heels of plugin support[0] landing for juju-core I wanted to
> announce the first "plugin" for juju, juju-test, which is a re-write of the
> `jitsu test` command making it juju-core aware in addition to juju-0.x.
> While I followed much of the original spec[1] for this (as well as the same
> general structure of jitsu test) I took the liberty to add a few additional
> components that will help in the automation of functional tests in charms
> as well as give authors more control over testing their charms.
>
> For starters a lot of the options have been made optional, defaulting to
> "sane" values. So running `juju test` in a charm will result in charm
> functional tests being executed.
>
> Something that plagued the jitsu implementation was a lack of due
> diligence during bootstrap. In that if a bootstrap failed either from an
> exit code > 0 or if the cloud just didn't provision machines it would
> result in tests "failing" though it was in no way a fault of the test. The
> first release will at least ensure a bootstrap is successful and the node
> is in a started state before running the test.
>
> Timeouts have been added when necessary (both for during the bootstrap as
> well as for the test execution). These are configurable from the command
> line.
>
> You can "break" on a failed test which will leave the environment running
> and halt all testing.
>
> You can execute one or more tests by passing them as command-line
> parameters
>
> There's an entire help (-h|--help) with all options and their descriptions.
>
> There's verbosity controls via (-v[v,] and -q)
>
> There are still a few parts from the jitsu test program that I haven't
> completed yet, mainly the ability to use --isolate-environment. If there's
> someone with a use case for this would you let me know? I want to make sure
> it gets implemented in a way that works for you. There is also a bit of
> code cleanup and consolidation in order. Finally, output from the charm
> tests isn't displayed at all (yet). This will need to be fixed, again just
> trying to get this out as first revision.
>
> --
>
> This is a first cut of the tool, I welcome everyone's feedback,
> implementation nitpicks  merge proposals, and bugs. I really want to make
> sure this testing tool is as robust and open as it needs to be to run a
> wide diversity of testing layouts. There's still a few more tests to be
> written as well but I feel I've sat on this long enough and wanted to get
> more eyes on.
>
> --
>
> Installation. Branch lp:juju-plugins, symlink
> juju-plugins/plugins/juju_test.py to juju-test somewhere in your path. Run
> juju-test -h inside a charm directory with embedded tests! I'll be working
> on getting the setup.py and some basic packaging in a ppa soon.
>
> [0]:
> https://blueprints.launchpad.net/juju-core/+spec/s-cloud-juju-core-plugin
> [1]: https://juju.ubuntu.com/docs/charm-tests.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20130523/9bd2a889/attachment.html>