<div dir="ltr">To avoid creating one giant email, I'll send the second half of my testing update as a follow up. So now you have two long emails to read! The juju-test is one half of the two pronged functional testing plan. Much like hooks, tests can be written in whatever language using whichever testing frameworks you choose. What I also plan on producing this cycle is a testing harness (think charm-helpers but for functional testing) that will help simplify testing for those who choose to use it. It will be a completely optional, stand alone piece of software (much like the charm-helpers project), that can be used with any language*. I'll have a working prototype of that posted to the list in the up coming week. The aim is to lower the barrier for those wishing to write charms by abstracting some of the more tedious and burdensome functions to a few commands that can be executed by tests, while not stopping people from using their own methods of testing.<div>
<br></div><div style>* Any language that can "shell" out commands</div><div style><br></div><div style>Thanks,</div><div style>Marco Ceppi</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 23, 2013 at 6:08 PM, Marco Ceppi <span dir="ltr"><<a href="mailto:marco.ceppi@canonical.com" target="_blank">marco.ceppi@canonical.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi everyone,<div><br></div><div>Hot on the heels of plugin support[0] landing for juju-core I wanted to announce the first "plugin" for juju, juju-test, which is a re-write of the `jitsu test` command making it juju-core aware in addition to juju-0.x. While I followed much of the original spec[1] for this (as well as the same general structure of jitsu test) I took the liberty to add a few additional components that will help in the automation of functional tests in charms as well as give authors more control over testing their charms.</div>
<div><br></div><div>For starters a lot of the options have been made optional, defaulting to "sane" values. So running `juju test` in a charm will result in charm functional tests being executed.</div>
<div><br></div><div>Something that plagued the jitsu implementation was a lack of due diligence during bootstrap. In that if a bootstrap failed either from an exit code > 0 or if the cloud just didn't provision machines it would result in tests "failing" though it was in no way a fault of the test. The first release will at least ensure a bootstrap is successful and the node is in a started state before running the test.</div>
<div><br></div><div>Timeouts have been added when necessary (both for during the bootstrap as well as for the test execution). These are configurable from the command line.</div><div><br></div><div>
You can "break" on a failed test which will leave the environment running and halt all testing.</div><div><br></div><div>You can execute one or more tests by passing them as command-line parameters</div>
<div><br></div><div>There's an entire help (-h|--help) with all options and their descriptions.</div><div><br></div><div>There's verbosity controls via (-v[v,] and -q)</div><div><br></div>
<div>There are still a few parts from the jitsu test program that I haven't completed yet, mainly the ability to use --isolate-environment. If there's someone with a use case for this would you let me know? I want to make sure it gets implemented in a way that works for you. There is also a bit of code cleanup and consolidation in order. Finally, output from the charm tests isn't displayed at all (yet). This will need to be fixed, again just trying to get this out as first revision.</div>
<div><br></div><div>--</div><div><br></div><div>This is a first cut of the tool, I welcome everyone's feedback, implementation nitpicks merge proposals, and bugs. I really want to make sure this testing tool is as robust and open as it needs to be to run a wide diversity of testing layouts. There's still a few more tests to be written as well but I feel I've sat on this long enough and wanted to get more eyes on.</div>
<div><br></div><div>--</div><div><br></div><div>Installation. Branch lp:juju-plugins, symlink juju-plugins/plugins/juju_test.py to juju-test somewhere in your path. Run juju-test -h inside a charm directory with embedded tests! I'll be working on getting the setup.py and some basic packaging in a ppa soon.</div>
<div><br></div><div>[0]: <a href="https://blueprints.launchpad.net/juju-core/+spec/s-cloud-juju-core-plugin" target="_blank">https://blueprints.launchpad.net/juju-core/+spec/s-cloud-juju-core-plugin</a></div><div>[1]: <a href="https://juju.ubuntu.com/docs/charm-tests.html" target="_blank">https://juju.ubuntu.com/docs/charm-tests.html</a></div>
</div>
</blockquote></div><br></div>