New Package for Feature Tests

[Ian Booth]

Thank you Katherine, it's great to see this important work come to fruition.

One area of the code in particular which will benefit from this is the CLI,
implemented in cmd/juju. Historically, cmd/juju "unit" tests were written on top
of a full stack (as an aside, any test suite which embeds JujuConnSuite is not a
unit test). Recently implemented commands have tests which stub out the api
client and are written as true unit tests. However, these are missing end-end
integration tests.

Obviously we should look to split, as time allows, existing command tests into
unit tests and feature tests. But could authors of recently added command tests
which are missing feature tests please go ahead and add those.

To reinforce what Katherine says, the feature tests should only really cover the
happy path, to ensure that everything is wired together and working properly.

Moving forward, reviewers should look to push back on branches which 1) do not
use proper unit tests, 2) do not have feature tests.

On 09/01/15 12:03, Katherine Cox-Buday wrote:
> Hey everyone,
> 
> I just landed a PR which introduces a new package for Juju which is
> intended to host long-running end-to-end feature tests. You can have a look
> here: github.com/juju/juju/blob/master/featuretests/doc.go
> 
> A little context as I understand it:
> 
>    1. This is the direct result of the team's discussion about segregating
>    long-running tests from short-running tests.
>    2. It is the intention of the team that tests now be written thus:
>       - Light-weight unit tests alongside Juju core packages.
>       - End-to-end feature tests in this new package.
> 
> Hopefully this allows us to be more agile as we modify code, but still
> maintain the safety-net of end-to-end tests. The main difference for me is
> that the bulk of our tests -- where we test edge-cases, all permutations of
> calls, etc. -- will now be in lightweight unit tests. The heavier-weight,
> end-to-end tests will now be used in a 1:1 ratio with user-facing features,
> and the number of these that we have to maintain should drop off a bit.
> 
> This has been a great team effort to steer a very large change; kudos to
> you all!
> 
> -
> Katherine
> 
> 
>