Unit Tests & Integration Tests

[roger peppe]

On 11 September 2014 16:29, Matthew Williams
<matthew.williams at canonical.com> wrote:
> Hi Folks,
>
> There seems to be a general push in the direction of having more mocking in
> unit tests. Obviously this is generally a good thing but there is still
> value in having integration tests that test a number of packages together.
> That's the subject of this mail - I'd like to start discussing how we want
> to do this. Some ideas to get the ball rolling:

Personally, I don't believe this is "obviously" a good thing.
The less mocking, the better, in my view, because it gives
better assurance that the code will actually work in practice.

Mocking also implies that you know exactly what the
code is doing internally - this means that tests written
using mocking are less useful as regression tests, as
they will often need to be changed when the implementation
changes.

Two good reasons to mock stuff in tests:

1) because it's awkward or impossible to use the real code in a test
environment.
2) because the real code is slow or unreliable.

When neither of these two criteria apply, I believe we should
avoid mocking.

As for integration tests, I think we should run them by default,
but provide a flag for skipping slow tests. The go test command has a "-short"
flag that can be used to skip long-running tests. Perhaps
we could consider doing something similar (perhaps triggered
with an environment variable).

  cheers,
    rog.

> Having integration tests spread about the package and having environment
> variables that switch on/ off them being run
>
> $ JUJU_INTEGRATION go test ./...
>
> We could make use of build tags:
>
> $ go test -tags integration ./...
>
> We could put all the integration tests in a single package:
>
> $ go test github.com/juju/juju/integrationtests/...
>
>
> Thoughts?
>
> Matty
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>