Intermittent test failures with juju-core

[Tim Penhey]

Hi All (again),

After my issues getting goose updated and juju-core at the right
revision, I had some weird results with the tests.

If the trunk branch is not protected by a robot doing merges, I'll often
check the tests for trunk on updating them as I know how easy it is to
merge stuff that breaks other tests if not checking things carefully [1]

So... running `go test launchpad.net/juju-core/...` failed the first
time - to my surprise.

http://pastebin.ubuntu.com/1654636/

I was talking with Ian at the time, and he mentioned that he had heard
of some intermittent test failures (like the old windmill tests in
launchpad), so I ran the test suite again.

This time through I only had one group of failures instead of two.

http://pastebin.ubuntu.com/1654649/

As you can see, the tests for launchpad.net/juju-core/environs/openstack
are now passing.  Now I did nothing in between these test runs.

Having go different results once, I ran the tests again.

This time they all passed.  Not believing it, I ran them again (and
waited [2]), all passed again, run again, all passed, run again, ...
I was all ready to say they all passed again, but no... a different test
failed this time:

http://pastebin.ubuntu.com/1654670/


Now I'm sure I don't need to tell you how problematic intermittent
failing tests are.  Do we have any plans to fix these?


Tim


[1] We really should have a protected trunk that normal devs don't have
permission to write to, but the landing robot (like tarmac) does.

[2] launchpad.net/juju-core/worker/uniter	89.354s
What are we doing here that takes almost a minute and a half?