Intermittent test failures with juju-core

[roger peppe]

i really thought i'd fixed the state/api test failure.
i couldn't reproduce it.

i'll push an intermediate branch that changes the test to
accept either error, and try to find where my understanding
of the code is lacking.

sorry about that.

On 15 February 2013 01:21, Tim Penhey <tim.penhey at canonical.com> wrote:
> Hi All (again),
>
> After my issues getting goose updated and juju-core at the right
> revision, I had some weird results with the tests.
>
> If the trunk branch is not protected by a robot doing merges, I'll often
> check the tests for trunk on updating them as I know how easy it is to
> merge stuff that breaks other tests if not checking things carefully [1]
>
> So... running `go test launchpad.net/juju-core/...` failed the first
> time - to my surprise.
>
> http://pastebin.ubuntu.com/1654636/
>
> I was talking with Ian at the time, and he mentioned that he had heard
> of some intermittent test failures (like the old windmill tests in
> launchpad), so I ran the test suite again.
>
> This time through I only had one group of failures instead of two.
>
> http://pastebin.ubuntu.com/1654649/
>
> As you can see, the tests for launchpad.net/juju-core/environs/openstack
> are now passing.  Now I did nothing in between these test runs.
>
> Having go different results once, I ran the tests again.
>
> This time they all passed.  Not believing it, I ran them again (and
> waited [2]), all passed again, run again, all passed, run again, ...
> I was all ready to say they all passed again, but no... a different test
> failed this time:
>
> http://pastebin.ubuntu.com/1654670/
>
>
> Now I'm sure I don't need to tell you how problematic intermittent
> failing tests are.  Do we have any plans to fix these?
>
>
> Tim
>
>
> [1] We really should have a protected trunk that normal devs don't have
> permission to write to, but the landing robot (like tarmac) does.
>
> [2] launchpad.net/juju-core/worker/uniter       89.354s
> What are we doing here that takes almost a minute and a half?
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev


On 15 February 2013 01:21, Tim Penhey <tim.penhey at canonical.com> wrote:
> Hi All (again),
>
> After my issues getting goose updated and juju-core at the right
> revision, I had some weird results with the tests.
>
> If the trunk branch is not protected by a robot doing merges, I'll often
> check the tests for trunk on updating them as I know how easy it is to
> merge stuff that breaks other tests if not checking things carefully [1]
>
> So... running `go test launchpad.net/juju-core/...` failed the first
> time - to my surprise.
>
> http://pastebin.ubuntu.com/1654636/
>
> I was talking with Ian at the time, and he mentioned that he had heard
> of some intermittent test failures (like the old windmill tests in
> launchpad), so I ran the test suite again.
>
> This time through I only had one group of failures instead of two.
>
> http://pastebin.ubuntu.com/1654649/
>
> As you can see, the tests for launchpad.net/juju-core/environs/openstack
> are now passing.  Now I did nothing in between these test runs.
>
> Having go different results once, I ran the tests again.
>
> This time they all passed.  Not believing it, I ran them again (and
> waited [2]), all passed again, run again, all passed, run again, ...
> I was all ready to say they all passed again, but no... a different test
> failed this time:
>
> http://pastebin.ubuntu.com/1654670/
>
>
> Now I'm sure I don't need to tell you how problematic intermittent
> failing tests are.  Do we have any plans to fix these?
>
>
> Tim
>
>
> [1] We really should have a protected trunk that normal devs don't have
> permission to write to, but the landing robot (like tarmac) does.
>
> [2] launchpad.net/juju-core/worker/uniter       89.354s
> What are we doing here that takes almost a minute and a half?
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev