Heads up: change in snapd test workers

[Gustavo Niemeyer]

Hello there,

We're starting to blow up the 50 minutes time quota provided by Travis
during CI of master, which means PRs may now fail for no reason other than
taking too long.

On quick inspection, prepare.sh claims to have fixed the original problem
that motivated the ubuntu-core-16-64-fixme spread system, and looking at
test results using two workers on 14.04 and ubuntu-core-64 might cut about
20 minutes of testing time.

I've pushed a PR and validated the theory in practice:

    https://github.com/snapcore/snapd/pull/2928

With this small tweak, we've indeed cut down on about 20 minutes of testing
time, going back to ~30 minutes.

The main cost of this change is just one more machine used per full test
run, and the old machine that was specific to the one bogus test we had is
now being used as a second standard worker that can parallelize all tests
targeting ubuntu-core-16-64. We're keeping that one machine allocated for
more time, but that pays off because first worker that used to take 50+
minutes now takes 30 as well.

Seems to work, but I'm leaving on holiday, so two points to keep in mind:

- The bug that seemed fixed is perhaps not fixed, and I was just lucky. If
that is the case, have a look at the PR above and either revert the change
or fix the remaining bits.

- Instant contention increases. We're now running at 8 machines per full
run (16.04-64+16.04-32+14.04+core-16, times two). On the down side, we may
see more of the "no machines available" complaint. That said, the whole run
takes less time, so trailing machines get deallocated faster. I expect the
net effect here to be positive because Travis also has a maximum number of
jobs, so even if we have machines available, that job taking 50 minutes may
be holding back the follow ups.

Anyway, just wanted to share these details as this was a last minute change
merged with almost no discussion. If you see issues, that's where to look.

Have fun!


gustavo @ http://niemeyer.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/snapcraft/attachments/20170223/d5b72132/attachment.html>