proposed-migration results now contain cloud testing results

[Martin Pitt]

Hello all,

In the past three weeks I've been working on and off on providing a
new cloud-based infrastructure for running our autopkgtests for gating
-proposed in the cloud. So far we've run them on four manually
administered servers in the lab, through two Jenkins instances and a
rather hideous pile of hacks and tens of thousands of little state
files which got rsynced in between hosts. This hasn't been very
reliable nor scalable, so it's time to replace this with something
much simpler and robust: We want to run the tests in temporary cloud
instances (in Canonical ScalingStack), use AMQP for request/job
distribution, OpenStack swift to store the result logs/artifacts, and
debci to browse the results.

Today I enabled this in "britney", the scripts which decide whether a
package in -proposed is allowed to enter the development series. I'm
sure most of you have seen this report before:

  http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html

The new thing is that you now see some new lines like

  (informational) cloud autopkgtest for kde4libs 4:4.14.6-4~ubuntu4: amd64: Pass, i386: Regression

with clickable links to the results on http://autopkgtest.ubuntu.com/ .

Rollout plan
------------
These cloud-based tests are now being triggered by britney in addition
to the usual Jenkins results, but aren't being taken into account for
the decision. This is why failures are currently shown as
"Regression", there is no "Always failed" state yet as these
"(informational)" lines are just temporarily added into the report.

This will now be on a trial run for at least two weeks (I'm on holiday
next week), to compare results, shake out bugs in the infrastructure,
and generally make this production ready. The biggest blocker right
now is that we don't yet have enough horse power in ScalingStack to
keep up with the flood (max 10 instances right now). This is being
worked on, but might still take a few weeks.

After this becomes sufficiently robust and scalable, we'll switch over
to these as primary results, and keep the old machinery as a backup
for a few weeks longer.

XSS feeds
---------
If you have some "pet" packages, you are welcome to subscribe to their
"news" feeds, which will get a new item whenever the package starts
failing or starts succeeding (not every individual result). E. g.

  http://autopkgtest.ubuntu.com/packages/u/udisks2/

has this little orange "broadcast" symbol after the package name
heading, leading to

  http://autopkgtest.ubuntu.com/data/feeds/u/udisks2.xml

Code
----
If you are interested in taking a look or helping out, this is the
relevant code:

 * debci for browsing the results, powering http://autopkgtest.ubuntu.com/
   http://anonscm.debian.org/cgit/collab-maint/debci.git

 * autopkgtest for actually exercising the tests (we are using the
   "ssh" runner with the "nova" setup script):
   http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git

 * The "worker" which accepts test requests via AMQP, runs adt-run,
   and puts the results into Swift; and a script to download new
   results for debci (both are being integrated into debci, but right
   now they reside here)
   Plus the charms and a "deploy.sh" script to roll it all out into
   the cloud:

   https://git.launchpad.net/~pitti/+git/autopkgtest-cloud/

 * britney, which now does direct AMQP requests and swift result
   downloads instead of the indirection (and much duplicated work) in
   lp:auto-package-testing:

   http://bazaar.launchpad.net/~ubuntu-release/britney/britney2-ubuntu
   (in particular autopkgtest.py; you can run tests with
    python tests/test_autopkgtest.py)

Bugs
----
If you encounter bugs, please file them here:

  https://bugs.launchpad.net/auto-package-testing/

This is not entirely correct as lp:auto-package-testing is the old
stuff which we want to get rid of; but for bug tracking it's a good
enough place.

Thanks,

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel-announce/attachments/20150714/05b11bfe/attachment.pgp>