proposed-migration autopkgtest now switched to new cloud infrastructure

[Martin Pitt]

Hello again,

The new cloud-based autopkgtest infrastructure that got announced in
[1] has run for three weeks and is working reasonably reliable now
(which is: already way more reliable than the old Jenkins based
machinery). Thus it is time to flip the switch, and I just rolled out
proposed-migration (aka britney) to only trigger and evaluate
autopkgtests from the cloud setup.

Thus

  http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html

is now back to only showing one set of tests again, the
"(informational)" bits are gone and are now the primary data.

Some things to be aware of
==========================

 - "Regression" is now determined on a per-architecture level, i. e.
   if a package never succeeded on i386 but did succeed on amd64, the
   former will count as "always failed" (and thus not block
   promotion), the latter as "regression".

 - No more Jenkins. http://autopkgtest.ubuntu.com/ (a debci instance
   configured for Ubuntu) is now the (completely public) results
   browser for human consumption, with a web UI and a global and
   per-package news feed.

 - There are no email notifications about regressions right now. It
   seems most people ignored them as they produced too much noise due
   to flaky tests or infrastructure (and us retrying them), and they
   also notified the wrong person (the uploader of the package with
   broken tests, not the uploader of the package which caused the
   tests to break). Bringing this back is being worked on, but for now
   I'll ping responsible people directly (I. e. what I've already done
   already anyway for a long time).

 - debci also produces machine readable JSON status files for every
   package, run, and a global "packages.json" for a whole
   release/arch, for example:
   http://autopkgtest.ubuntu.com/data/status/wily/amd64/packages.json

 - If you run some automation/reporting on top of autopkgtest data
   (like the kernel team does), you are welcome to use the above JSON
   files, but I highly recommend querying the results from swift
   directly. The Swift API [2] offers reasonably flexible querying
   with plain/JSON/XML output, and unlike debci (which changes every
   now and then) the container format won't change. Please get in
   touch with me if you want to build/update reporting from this data.


Next steps
==========

 - Bring back email notifications. (~ 1 week)

 - Add armhf/ppc64el testing. I integrated our existing testbed
   hardware into the new system already, so this is merely an issue of
   changing the britney configuration. But I want to wait until after
   the big gcc 5 transition to avoid creating unnecessary stumbling
   blocks. Note that after that, regressions on these architectures
   will lead to blocking the package, unlike with the old
   infrastructure where these arches were only informational. But as
   we do per-architecture regression detection now, we can start
   enforcing them.

 - Document the whole system. (~ 2 weeks)

Bugs
====
Please continue filing bugs at

  https://bugs.launchpad.net/auto-package-testing/


Thanks,

Martin

[1] https://lists.ubuntu.com/archives/ubuntu-devel-announce/2015-July/001141.html
[2] http://developer.openstack.org/api-ref-objectstorage-v1.html#storage_object_services
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel-announce/attachments/20150804/3871421c/attachment.pgp>