autopkgtest - automated package testing

Ian Jackson ian at
Thu Mar 1 16:04:59 GMT 2007

The automated package testing system I've been working on is now in a
state where I'd like other people to start thinking about adding tests
to their packages, and running them before uploading.  I'm talking to
our sysadmins about the plumbing needed to run it automatically on the
datacentre machines, but failing that I will probably set something up
to use a few spare cycles on a machine I have at home.

So, a quick run-down:

 * autopkgtest runs tests that are included in the source package.

 * Tests are run on the _installed_ copy of a package.  This might be
   different to the version of the package which contains the tests.

 * The tests have to be described in a debian/tests/control file
   but typically this is very straightforward.

 * The tests can be run
    - on your development system
    - in a chroot
    - in an ephemeral Xen virtual machine
   with increasing levels of isolation.

 * autopkgtest can automatically build and install packages
   (a bit like pbuilder but different).  For example, it can be
   told to build a source package and automatically test the resulting

To use it yourself you can install autopkgtest.  This package

  * adt-run, the main test runner, and its manpage which
     describes its invocation in detail.
  * /usr/share/doc/README.tests-specification which
     describes how to declare tests in a source package.
  * `virtualisation servers' which allow the use of the main system
    or a chroot as the testbed.

The documentation is in a `reference manual' or `specification'
style.  If someone would like to volunteer to write something more
tutorial-ish please get in touch: I find that I don't get on well with
tutorials and don't write them well either.

The packages `dovecot' and `mawk' contain some tests and give a
flavour of one way things can be plumbed into autopkgtest.

There are likely still to be bugs in adt-run (and in the other
functionality) since currently nearly no-one but me has ever run them
and my Python isn't particularly good :-).

If you are interested in the Xen arrangements, take a look at the
autopkgtest-xenlvm package.  This contains scripts which given a
working Xen host setup, and other suitable resources, can:
  * set up a testbed template (an install made with pbuilder,
    with some relevant LVM and Xen glue)
  * instantiate the template into a running instance of a disposable
    testbed, and discard it when it is no longer needed
  * run commands on the disposable testbed, while it is up
  * provide a virtualisation server which plumbs the above facilities
    into adt-run.


More information about the ubuntu-devel mailing list