[Acked] [PATCH 0/5] autopkgtest skippable fixes

[Andy Whitcroft]

On Thu, Mar 07, 2019 at 03:32:17PM +0000, Dimitri John Ledkov wrote:
> This is a set of patches to unbreak autopkgtests, as designed and
> agreed in Malta kernel room mini-round-table between xnox, apw,
> sforshee, bjf.
> 
> The general notio is that we don't skip tests enough, when they are
> over-triggered, and thus we keep on attempting to run them, when we
> should be bailing out.
> 
> It is hard for me to build a full abi bump kernel, with a matching
> bumped meta, and signed variants, so I monkey patched these tests into
> various trees, and tried to run them with appropriate envrionment
> variable as seen in britney results.
> 
> Here are new things:
> 
> $ cd linux-azure-4.18.0/ && ./debian/tests/ubuntu-regression-suite
> ubuntu-regression-suite is pointless, if one cannot boot the kernel
> $ echo $?
> 77
> 
> $ cd linux-5.0.0/ && ADT_TEST_TRIGGERS=binutils/2.32-5ubuntu1 ./debian/tests/ubuntu-regression-suite 
> ubuntu-regression-suite is not requested, as there is no linux-meta trigger
> $ echo $?
> 77
> 
> I also validated that the current expected tests do run correctly, e.g.:
> 
> $ ADT_TEST_TRIGGERS=binutils/2.32-5ubuntu1 ./debian/tests/rebuild 
> dpkg-buildpackage: info: source package linux
> dpkg-buildpackage: info: source version 5.0.0-7.8
> dpkg-buildpackage: info: source distribution disco
> ...
> (build attempted)
> 
> $ ADT_TEST_TRIGGERS=linux-meta/5.0.0.7.8 ./debian/tests/ubuntu-regression-suite 
> Source Package Version: 5.0.0-7.8
> Running Kernel Version: 4.19.0-13.14
> ERROR: running version does not match source package
> ...
> (well, i did fake the trigger, but yeah, the right kernel is pulled
> in, in this case, and the test will be run)
> 
> Please review these patches and apply them against all kernel
> flavours, in all releases.
> 
> Dimitri John Ledkov (5):
>   Set +x on rebuild testcase.
>   Skip rebuild test, for regression-suite deps.
>   Make ubuntu-regression-suite skippable on unbootable kernels.
>   make rebuild use skippable error codes when skipping.
>   Only run regression-suite, if requested to.
> 
>  debian/tests/control                 |  4 ++--
>  debian/tests/rebuild                 |  4 ++--
>  debian/tests/ubuntu-regression-suite | 26 ++++++++++++++++++++++++++
>  3 files changed, 30 insertions(+), 4 deletions(-)
>  mode change 100644 => 100755 debian/tests/rebuild

This all looks to do what is claimed and feels like it adds up to what
we wanted to achieve.  The exit codes look correct in the skippable
cases.  Lets get this applied to disco double quick and see how it goes.

Acked-by: Andy Whitcroft <apw at canonical.com>

-apw