Fwd: ACK/cmt: [PATCH 0/5] autopkgtest skippable fixes

[Dimitri John Ledkov]

On 3/27/19 3:50 PM, seth.forshee at canonical.com (Seth Forshee) wrote:
> On Thu, Mar 07, 2019 at 03:32:17PM +0000, Dimitri John Ledkov wrote:
>> This is a set of patches to unbreak autopkgtests, as designed and
>> agreed in Malta kernel room mini-round-table between xnox, apw,
>> sforshee, bjf.
>>
>> The general notio is that we don't skip tests enough, when they are
>> over-triggered, and thus we keep on attempting to run them, when we
>> should be bailing out.
>
> I think this looks sane. There are a few details that I'm not informed
> enough about to provide useful review, so I'd still like apw to have a
> look.
>

For example, at the moment in disco gcc-7 & gcc-8 has been stuck for 6
days now in disco-proposed.

They trigger linux tests and run both the rebuild test (good and passes)
and ubuntu-regression-testsuite (shouldn't run and fails due to lack of
matching meta trigger). With these patches applies, the
ubuntu-regression-testsuite will be correctly skipped / not attempted to
be executed, as for these triggers we only care about the rebuild test.

Note this is currently blocking velocity of updating the toolchain in
Disco. And there is _no_ correct syntax to override this in hints. As we
do want to see the results of the rebuild test, and block toolchain
migration if rebuild test fails. (It currently passes, but flagged up as
false negative).

This has costed my engineering time to type this email, and a release
team member to investigate this and flag it up. Yet, it could have
sailed through automatically with the patches I have proposed.

http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#gcc-7

http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html#gcc-8

> One thing, we do require your signed-off-by for kernel patches. I
> presume you are okay with us adding that when we apply the patches?

Yes please, and sorry for not including those.

> Aside from that:
>
> Acked-by: Seth Forshee <seth.forshee at canonical.com>
>
> Thanks,
> Seth
>
>>
>> It is hard for me to build a full abi bump kernel, with a matching
>> bumped meta, and signed variants, so I monkey patched these tests into
>> various trees, and tried to run them with appropriate envrionment
>> variable as seen in britney results.
>>
>> Here are new things:
>>
>> $ cd linux-azure-4.18.0/ && ./debian/tests/ubuntu-regression-suite
>> ubuntu-regression-suite is pointless, if one cannot boot the kernel
>> $ echo $?
>> 77
>>
>> $ cd linux-5.0.0/ && ADT_TEST_TRIGGERS=binutils/2.32-5ubuntu1 ./debian/tests/ubuntu-regression-suite
>> ubuntu-regression-suite is not requested, as there is no linux-meta trigger
>> $ echo $?
>> 77
>>
>> I also validated that the current expected tests do run correctly, e.g.:
>>
>> $ ADT_TEST_TRIGGERS=binutils/2.32-5ubuntu1 ./debian/tests/rebuild
>> dpkg-buildpackage: info: source package linux
>> dpkg-buildpackage: info: source version 5.0.0-7.8
>> dpkg-buildpackage: info: source distribution disco
>> ...
>> (build attempted)
>>
>> $ ADT_TEST_TRIGGERS=linux-meta/5.0.0.7.8 ./debian/tests/ubuntu-regression-suite
>> Source Package Version: 5.0.0-7.8
>> Running Kernel Version: 4.19.0-13.14
>> ERROR: running version does not match source package
>> ...
>> (well, i did fake the trigger, but yeah, the right kernel is pulled
>> in, in this case, and the test will be run)
>>
>> Please review these patches and apply them against all kernel
>> flavours, in all releases.
>>
>> Dimitri John Ledkov (5):
>>   Set +x on rebuild testcase.
>>   Skip rebuild test, for regression-suite deps.
>>   Make ubuntu-regression-suite skippable on unbootable kernels.
>>   make rebuild use skippable error codes when skipping.
>>   Only run regression-suite, if requested to.
>>
>>  debian/tests/control                 |  4 ++--
>>  debian/tests/rebuild                 |  4 ++--
>>  debian/tests/ubuntu-regression-suite | 26 ++++++++++++++++++++++++++
>>  3 files changed, 30 insertions(+), 4 deletions(-)
>>  mode change 100644 => 100755 debian/tests/rebuild
>>
>> --
>> 2.20.1
>>
>>
>> --
>> kernel-team mailing list
>> kernel-team at lists.ubuntu.com
>> https://lists.ubuntu.com/mailman/listinfo/kernel-team
>
>





-- 
Regards,

Dimitri.