Upgrade of autopkgtest on autopkgtest.ubuntu.com affecting test results

[Paride Legovini]

Hi ubuntu-release,

A recent update has been done on autopkgtest.ubuntu.com, transitioning
from autopkgtest 5.28 to 5.34. As was the case before, this version
includes a patch enabling tests to run on a dpkg foreign architecture
(typically i386 on amd64), along with several minor infrastructure
related patches. (Efforts are underway to upstream these patches: we aim
at having a minimal diff, and eventually no diff at all.)

It is important to note that this update may impact autopkgtest results.

Before this change, some dep8 test restrictions (as in Restrictions: in
d/t/control) were not supported, resulting in skipped tests. Now that
all restrictions are supported, previously skipped tests will get
executed and may fail, thereby switching the reference result for a
package from "pass" or "neutral" to "fail". Consequently, triggering a
migration-reference/0 may be necessary to reset the baseline and let
packages migrate. This is particularly relevant for SRUs, where
stability of the reference result is typically expected but may not be
guaranteed in this case.

The fingerprint of this scenario is the presence of a log message like:

autopkgtest [23:04:07]: @@@@@@@@@@@@@@@@@@@@ summary
sometest          SKIP unknown restriction <restriction-name>

in the reference (= old) test run output, and then the failure
of "sometest" in a recent test run, e.g.

autopkgtest [23:04:07]: @@@@@@@@@@@@@@@@@@@@ summary
sometest          FAIL non-zero exit status 1

We are reachable on IRC for assistance and clarification.

On behalf of the Ubuntu Release Management team,

--
Paride