autopkgtest missed regression for kinetic

[Jeremy Bicha]

On Sun, May 15, 2022 at 4:26 PM Steve Langasek
<steve.langasek at ubuntu.com> wrote:
> > The test has never passed; it has had neutral results and failing results,
> > but by policy, proposed-migration does not treat neutral->fail as a
> > migration-blocking regression, only pass->fail.  Neutral test results are
> > informational only.
>
> Having said this, I am just this moment looking at cases where britney is
> treating neutral->fail as a regression.  I still believe the intended policy
> is as I described, but one way or another there appear to be some bugs here
> in the implementation ;)

devhelp's autopkgtest is marked as "superficial". In Debian, a passing
superficial autopkgtest won't speed up migration to Testing. But if
the test fails, it indicates that there is a serious bug and migration
to Testing is blocked. Please see the description of the intent of
"superficial":

https://salsa.debian.org/ci-team/autopkgtest/-/blob/master/doc/README.package-tests.rst

Thank you,
Jeremy Bicha