Review of current autopkgtest failures, bucket into teams

[Dmitry Shachnev]

Hi,

On Thu, Jul 25, 2013 at 3:34 PM, Martin Pitt <martin.pitt at ubuntu.com> wrote:
> Hello all,
>
> in the past few weeks we got quite a number of autopkgtest
> regressions, as well as a bunch of new tests added which either never
> worked, or only worked once, etc. I usually review every failure
> whether it's caused by infrastructure and also fix quite a few which
> e. g. come in through Debian, but I can only keep up so much. So I
> think we need to find a better process for this.
>
> For some failures it is quite easy to spot a particular person who
> should own this, but for others that's not the case. For example,
> ubuntu-release-upgrader hasn't changed since raring (and succeeded
> there), now it's failing because update-manager changed. The
> autopkgtests are usually meant to hold back these cases, but we have
> piled up quite a lot of release team overrides by now, which subvert
> that process (that's not meant as a criticism per se, I know that we
> sometimes need to push stuff through).
>
> I reviewed the current list of failures and bucketed them into
> team/person, and also noted whether that test never succeeded or
> regressed. The latter are particularly worrysome and ought to be fixed
> ASAP, for the former we could discuss whether nobody cares about this
> package (any more) and we just drop the test (examples:
> unity-firefox-extension or oneconf).
>
> For the team-owned ones I propose that the relevant team's tech lead
> is a default fallback if there is no clear owner, and fans out these
> to developers?
>
> For the broken ones from Debian, we could maybe do some community
> rallying / hack day. If anyone is interested in fixing them, please
> stop by #ubuntu-quality and I'm happy to provide
> guidance/testing/sponsoring.
>
> Thanks,
>
> Martin
>
> == broken import from Debian ==
> saucy-adt-abi-compliance-checker (never worked)
> saucy-adt-beets (regression)
> saucy-adt-libgtkada (never worked)
> saucy-adt-plv8 (never worked)
> saucy-adt-postgresql-pgmp (never worked)
> saucy-adt-pyparsing (broken test metadata, never worked)
> saucy-adt-python-csb (never worked)
> saucy-adt-s3ql (never worked)
>
> == problem with infrastructure (QA owned) ==
> saucy-adt-adequate (we don't support "breaks-testbed")
> saucy-adt-linux (VM is too small)
> saucy-adt-ubuntuone-client (supposed to work now, problem with "allow-stderr")
>
> == foundations ==
> saucy-adt-apt-clone (regression)
> saucy-adt-eglibc (rebuild test FTBFS)
> saucy-adt-lintian (regression)
> saucy-adt-pandas (regression)
> saucy-adt-python2.7 (never worked on amd64, ok on i386)
> saucy-adt-python3.3 (never worked on amd64, ok on i386)
> saucy-adt-ubuntu-release-upgrader (regression)
>
> == desktop ==
> saucy-adt-colord (never worked)
> saucy-adt-firefox (regression)
> saucy-adt-friends (regression)
> saucy-adt-libreoffice (regression)
> saucy-adt-notify-osd (regression)
> saucy-adt-oneconf (never worked)
> saucy-adt-software-center (never worked)
> saucy-adt-unity-firefox-extension (never worked)
>
> == server ==
> saucy-adt-lxc (regression)
> saucy-adt-maas-cluster-controller (never worked)
> saucy-adt-maas-manual (never worked)
> saucy-adt-mysql-5.5 (just one test failing, never worked)
> saucy-adt-nagios3 (could be a python-simplejson regression on amd64, regression)
>
> == uploaded by particular developer ==
> saucy-adt-dirspec (never worked, dobey)
> saucy-adt-freeradius (regression, yolanda)
> saucy-adt-gvfs (regression, pitti)
> saucy-adt-heat (never worked, yolanda)
> saucy-adt-mailman (regression, yolanda)
> saucy-adt-quantum (regression, yolanda)
> saucy-adt-squid3 (never worked on amd64, i386 ok, yolanda)

In case of pyparsing, the tests were added as Ubuntu delta, but then
were dropped by syncing a new version from Debian. Looks like Jenkins
isn't handling such cases properly (I guess there are more common
cases where it'll fail currently, like package renaming).

Pandas test is fixed now.

--
Dmitry Shachnev