[Bug 1892358] Re: autopkgtest success rate dropped inhibiting proposed migration

[Christian Ehrhardt ]

Thank you Rbalint.

Thanks to xnox I already found 1891716 and 1886886 which are sort of
remotely related (to the next upload), I guess this bug will serve for
the "marking systemd-fsckd test as flaky" task then.

Can we do something about this bad fstab in LXD or is there a bug to
track?

Essentially I'd like to know when it overall is worth again to hit retry
on my tests :-)

** Also affects: qemu (Ubuntu)
   Importance: Undecided
       Status: New

** Tags added: update-excuses

** Tags removed: update-excuses
** Tags added: update-excuse

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1892358

Title:
  autopkgtest success rate dropped inhibiting proposed migration

Status in kbd package in Ubuntu:
  Confirmed
Status in qemu package in Ubuntu:
  Confirmed
Status in systemd package in Ubuntu:
  New

Bug description:
  Hi,
  we had such cases in the past like bug 1817721 for bionic and maybe bug 1892130 is about the same as well. There were more but I didn't want to search for all of them - what I checked is that there are no open ones clearly pointing out the recent further drop in already flaky subtests.

  In particular the tests "tests-in-lxd" and "systemd-fsckd" were known
  to be flaky before, but got even worse.

  Here stats of the last 40 runs, it might be a coincidences that this
  is after 246-2ubuntu1 landed. Could as well be any other change

  groovy
    amd64
  tests-in-lxd                   (F 42% S  0% B 10% => P 45%/) FFFFBFFFFFFFB....FF.B.....F.....F...FBF
  build-login                    (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  unit-config                    (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  networkd-testpy                (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  boot-and-services              (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  boot-smoke                     (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  logind                         (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  storage                        (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  upstream                       (F 35% S  0% B 10% => P 52%/) ..FFB.FFF.FFB....FF.B.....F.F..F....FBF
  udev                           (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
  systemd-fsckd                  (F 37% S  0% B 10% => P 50%/) FFFFBFFFFFFFB.FF...FB.....F..........B.
  root-unittests                 (F  0% S  0% B 10% => P 87%/) ....B.......B.......B................B.
    ppc64el
  tests-in-lxd                   (F 25% S  0% B  0% => P 75%/) FFFF....FF............FFF.....F.........
  systemd-fsckd                  (F 35% S  0% B  0% => P 65%/) FFFFFFF...FF........F....FF.F..F........
  root-unittests                 (F  2% S  0% B  0% => P 97%/) ..............................F.........
    s390x
  tests-in-lxd                   (F 52% S  0% B  0% => P 47%/) FFFFFFF.FFFFFFF.FF.........FFFF...F.....
  timedated                      (F  2% S  0% B  0% => P 97%/) ...........F............................
  upstream                       (F 17% S  0% B  0% => P 82%/) .....F......F.F.............FFF...F.....
  systemd-fsckd                  (F 32% S  0% B  0% => P 67%/) FFFFFFF..FF..F.................FF..F....
  root-unittests                 (F 10% S  0% B  0% => P 90%/) ............................FFF...F.....
    arm64
  tests-in-lxd                   (F 40% S  0% B  2% => P 57%/) FFFFF.B...FFF.FF..F..F.........FFF.F....
  logind                         (F  2% S  0% B  2% => P 95%/) ......B...................F.............
  upstream                       (F 22% S  0% B  2% => P 75%/) ...F.FB.....F.F.............F..FFF.F....
  root-unittests                 (F 12% S  0% B  2% => P 85%/) ......B.F...........F.F........F...F....

  (I'm sure LP will make this unreadable, but is is nice in monospace)

  Whatever the root cause is - the success rate of these has reduced so
  much that the (even formerly questionable) practice of retry-until-
  success won't work anymore.

  
  I have run the two tests in a local VM and systemd-fsckd works there while tests-in-lxd seems to trip over the old flaky fellow being "boot-and-services".

  We had the discussion in the past, but I think I need to again bring
  up the suggestion to skip "tests-in-lxd" and "systemd-fsckd" until
  they are on reasonable success rates.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/kbd/+bug/1892358/+subscriptions