[Bug 1829450] Re: test_systemd_fsck_with_plymouth_failure passes but marked expected failure

[Dimitri John Ledkov]

"and so causes autopkgtest failures." is not true.

>From http://autopkgtest.ubuntu.com/packages/systemd/bionic/amd64
E.g. https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/s/systemd/20190529_090547_1f40a@/log.gz

Ensure that a failing plymouth doesn't prevent fsckd to reconnect/exit
... unexpected success

FAILED (unexpected successes=1)

autopkgtest [09:05:38]: test systemd-fsckd: -----------------------]
systemd-fsckd        PASS

Note systemd-fsckd only fails on failures & errors, and passed with unexpected successes:
if len(result.failures) != 0 or len(result.errors) != 0:
    return_code = 1

I also wonder what has changed to make this test case reliable, in the
past it used to be flakey, that's why i switched it to expectedfailure.
I guess it's ok to reenable this in devel.

I see no need to reenable it in stable series. It has a regression
potential, since it used to be flakey, it may start to be flakey again
blocking SRU migrations.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1829450

Title:
  test_systemd_fsck_with_plymouth_failure passes but marked expected
  failure

Status in systemd package in Ubuntu:
  In Progress
Status in systemd source package in Xenial:
  Invalid
Status in systemd source package in Bionic:
  In Progress
Status in systemd source package in Cosmic:
  In Progress
Status in systemd source package in Disco:
  In Progress
Status in systemd source package in Eoan:
  In Progress

Bug description:
  [impact]

  this test case was marked as expected failure in the past to try to
  ignore failures, but it now passes and so causes autopkgtest failures.

  [test case]

  run autopkgtest and check for the
  'test_systemd_fsck_with_plymouth_failure' test, e.g.:

  test_systemd_fsck_with_plymouth_failure (__main__.FsckdTest)
  Ensure that a failing plymouth doesn't prevent fsckd to reconnect/exit ... bash: line 1:   786 Killed                  /tmp/autopkgtest.84SCbj/build.1op/src/debian/tests/systemd-fsckd 2> >(tee -a /tmp/autopkgtest.84SCbj/systemd-fsckd-stderr >&2) > >(tee -a /tmp/autopkgtest.84SCbj/systemd-fsckd-stdout)
  autopkgtest [16:47:57]: test process requested reboot with marker test_systemd_fsck_with_plymouth_failure:0
  Connection to 10.44.40.6 closed by remote host.
  autopkgtest-virt-ssh: WARNING: ssh connection failed. Retrying in 3 seconds...
  test_systemd_fsck_with_plymouth_failure (__main__.FsckdTest)
  Ensure that a failing plymouth doesn't prevent fsckd to reconnect/exit ... unexpected success

  ----------------------------------------------------------------------
  Ran 1 test in 17.952s

  FAILED (unexpected successes=1)

  [regression potential]

  low; test case expectation fix only.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1829450/+subscriptions