[Bug 1583979] [NEW] test fails randomly if two Tests: blocks are used in control files

Fri May 20 09:04:50 UTC 2016

Public bug reported:

When two Tests: blocks are used in the debian/tests/control file of the
kdump-tools DEP8 tests, the test-crashdump test fails on random
occasions.

Here is the content of the control file :

#Tests: install test-crashdump                                                                                                
Tests: install                                                                                                                
Depends: linux-crashdump                                                                                                      
Restrictions: needs-root, isolation-machine, allow-stderr                                                                     

Tests: test-crashdump                                                                                                         
Depends: linux-crashdump                                                                                                      
Restrictions: needs-root, isolation-machine, allow-stderr                                                                     

When the following d/t/control file is used, the test-crashdump test
will work each time :

Tests: install test-crashdump                                                                                                
Depends: linux-crashdump                                                                                                      
Restrictions: needs-root, isolation-machine, allow-stderr                                                                     

Description of the DEP8 tests:
==============================
The 'install' test will verify that installation and configuration of kdump-tools and makedumpfile have been done correctly according to the 'linux-crashdump' dependancy of the test. It will then call /tmp/autopkgtest-reboot with the enable-crashkernel marker.

The second test 'test-crashdump' will set the ADT_REBOOT_MARK to local
and trigger a kernel crash dump. This is the test that fails when two
Tests: blocks are present. Most of the time it fails by not being able
to mount Root and drops to rescue.

If only one Tests: block is used, the kernel crash dump will run kdump-
tools that will trigger makedumpfile and reboot to complete the test.

How to reproduce the problem:
=============================
$ dget https://launchpad.net/~louis-bouchard/+archive/ubuntu/kdump-dep8-failure/+files/makedumpfile_1.5.9-6~dep8.dsc
$ adt-buildvm-ubuntu-cloud
$ cd makedumpfile-1.5.9
$ adt-run --unbuilt-tree $(pwd) --- qemu --show-boot ../../adt-yakkety-amd64-cloud.img

Here is a capture of a failure :

[  OK  ] Mounted Huge Pages File System.                                                                             [15/1787]
[    2.625656] systemd[1]: Mounted Debug File System.
[  OK  ] Mounted Debug File System.
[    2.627135] systemd[1]: Started Journal Service.
[  OK  ] Started Journal Service.
[FAILED] Failed to start Remount Root and Kernel File Systems.
See 'systemctl status systemd-remount-fs.service' for details.
[  OK  ] Started Create list of required sta...ce nodes for the current kernel.
[  OK  ] Started Set console scheme.
...
[FAILED] Failed to start Create Volatile Files and Directories.
See 'systemctl status systemd-tmpfiles-setup.service' for details.
[  OK  ] Started Set console font and keymap.
[  OK  ] Started Tell Plymouth To Write Out Runtime Data.
         Starting Update UTMP about System Boot/Shutdown...
[FAILED] Failed to start Network Time Synchronization.
See 'systemctl status systemd-timesyncd.service' for details.

The test will evenutally timeout.

It is worth mentionning that, while debugging the issue, I have often
seen cases where by removing the --show-boot and attaching to the TTYS0
console with minicom on the second QEMU process that does the kernel
crash, the test will succeed even with two Tests: blocks.

Might be useful to mention that, when two Tests: blocks are used, two
qemu processes are executed.

I was able to get one successful test run with two Tests: block by
attaching to the console while writing up this bug and two failures w/o
console attached, with or without the --show-boot

** Affects: autopkgtest (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to autopkgtest in Ubuntu.
https://bugs.launchpad.net/bugs/1583979

Title:
  test fails randomly if two Tests: blocks are used in control files

Status in autopkgtest package in Ubuntu:
  New

Bug description:
  When two Tests: blocks are used in the debian/tests/control file of
  the kdump-tools DEP8 tests, the test-crashdump test fails on random
  occasions.

  Here is the content of the control file :

  #Tests: install test-crashdump                                                                                                
  Tests: install                                                                                                                
  Depends: linux-crashdump                                                                                                      
  Restrictions: needs-root, isolation-machine, allow-stderr                                                                     

  Tests: test-crashdump                                                                                                         
  Depends: linux-crashdump                                                                                                      
  Restrictions: needs-root, isolation-machine, allow-stderr                                                                     

  When the following d/t/control file is used, the test-crashdump test
  will work each time :

  Tests: install test-crashdump                                                                                                
  Depends: linux-crashdump                                                                                                      
  Restrictions: needs-root, isolation-machine, allow-stderr                                                                     

  Description of the DEP8 tests:
  ==============================
  The 'install' test will verify that installation and configuration of kdump-tools and makedumpfile have been done correctly according to the 'linux-crashdump' dependancy of the test. It will then call /tmp/autopkgtest-reboot with the enable-crashkernel marker.

  The second test 'test-crashdump' will set the ADT_REBOOT_MARK to local
  and trigger a kernel crash dump. This is the test that fails when two
  Tests: blocks are present. Most of the time it fails by not being able
  to mount Root and drops to rescue.

  If only one Tests: block is used, the kernel crash dump will run
  kdump-tools that will trigger makedumpfile and reboot to complete the
  test.

  How to reproduce the problem:
  =============================
  $ dget https://launchpad.net/~louis-bouchard/+archive/ubuntu/kdump-dep8-failure/+files/makedumpfile_1.5.9-6~dep8.dsc
  $ adt-buildvm-ubuntu-cloud
  $ cd makedumpfile-1.5.9
  $ adt-run --unbuilt-tree $(pwd) --- qemu --show-boot ../../adt-yakkety-amd64-cloud.img

  Here is a capture of a failure :

  [  OK  ] Mounted Huge Pages File System.                                                                             [15/1787]
  [    2.625656] systemd[1]: Mounted Debug File System.
  [  OK  ] Mounted Debug File System.
  [    2.627135] systemd[1]: Started Journal Service.
  [  OK  ] Started Journal Service.
  [FAILED] Failed to start Remount Root and Kernel File Systems.
  See 'systemctl status systemd-remount-fs.service' for details.
  [  OK  ] Started Create list of required sta...ce nodes for the current kernel.
  [  OK  ] Started Set console scheme.
  ...
  [FAILED] Failed to start Create Volatile Files and Directories.
  See 'systemctl status systemd-tmpfiles-setup.service' for details.
  [  OK  ] Started Set console font and keymap.
  [  OK  ] Started Tell Plymouth To Write Out Runtime Data.
           Starting Update UTMP about System Boot/Shutdown...
  [FAILED] Failed to start Network Time Synchronization.
  See 'systemctl status systemd-timesyncd.service' for details.

  The test will evenutally timeout.

  It is worth mentionning that, while debugging the issue, I have often
  seen cases where by removing the --show-boot and attaching to the
  TTYS0 console with minicom on the second QEMU process that does the
  kernel crash, the test will succeed even with two Tests: blocks.

  Might be useful to mention that, when two Tests: blocks are used, two
  qemu processes are executed.

  I was able to get one successful test run with two Tests: block by
  attaching to the console while writing up this bug and two failures
  w/o console attached, with or without the --show-boot

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/autopkgtest/+bug/1583979/+subscriptions