[Bug 818177] Re: boot failures because 'udevadm exit' does not kill udevd worker threads

[Steve Langasek]

I've just pushed a new udev package to ppa:vorlon/ppa which should
provide better debugging capabilities.  It unconditionally runs 'udevadm
monitor -e' before starting udevd, and spits out the complete list of
kernel/udev events after udevd is stopped in the initramfs - as well as
spitting out a process list with 'ps -ef' so we can see what's been left
behind that's still holding /dev open and preventing the mount -omove.

Anyone who's reproducing these boot failures (other than the LVM
vgchange race case), please test with this updated package (~ppa3) and
attach the full console output to the bug.  If you can also attach the
full console output to the bug from a successful boot, that will be
useful for comparison.

A few comments in summary about the things that have already been tried that didn't work:
 pkill after udevadm control --exit -- this was a distraction.  The ps output I was looking at when I jumped to this conclusion was the output from running ps *before* running --exit, so it doesn't really tell us anything - especially since 'pkill udevd' vs 'udevadm control --exit' have nearly identical behavior, with the difference that udevadm is designed to wait for udevd to shut down before returning and the pkill call *may* catch processes that udevd has somehow managed to lose track of (a possible, but not likely, scenario).  The new test package should tell us more, by showing us only those processes still running *after* udevadm is called.

ps -ef not returning anything when run after the mount call (comment
#75) - yes, because the initramfs script in question is 'set -e'; so as
soon as the mount -omove call fails, the script terminates.  We should
grab the ps output before the mount call - this is done in the test
package.

non-empty /dev on the root filesystem - this will definitely mask the
problem in many cases.  For ease of reproducing the issue, best to do
the following from the booted system:

  sudo mount -obind / /mnt
  sudo rm -rf /mnt/dev/*
  sudo umount /mnt

thus ensuring /dev is empty and that any failure to move the /dev mount
in the initramfs will be caught early.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to udev in Ubuntu.
https://bugs.launchpad.net/bugs/818177

Title:
  boot failures because 'udevadm exit' does not kill udevd worker
  threads

Status in Release Notes for Ubuntu:
  Triaged
Status in “linux” package in Ubuntu:
  Invalid
Status in “udev” package in Ubuntu:
  Triaged
Status in “linux” source package in Oneiric:
  Invalid
Status in “udev” source package in Oneiric:
  Triaged
Status in “linux” source package in Precise:
  Invalid
Status in “udev” source package in Precise:
  Triaged

Bug description:
  After running a do-release-upgrade -d on a natty server, I'm unable to
  boot the machine properly.

  It looks like it doesn't see the local disks anymore.

  I think this looks quite a lot like the debbug linked from here:
  https://bugs.launchpad.net/ubuntu/oneiric/+source/linux/+bug/746751

  ProblemType: Bug
  DistroRelease: Ubuntu 11.10
  Package: linux-image-3.0.0-7-server 3.0.0-7.8
  ProcVersionSignature: Ubuntu 3.0.0-7.8-server 3.0.0
  Uname: Linux 3.0.0-7-server x86_64
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 2011-07-29 13:34 seq
   crw-rw---- 1 root audio 116, 33 2011-07-29 13:34 timer
  AplayDevices: Error: [Errno 2] No such file or directory
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CRDA: Error: [Errno 2] No such file or directory
  Date: Fri Jul 29 13:40:10 2011
  HibernationDevice: RESUME=UUID=844d17fa-ff69-4250-844b-33b930236c56
  IwConfig:
   lo        no wireless extensions.
   
   eth0      no wireless extensions.
   
   eth1      no wireless extensions.
  MachineType: HP ProLiant DL380 G5
  PciMultimedia:
   
  ProcEnviron:
   LANGUAGE=en_US:
   PATH=(custom, user)
   LANG=en_US
   SHELL=/bin/bash
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.0.0-7-server root=/dev/mapper/User Name-root ro INIT_VERBOSE=yes
  RelatedPackageVersions:
   linux-restricted-modules-3.0.0-7-server N/A
   linux-backports-modules-3.0.0-7-server  N/A
   linux-firmware                          1.56
  RfKill: Error: [Errno 2] No such file or directory
  SourcePackage: linux
  UpgradeStatus: Upgraded to oneiric on 2011-07-29 (0 days ago)
  dmi.bios.date: 06/28/2007
  dmi.bios.vendor: HP
  dmi.bios.version: P56
  dmi.chassis.type: 23
  dmi.chassis.vendor: HP
  dmi.modalias: dmi:bvnHP:bvrP56:bd06/28/2007:svnHP:pnProLiantDL380G5:pvr:cvnHP:ct23:cvr:
  dmi.product.name: ProLiant DL380 G5
  dmi.sys.vendor: HP

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-release-notes/+bug/818177/+subscriptions