[Bug 1027346] [NEW] mdadm race condition causes drop to busybox prompt

[James E. Blair]

Public bug reported:

Description:	Ubuntu 12.04 LTS
Release:	12.04

There seems to be a race condition with mdadm in initramfs-tools.  When
my server boots, I get a "Dropping to shell." message and a busybox
prompt, but no explanation whatsoever for why that happened.

The local-premount/mdadm script does the following:

  degraded_arrays || exit 0
  mountroot_fail || panic "Dropping to a shell."

degraded_arrays calls mdadm and checks its exit code.  mountroot_fails
also calls degraded_arrays, which calls mdadm again.  By changing the
degraded_arrays function to echo the exit code from mdadm, I determined
that on my server, the first call to mdadm from degraded_arrays returned
4, but the second call from mountroot_fail returned 0.  mountroot_fail
uses its call to determine whether to output a message about why the
user is being dropped into a shell and whether to try to boot in
degraded mode.  If it's called, but mdadm returns 0, mountroot_fail
always exits with an error but no explanation.

It seems very likely that this is a race condition since the two calls
to mdadm, which are _very_ close together return different values, with
no intervening work having been done by initramfs-tools.

I have corrected this locally by adding a local-top/ script that invokes
"wait_for_udev 10".

It seems like there are two issues that should be corrected:

1) The mdadm/udev race condition -- initramfs-tools should have a more
solid method of being sure that all the mdadm work is done before it
checks if the array is degraded.

2) The lack of explanation for being dropped at a busybox prompt.  The
code path that checks for a degraded array, checks again, and drops the
user at a prompt with no explanation if the values are _different_ makes
little sense.

** Affects: mdadm
     Importance: Undecided
         Status: New

** Affects: initramfs-tools (Ubuntu)
     Importance: Undecided
         Status: New

** Also affects: mdadm
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to initramfs-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1027346

Title:
  mdadm race condition causes drop to busybox prompt

Status in mdadm - Tool for managing linux software RAID arrays.:
  New
Status in “initramfs-tools” package in Ubuntu:
  New

Bug description:
  Description:	Ubuntu 12.04 LTS
  Release:	12.04

  There seems to be a race condition with mdadm in initramfs-tools.
  When my server boots, I get a "Dropping to shell." message and a
  busybox prompt, but no explanation whatsoever for why that happened.

  The local-premount/mdadm script does the following:

    degraded_arrays || exit 0
    mountroot_fail || panic "Dropping to a shell."

  degraded_arrays calls mdadm and checks its exit code.  mountroot_fails
  also calls degraded_arrays, which calls mdadm again.  By changing the
  degraded_arrays function to echo the exit code from mdadm, I
  determined that on my server, the first call to mdadm from
  degraded_arrays returned 4, but the second call from mountroot_fail
  returned 0.  mountroot_fail uses its call to determine whether to
  output a message about why the user is being dropped into a shell and
  whether to try to boot in degraded mode.  If it's called, but mdadm
  returns 0, mountroot_fail always exits with an error but no
  explanation.

  It seems very likely that this is a race condition since the two calls
  to mdadm, which are _very_ close together return different values,
  with no intervening work having been done by initramfs-tools.

  I have corrected this locally by adding a local-top/ script that
  invokes "wait_for_udev 10".

  It seems like there are two issues that should be corrected:

  1) The mdadm/udev race condition -- initramfs-tools should have a more
  solid method of being sure that all the mdadm work is done before it
  checks if the array is degraded.

  2) The lack of explanation for being dropped at a busybox prompt.  The
  code path that checks for a degraded array, checks again, and drops
  the user at a prompt with no explanation if the values are _different_
  makes little sense.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mdadm/+bug/1027346/+subscriptions