[Bug 1526984] Re: ISST-LTE: root mpath device unavailable after installation

Fri Feb 5 00:06:08 UTC 2016

Hello bugproxy, or anyone else affected,

Accepted multipath-tools into trusty-proposed. The package will build
now and be available at https://launchpad.net/ubuntu/+source/multipath-
tools/0.4.9-3ubuntu7.8 in a few hours, and then in the -proposed
repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to
enable and use -proposed.  Your feedback will aid us getting this update
out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: multipath-tools (Ubuntu Trusty)
       Status: Incomplete => Fix Committed

** Tags added: verification-needed

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1526984

Title:
  ISST-LTE: root mpath device unavailable after installation

Status in multipath-tools package in Ubuntu:
  Fix Released
Status in multipath-tools source package in Trusty:
  Fix Committed

Bug description:
  [Impact]
  This affects users of multipath systems where devices may be slow to be detected. (See below).

  On some systems disks may be slow to be detected due to a number of
  reasons: slow disks, spin delay, large number of devices to scan, etc.
  If the disk containing the root filesystem fails to be
  detected/scanned when the multipath initramfs scripts are run, the
  system will give up and fail to load the rootfs, which results in a
  failure to boot (and the user will be thrown into a busybox shell in
  the initramfs).

  [Test case]
  Boot the system.

  [Regression Potential]
  Given that this changes the load order for the system, it may cause some delay in booting due to the extra time taken to load all maps for the rootfs to become available through multipath. Also, crashes in multipathd may now affect early boot.

  ---

  Canonical FYI:

  We appear to have a regression in the current 14.04 environment back to
  the behavior originally outlined in Comment #10 of LP Bug 1429327.

  ---Problem---

  A multipath install of 14.04.04 fails to create the install mpath
  device on subsequent boots.  While the root filesystem ends up being
  mounted on top of one of the path devices, other partitions on that
  multipath device will not be automatically found.

  ---Additional details---

  For one example host, the install disk was set up with a root and
  swap partition:

  Dec 14 21:37:55 disk-detect: create: mpath0 (35000c5007655a10f) undef IBM     ,ST300MP0064
  Dec 14 21:37:55 disk-detect: size=279G features='0' hwhandler='0' wp=undef
  Dec 14 21:37:55 disk-detect: |-+- policy='round-robin 0' prio=1 status=undef
  Dec 14 21:37:55 disk-detect: | `- 0:0:7:0 sdc 8:32 undef ready running
  Dec 14 21:37:55 disk-detect: `-+- policy='round-robin 0' prio=1 status=undef
  Dec 14 21:37:55 disk-detect:   `- 1:0:3:0 sdd 8:48 undef ready running

  The resulting fstab entries after install were:

  /dev/mapper/mpath0-part2 /               ext4    errors=remount-ro 0 1
  /dev/mapper/mpath0-part3 none            swap    sw                0 0

  In post-install boots, the multipath command appears to be executed
  prior to any disks being discovered:

  Begin: Loading essential drivers ... done.
  Begin: Running /scripts/init-premount ... done.
  Begin: Mounting root file system ... [    0.612955] ipr: IBM Power RAID SCSI Device Driver version: 2.6.1 (March 12, 2015)
  [    0.612985] ipr 0000:80:00.0: Found IOA with IRQ: 0
  [    0.613478] ipr 0000:80:00.0: ibm,query-pe-dma-windows(53) 800000 8000000 20000164 returned 0
  Begin: Running /scripts/local-top ... [    0.613727] ipr 0000:80:00.0: ibm,create-pe-dma-window(54) 800000 8000000 20000164 10 21 returned 0 (liobn = 0x70000164 starting addr = 8000000 0)
  Begin: Loading multipath modules ... [    0.618790] device-mapper: multipath: version 1.9.0 loaded
  Success: loaded module dm-multipath.
  Failure: failed to load module dm-emc.
  done.
  Begin: Waiting for scsi storage ... done.
  Begin: Discovering multipaths ... [    0.623429] be2net 0002:01:00.0: be2net version is 10.6.0.2
  [    0.623587] be2net 0002:01:00.0: enabling device (0000 -> 0002)
  done.
  [    0.636785] ipr 0000:80:00.0: Using 64-bit direct DMA at offset 800000000000000
  [    0.637343] be2net 0002:01:00.0: PCIe error reporting enabled
  [    0.641435] ipr 0000:80:00.0: Received IRQ : 20
  [    0.641522] ipr 0000:80:00.0: Request for 2 MSIXs succeeded.
  [    0.645713] ipr 0000:80:00.0: Starting IOA initialization sequence.
  [    0.645719] scsi host0: IBM 0 Storage Adapter
  [    0.645741] ipr 0000:80:00.0: Starting IOA initialization sequence.
  [    0.645833] ipr 0001:a0:00.0: Found IOA with IRQ: 0
  [    0.646221] ipr 0000:80:00.0: Adapter firmware version: 13510C00
  [    0.647673] ipr 0000:80:00.0: IOA initialized.
  [    0.648009] ipr 0001:a0:00.0: Received IRQ : 448
  [    0.648041] ipr 0001:a0:00.0: Request for 2 MSIXs succeeded.
  [    0.651764] ipr 0001:a0:00.0: Starting IOA initialization sequence.
  [    0.651769] scsi host1: IBM 0 Storage Adapter
  [    0.651795] ipr 0001:a0:00.0: Starting IOA initialization sequence.
  [    0.652262] ipr 0001:a0:00.0: Adapter firmware version: 13510C00
  [    0.653702] ipr 0001:a0:00.0: IOA initialized.
  [    0.660800] scsi 0:3:0:0: No Device         IBM      57B1001SISIOA    0150 PQ: 0 ANSI: 0
  [    0.660811] scsi 0:3:0:0: Resource path: 0/FE
  [    0.672518] scsi 0:0:0:0: Direct-Access     IBM      ST300MP0064      7D0E PQ: 0 ANSI: 6
  [    0.672530] scsi 0:0:0:0: Resource path: 0/00-0E-08
  ...
  (first device in mpath0 seen at this point):
  [    0.710988] scsi 1:0:3:0: Direct-Access     IBM      ST300MP0064      7D0E PQ
  : 0 ANSI: 6
  [    0.710994] scsi 1:0:3:0: Resource path: 1/00-0E-0A

  The system ends up mounting the root filesystem from /dev/sdc2,
  and the post-mount run of multipath errors out creating mpath0:

  [   10.056286] device-mapper: table: 252:0: multipath: error getting device
  [   10.082644] device-mapper: table: 252:0: multipath: error getting device
  [   10.192614] device-mapper: table: 252:2: multipath: error getting device
  [   10.222609] device-mapper: table: 252:2: multipath: error getting device
  [   10.663458] device-mapper: table: 252:2: multipath: error getting device
  [   10.682690] device-mapper: table: 252:2: multipath: error getting device
  [   10.796743] device-mapper: table: 252:2: multipath: error getting device

  Since no other filesystems depend on mpath0 for this example config,
  the system is able to boot, but lacks swap due to the missing swap
  device.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1526984/+subscriptions