[Bug 1550448] Re: [PowerVM] Ubuntu 16.04 does not install bootloader on multiple PReP partitions in Software RAID1 configuration
Mathew Hodson
mathew.hodson at gmail.com
Fri May 20 21:13:59 UTC 2016
** Changed in: grub-installer (Ubuntu Trusty)
Status: New => Triaged
** Tags added: patch
--
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1550448
Title:
[PowerVM] Ubuntu 16.04 does not install bootloader on multiple PReP
partitions in Software RAID1 configuration
Status in grub-installer package in Ubuntu:
Fix Released
Status in grub-installer source package in Trusty:
Triaged
Bug description:
== Comment: #0 - Heitor Ricardo Alves de Siqueira - 2016-02-19 11:37:33 ==
---Problem Description---
When installing Ubuntu 16.04 to a RAID1 powerVM configuration, grub only installs to the first disk. The second disk will not have a valid PReP partition, rendering the system unbootable if the first disk fails in any way.
We are currently seeing this on a RAID1 install to two NVMe adapters.
---uname output---
Linux tul131p1 4.4.0-6-generic #21-Ubuntu SMP Tue Feb 16 20:31:37 UTC 2016 ppc64le GNU/Linux
Machine Type = 8286-42A lpar
---boot type---
kexec vmlinuz/initrd
---Kernel cmdline used to launch install---
default cmdline
---Install repository type---
Internet repository
---Install repository Location---
ports.ubuntu.com
---Point of failure---
Problem during post-install (stage 2) configuration or other problem seen after reboot
== Comment: #1 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 05:31:43 ==
Currently:
1) grub-install lists all PReP partitions in the system
2) and prefers a PReP partition on the same disk as the root/boot filesystem.
3) wipe it and install the bootloader code in it.
Idea:
0) keep that as-is / backward compatible, as it works OK so far, but slightly change a particular case
1) grub-install lists all PReP partitions in the system
2) and prefers a PReP partition on the same disk as the root/boot filesystem.
3) but if the root/boot filesystem is a MD device,
4) find its component disks
5) check which ones have a PReP partition
6) wipe it and install bootloader code in it.
== Comment: #4 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 08:54:04 ==
Hi Canonical,
This bug is a follow on to LP bug 1487365 (LTC BZ bug 129040) in the
area of PReP partitions and Software RAID installations.
@taco-screen-team
I'd suggest for the bug assignee/owner to be @mathieu-tl on this bug too.
== Comment: #5 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 09:00:23 ==
This patch for Xenial adds support in grub-installer for handling multiple PReP partitions on component devices of MD/dmadm devices.
It's backward compatible w/ the existing handling of other device types.
== Comment: #6 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 09:01:01 ==
Debugging logs from the installation:
Syslog:
~ # grep grub-install /var/log/syslog
Feb 26 11:09:15 anna[7389]: DEBUG: retrieving grub-installer 1.128ubuntu2
Feb 26 11:57:48 main-menu[1776]: INFO: Menu item 'grub-installer' selected
Feb 26 11:57:48 grub-installer: info: architecture: ppc64el/chrp_ibm
Feb 26 11:57:52 grub-installer: info: Identified partition label for /dev/md0p2: gpt
Feb 26 11:57:52 grub-installer: info: Wiping PReP partition /dev/sda1
Feb 26 11:57:52 grub-installer: info: Wiping PReP partition /dev/sdb1
Feb 26 11:57:52 grub-installer: info: Wiping PReP partition /dev/sdc1
Feb 26 11:57:53 grub-installer: info: Installing grub on '/dev/sdc1'
Feb 26 11:57:53 grub-installer: info: grub-install does not support --no-floppy
Feb 26 11:57:53 grub-installer: info: Running chroot /target grub-install --force "/dev/sda1"
Feb 26 11:57:54 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 11:58:02 grub-installer: Installation finished. No error reported.
Feb 26 11:58:02 grub-installer: info: grub-install ran successfully
Feb 26 11:58:02 grub-installer: info: grub-install does not support --no-floppy
Feb 26 11:58:02 grub-installer: info: Running chroot /target grub-install --force "/dev/sdb1"
Feb 26 11:58:02 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 11:58:11 grub-installer: Installation finished. No error reported.
Feb 26 11:58:11 grub-installer: info: grub-install ran successfully
Feb 26 11:58:11 grub-installer: info: grub-install does not support --no-floppy
Feb 26 11:58:11 grub-installer: info: Running chroot /target grub-install --force "/dev/sdc1"
Feb 26 11:58:11 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 11:58:20 grub-installer: Installation finished. No error reported.
Feb 26 11:58:20 grub-installer: info: grub-install ran successfully
Shell script tracing:
For the shell tracing, add the following to /usr/bin/grub-install:
exec 2>/grub.debug
set -x
~ # cat /grub.debug
<...>
+ wipe_bootdevs=
+ /usr/lib/grub-installer/prep-bootdev
+ wipe_bootdev=/dev/sda1
+ /usr/lib/grub-installer/prep-bootdev -l
+ [ /dev/sda = /dev/md ]
+ grep -q /dev/md[0-9]\+
+ echo /dev/md0
+ grep /dev/sda
+ grep -o /dev/[sv]d[a-z]\+
+ mdadm --detail --verbose /dev/md0
+ wipe_bootdevs= /dev/sda1
+ [ /dev/sdb = /dev/md ]
+ grep -q /dev/md[0-9]\+
+ echo /dev/md0
+ grep /dev/sdb
+ grep -o /dev/[sv]d[a-z]\+
+ mdadm --detail --verbose /dev/md0
+ wipe_bootdevs= /dev/sda1 /dev/sdb1
+ [ /dev/sdc = /dev/md ]
+ grep -q /dev/md[0-9]\+
+ echo /dev/md0
+ grep /dev/sdc
+ grep -o /dev/[sv]d[a-z]\+
+ mdadm --detail --verbose /dev/md0
+ wipe_bootdevs= /dev/sda1 /dev/sdb1 /dev/sdc1
+ unset prep_p
+ [ -z /dev/sda1 /dev/sdb1 /dev/sdc1 ]
+ [ -n /dev/sda1 ]
+ info Wiping PReP partition /dev/sda1
+ log info: Wiping PReP partition /dev/sda1
+ logger -t grub-installer info: Wiping PReP partition /dev/sda1
+ blockdev --getsz /dev/sda1
+ log-output -t grub-installer dd if=/dev/zero of=/dev/sda1 bs=512 count=14336
+ [ -n /dev/sdb1 ]
+ info Wiping PReP partition /dev/sdb1
+ log info: Wiping PReP partition /dev/sdb1
+ logger -t grub-installer info: Wiping PReP partition /dev/sdb1
+ blockdev --getsz /dev/sdb1
+ log-output -t grub-installer dd if=/dev/zero of=/dev/sdb1 bs=512 count=14336
+ [ -n /dev/sdc1 ]
+ info Wiping PReP partition /dev/sdc1
+ log info: Wiping PReP partition /dev/sdc1
+ logger -t grub-installer info: Wiping PReP partition /dev/sdc1
+ blockdev --getsz /dev/sdc1
+ log-output -t grub-installer dd if=/dev/zero of=/dev/sdc1 bs=512 count=14336
<...>
+ bootdevs=/dev/sdc1
+ bootdevs= /dev/sda1 /dev/sdb1 /dev/sdc1
<...>
+ info Running chroot /target grub-install --force "/dev/sda1"
+ log info: Running chroot /target grub-install --force "/dev/sda1"
+ logger -t grub-installer info: Running chroot /target grub-install --force "/dev/sda1"
+ log-output -t grub-installer chroot /target grub-install --force /dev/sda1
+ [ 0 = 0 ]
+ info grub-install ran successfully
+ log info: grub-install ran successfully
+ logger -t grub-installer info: grub-install ran successfully
<...>
+ info Running chroot /target grub-install --force "/dev/sdb1"
+ log info: Running chroot /target grub-install --force "/dev/sdb1"
+ logger -t grub-installer info: Running chroot /target grub-install --force "/dev/sdb1"
+ log-output -t grub-installer chroot /target grub-install --force /dev/sdb1
+ [ 0 = 0 ]
+ info grub-install ran successfully
+ log info: grub-install ran successfully
+ logger -t grub-installer info: grub-install ran successfully
<...>
+ info Running chroot /target grub-install --force "/dev/sdc1"
+ log info: Running chroot /target grub-install --force "/dev/sdc1"
+ logger -t grub-installer info: Running chroot /target grub-install --force "/dev/sdc1"
+ log-output -t grub-installer chroot /target grub-install --force /dev/sdc1
+ [ 0 = 0 ]
+ info grub-install ran successfully
+ log info: grub-install ran successfully
+ logger -t grub-installer info: grub-install ran successfully
<...>
== Comment: #7 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 09:01:29 ==
Test scenario:
3 individual disks (2 active, 1 spare)
$ for i in 1 2 3; do qemu-img create -f qcow2 disk$i.qcow2 8G; done
Formatting 'disk1.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
Formatting 'disk2.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
Formatting 'disk3.qcow2', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-system-ppc64 -enable-kvm \
-nographic -nodefaults \
-monitor stdio -serial pty \
-M pseries -smp 1,cores=1,threads=1 -m 8G \
\
-device spapr-vscsi \
-drive file=disk1.qcow2 \
-drive file=disk2.qcow2 \
-drive file=disk3.qcow2 \
-drive media=cdrom,file=xenial-server-ppc64el.iso \
\
-device spapr-vlan,netdev=net0,mac=4C:45:42:45:02:03 \
-netdev bridge,id=net0,br=br0 \
-S
1) On the 'Partition disks' dialog, select 'Go back', and 'Execute a
shell'
2) Check the 'grub-installer' script is present
~ # ls -l /usr/bin/grub-installer
-rwxr-xr-x 1 root root 44452 Jan 4 14:05 /usr/bin/grub-installer
3) Download and install the test package
~ # wget http://ausgsa.ibm.com/~mauricfo/public/bugs/bz137572/v1/grub-installer_1.128ubuntu2multiprep1_ppc64el.udeb
~ # udpkg -i *.udeb
<... ignore the messages ...>
4) Verify the patch is present
~ # grep MD/mdadm /usr/bin/grub-installer
# On MD/mdadm devices, each component device may have a PReP partition.
# On MD/mdadm devices, each component device may have a PReP partition.
5) Return to the installation.
~ # exit
6) Proceed normally with a Software RAID based partitioning
*with a PReP partition per component device.*
7) On the 'Installation finished' prompt (no errors expected), select
'Go Back', and 'Execute a shell'
8) Verify all the PReP partitions on the component devices
were covered by grub-installer.
(e.g., notice the 'Wiping PReP partition' and 'chroot /target grub-install' messages)
~ # grep -i grub-install /var/log/syslog
Feb 26 13:19:45 anna[7383]: DEBUG: retrieving grub-installer 1.128ubuntu2
Feb 26 13:22:22 grub-installer: info: architecture: ppc64el/chrp_ibm
Feb 26 13:34:36 main-menu[1769]: INFO: Menu item 'grub-installer' selected
Feb 26 13:34:36 grub-installer: info: architecture: ppc64el/chrp_ibm
Feb 26 13:34:38 grub-installer: info: Identified partition label for /dev/md0p2: gpt
Feb 26 13:34:39 grub-installer: info: Wiping PReP partition /dev/sda1
Feb 26 13:34:39 grub-installer: info: Wiping PReP partition /dev/sdb1
Feb 26 13:34:39 grub-installer: info: Wiping PReP partition /dev/sdc1
Feb 26 13:34:55 grub-installer: info: Installing grub on '/dev/sdc1'
Feb 26 13:34:55 grub-installer: info: grub-install does not support --no-floppy
Feb 26 13:34:55 grub-installer: info: Running chroot /target grub-install --force "/dev/sda1"
Feb 26 13:34:55 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 13:35:04 grub-installer: Installation finished. No error reported.
Feb 26 13:35:04 grub-installer: info: grub-install ran successfully
Feb 26 13:35:04 grub-installer: info: grub-install does not support --no-floppy
Feb 26 13:35:04 grub-installer: info: Running chroot /target grub-install --force "/dev/sdb1"
Feb 26 13:35:04 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 13:35:13 grub-installer: Installation finished. No error reported.
Feb 26 13:35:13 grub-installer: info: grub-install ran successfully
Feb 26 13:35:13 grub-installer: info: grub-install does not support --no-floppy
Feb 26 13:35:13 grub-installer: info: Running chroot /target grub-install --force "/dev/sdc1"
Feb 26 13:35:13 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 13:35:23 grub-installer: Installation finished. No error reported.
Feb 26 13:35:23 grub-installer: info: grub-install ran successfully
9) Return to the installation, and select 'Finish the installation'.
~ # exit
Boot tests:
During boot, the SLOF firmware could successfully detect and load the GRUB2 code from
the PReP partition of *each individual disk* (test: run guest with either disk1, disk2, or disk3.qcow2).
Trying to load: from: disk ...
No DOS disk-label found.
Successfully loaded
error: no suitable video mode found.
<...>
GNU GRUB version 2.02~beta2-36
<...>
The only boot failure case is with only the spare disk (disk3.qcow2), since GRUB2 can't load
the remaining of its code from one of the active devices.
(notice it's a GRUB2 error message, so PReP was correctly detected and GRUB2 loaded from it.)
Trying to load: from: disk ...
No DOS disk-label found.
Successfully loaded
error: disk `mduuid/b3069040df508c48c7c8ddc6a19a6bc0,2' not found.
Entering rescue mode...
grub rescue>
== Comment: #8 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 09:03:00 ==
Hi Heitor,
Can you please check it works correctly on your PowerVM environment?
I verified this works correctly on a qemu-kvm guest.
Thanks
== Comment: #9 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 10:41:01 ==
Hi Canonical,
The patch is verified for 14.04 too. Please apply on 14.04 as well.
Thanks
== Comment: #10 - Mauricio Faria De Oliveira <mauricfo at br.ibm.com> - 2016-02-26 12:30:05 ==
(In reply to comment #9)
> The patch is verified for 14.04 too. Please apply on 14.04 as well.
Feb 26 14:17:48 anna[6955]: DEBUG: retrieving grub-installer 1.78ubuntu20.3
Feb 26 15:31:21 main-menu[1787]: INFO: Menu item 'grub-installer' selected
Feb 26 15:31:21 grub-installer: info: architecture: ppc64el/chrp_ibm
Feb 26 15:31:34 grub-installer: info: Identified partition label for /dev/md0p2: loop
Feb 26 15:31:34 grub-installer: info: Wiping PReP partition /dev/sda1
Feb 26 15:31:34 grub-installer: info: Wiping PReP partition /dev/sdb1
Feb 26 15:31:34 grub-installer: info: Wiping PReP partition /dev/sdc1
Feb 26 15:31:50 grub-installer: info: Installing grub on '/dev/sdc1'
Feb 26 15:31:50 grub-installer: info: grub-install does not support --no-floppy
Feb 26 15:31:50 grub-installer: info: Wiping PReP partition /dev/sda1
Feb 26 15:31:50 grub-installer: info: Running chroot /target grub-install --force "/dev/sda1"
Feb 26 15:31:50 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 15:31:59 grub-installer: Installation finished. No error reported.
Feb 26 15:31:59 grub-installer: info: grub-install ran successfully
Feb 26 15:31:59 grub-installer: info: grub-install does not support --no-floppy
Feb 26 15:31:59 grub-installer: info: Wiping PReP partition /dev/sdb1
Feb 26 15:31:59 grub-installer: info: Running chroot /target grub-install --force "/dev/sdb1"
Feb 26 15:31:59 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 15:32:08 grub-installer: Installation finished. No error reported.
Feb 26 15:32:08 grub-installer: info: grub-install ran successfully
Feb 26 15:32:08 grub-installer: info: grub-install does not support --no-floppy
Feb 26 15:32:08 grub-installer: info: Wiping PReP partition /dev/sdc1
Feb 26 15:32:08 grub-installer: info: Running chroot /target grub-install --force "/dev/sdc1"
Feb 26 15:32:08 grub-installer: Installing for powerpc-ieee1275 platform.
Feb 26 15:32:16 grub-installer: Installation finished. No error reported.
Feb 26 15:32:16 grub-installer: info: grub-install ran successfully
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub-installer/+bug/1550448/+subscriptions
More information about the Ubuntu-sponsors
mailing list