[Bug 1928040] Re: GCE instances drop to a grub prompt when GRUB_DISTRIBUTOR=Debian is set

[Launchpad Bug Tracker]

This bug was fixed in the package grub2-signed - 1.167~16.04.2

---------------
grub2-signed (1.167~16.04.2) xenial; urgency=medium

  * Remove the line GRUB_DISTRIBUTOR=Debian from
    /etc/default/grub.d/50-cloudimg-settings.cfg on select cloud images,
    namely those produced for GCE before 2020-11-11. This ensures that UEFI
    instances can locate efi executables and grub.cfg in correct directories
    since grub 2.04 seems to enforce pedantic locations. (LP: #1928040)

 -- Matthew Ruffell <matthew.ruffell at canonical.com>  Tue, 11 May 2021
19:38:29 +1200

** Changed in: grub2-signed (Ubuntu Xenial)
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to grub2-signed in Ubuntu.
https://bugs.launchpad.net/bugs/1928040

Title:
  GCE instances drop to a grub prompt when GRUB_DISTRIBUTOR=Debian is
  set

Status in grub2-signed package in Ubuntu:
  Invalid
Status in grub2-signed source package in Xenial:
  Fix Released

Bug description:
  [Impact]

  GCE cloud instances started with images released prior to 2020-11-11
  will fail to reboot when the newest grub2 2.02~beta2-36ubuntu3.32
  packages are installed from -updates.

  Upon reboot, the instance drops down to a grub prompt, and ceases to
  boot any further.

  The output displayed is:

  BdsDxe: loading Boot0003 "debian" from HD(15,GPT,<UUID>,0x2800,0x35000)/\EFI\debian\shimx64.efi
  BdsDxe: starting Boot0003 "debian" from HD(15,GPT,<UUID>,0x2800,0x35000)/\EFI\debian\shimx64.efi

  UEFI: Attempting to start image.
  Description: debian
  FilePath: HD(15,GPT,36903981-58D9-4718-A4EC-3D3E6CF6AF42,0x2800,0x35000)/\EFI\debian\shimx64.efi
  OptionNumber: 3.

  GNU GRUB version 2.04
  grub>

  Now, on GCE cloud images prior to 2020-11-11, /etc/default/grub.d/50
  -cloudimg-settings.cfg contained this line:

  GRUB_DISTRIBUTOR=Debian

  In images after 2020-11-11, this line was REMOVED from
  /etc/default/grub.d/50-cloudimg-settings.cfg and instead, images fell
  back to the below line in /etc/default/grub

  GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`

  The above line always returns "Ubuntu".

  The new grub2 version 2.04-1ubuntu44 now always looks for the grub.cfg
  in /EFI/ubuntu, with no fallback to look in the directory the EFI grub
  executable was booted from.  Since GRUB_DISTRIBUTOR=Debian means the
  grub config is only written to /EFI/debian/grub.cfg, the new grub
  fails to find its config despite being booted from
  /EFI/debian/grubx64.efi, and hence we drop to a grub shell.

  [Testcase]

  You can start an instance up on Google Cloud with an affected image
  with the below command:

  gcloud compute instances create test-xenial-1
  --image=ubuntu-1604-xenial-v20200429 --image-project=ubuntu-os-cloud

  From there:

  $ sudo apt update
  $ sudo apt install grub-common grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed grub-pc-bin grub2-common
  $ sudo reboot

  The instance will not come back up, and you will see a grub shell in
  the logs on GCP.

  You can also reproduce in KVM. Simply add:

  GRUB_DISTRIBUTOR=Debian

  to a file in /etc/default/grub.d/50-cloudimg-settings.cfg then:

  $ sudo update-grub
  $ sudo reboot

  Test packages are available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1928040-test

  If you install these test packages, you should be able to upgrade grub
  and reboot without issue.

  [Where problems could occur]

  We will be changing the grub configuration for every Google cloud
  instance started with an image produced before 2020-11-11, and there
  is risk that we could make a change which prevents instances from
  booting. We should proceed with caution and make sure to test older
  and newer images.

  The fix will be targeted to the /etc/default/grub.d/50-cloudimg-
  settings.cfg file only, so only cloud instances would get the change,
  and only if the grub versions match particular versions.

  [Other info]

  A workaround is to remove the below line from /etc/default/grub.d/50
  -cloudimg-settings.cfg before installing the new grub packages.

  GRUB_DISTRIBUTOR=Debian

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub2-signed/+bug/1928040/+subscriptions