[Bug 1960758] Re: UEFI libvirt servers can't boot on Ubuntu 20.04 hypervisors with Ussuri/Victoria

[Robie Basak]

Thanks Mauricio. I'm interested to hear what Andreas thinks about the
previous conffile changes and its acceptability this time.

One other side of this is that this patch is being added as a workaround
to Focal only, right? Are there any scenarios where users might carry
forward this parameter to newer releases? What would happen if they did,
and what are our expectations about that?

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1960758

Title:
  UEFI libvirt servers can't boot on Ubuntu 20.04 hypervisors with
  Ussuri/Victoria

Status in Ubuntu Cloud Archive:
  Invalid
Status in Ubuntu Cloud Archive ussuri series:
  Triaged
Status in Ubuntu Cloud Archive victoria series:
  Triaged
Status in OpenStack Compute (nova):
  Invalid
Status in OpenStack Compute (nova) ussuri series:
  Invalid
Status in OpenStack Compute (nova) victoria series:
  Invalid
Status in nova package in Ubuntu:
  Invalid
Status in nova source package in Focal:
  In Progress

Bug description:
  Impact:
  ===

  Currently, setting `hw_firwmare_type=uefi` may create
  _unbootable_ servers on 20.04 hypervisors with Ussuri
  and Victoria (Wallaby and later are OK).

  We should not use the Secure Boot firmware on the 'pc'
  machine type, as 'q35' is _required_ by OVMF firmware
  if SMM feature is built (usually the case, to actually
  secure the SB feature).
  [See comment #6 for research and #7 for test evidence.]

  We should not use the Secure Boot firmware on the 'q35'
  machine type _either_, as it might not work regardless,
  since other libvirt XML options such as SMM and S3/S4
  disable may be needed for Secure Boot to work, but are
  _not_ configured by Openstack Ussuri (no SB support).

  
  Approach:
  ===

  Considering how long Focal/Ussuri have been out there
  (and maybe worked with UEFI enabled for some cases?)
  add a config option to _opt-in_ to actually supported
  UEFI loaders for nova/libvirt.

  This seems to benefit downstream/Ubuntu more (although
  other distros might be affected) add the config option
  "ubuntu_libvirt_uefi_loader_path" (disabled by default)
  in the DEFAULT libvirt config section (so it can be set
  in nova-compute charm's 'config-flags' option).

  
  Test Plan:
  ===

  $ openstack image set --property hw_firmware_type=uefi $IMAGE
  $ openstack server create --image $IMAGE --flavor $FLAVOR --network $NETWORK uefi-server

  (with patched packages:)
  Set `ubuntu_libvirt_uefi_loader_path = true` in `[DEFAULT]` in /etc/nova/nova.conf
  (eg `juju config nova-compute config-flags='ubuntu_libvirt_uefi_loader_path=true'`)
  $ openstack server stop uefi-server
  $ openstack server start uefi-server

  - Expected Result:

  The server's libvirt XML uses UEFI _without_ Secure Boot.

          <loader readonly='yes'
  type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader>

  The guest boots, and console log confirms UEFI mode:

          $ openstack console log show srv | grep -i -e efi -e bios
          ...
          Creating boot entry "Boot0003" with label "ubuntu" for file "\EFI\ubuntu\shimx64.efi"
          ...
          [    0.000000] efi: EFI v2.70 by EDK II
          [    0.000000] efi:  SMBIOS=0x7fbcd000  ACPI=0x7fbfa000  ACPI
          2.0=0x7fbfa014  MEMATTR=0x7eb30018
          [    0.000000] SMBIOS 2.8 present.
          [    0.000000] DMI: OpenStack Foundation OpenStack Nova, BIOS 0.0.0 02/06/2015
          ...

  - Actual Result:

  The server's libvirt XML uses UEFI _with_ Secure Boot.

          <loader readonly='yes'
  type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader>

  The guest doesn't boot; empty console log; qemu-kvm looping at 100%
  CPU.

          $ openstack console log show srv | grep -i -e efi -e bios
          $ openstack console log show srv | wc -l
          0

          $ juju run --app nova-compute 'top -b -d1 -n5 | grep qemu'
            67205 libvirt+  ... 100.0   1.4   1:18.35 qemu-sy+
            67205 libvirt+  ... 100.0   1.4   1:19.36 qemu-sy+
            67205 libvirt+  ...  99.0   1.4   1:20.36 qemu-sy+
            67205 libvirt+  ... 101.0   1.4   1:21.37 qemu-sy+
            67205 libvirt+  ... 100.0   1.4   1:22.38 qemu-sy+

  
  Where problems could occur:
  ===

  The changes are opt-in with `ubuntu_libvirt_uefi_loader_path=true`,
  so users are not affected by default.

  Theoretically, regressions would more likely manifest and be contained
  in nova's libvirt driver, when `hw_firwmare_type=uefi` (not by default).

  The expected symptoms of regressions are boot failures (server starts
  from openstack perspective, but doesn't boot to the operating system).

  
  Other Info:
  ===

  - Hypervisor running Ubuntu 20.04 LTS (Focal)
  - Nova packages from Ussuri (Ubuntu Archive) or Victoria (Cloud Archive).

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1960758/+subscriptions