[Bug 1904588] [NEW] Network boot from MAAS sometimes fails at "grub>" prompt

[Rod Smith]

Public bug reported:

On SOME (but not all) boots via MAAS, GRUB hangs at the "grub>" prompt.
This happens about 10% or 20% of the time on affected servers (at least
ostwald and meitner, two Supermicro servers).

The MAAS rackd.log file shows that the node has requested and received
GRUB:

2020-11-17 10:08:47 provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 10.1.10.82
2020-11-17 10:08:47 provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 10.1.10.82
2020-11-17 10:08:50 provisioningserver.rackdservices.tftp: [info] grubx64.efi requested by 10.1.10.82

On a failed boot, the process then stops; the node does not request
grub.cfg, as happens normally. Watching the console, I see several
notices that read "error: Couldn't send network packet." (See attached
screen shot.)

At the grub> prompt, net_ls_addr shows the expected IP address, and
net_ls_routes shows a routing table; however, net_bootps results in an
error message stating "can't find command `net_bootps`" (see second
attached screen shot.)

Once the system is hung, typing "exit" at the "grub>" prompt causes the
server to try the next boot option, which usually works (booting via
another network interface, in the case of our servers).

As noted, this problem occurs on a minority of boots. It can affect
reboots after deployment, and if it occurs during deployment, it can
prevent deployment because the server will hang at the "grub>" prompt.

** Affects: grub2 (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: hwcert-server

** Attachment added: "Screen shot showing GRUB complaint about not being able to send packets"
   https://bugs.launchpad.net/bugs/1904588/+attachment/5435179/+files/iKVM_capture.jpg

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to grub2 in Ubuntu.
https://bugs.launchpad.net/bugs/1904588

Title:
  Network boot from MAAS sometimes fails at "grub>" prompt

Status in grub2 package in Ubuntu:
  New

Bug description:
  On SOME (but not all) boots via MAAS, GRUB hangs at the "grub>"
  prompt. This happens about 10% or 20% of the time on affected servers
  (at least ostwald and meitner, two Supermicro servers).

  The MAAS rackd.log file shows that the node has requested and received
  GRUB:

  2020-11-17 10:08:47 provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 10.1.10.82
  2020-11-17 10:08:47 provisioningserver.rackdservices.tftp: [info] bootx64.efi requested by 10.1.10.82
  2020-11-17 10:08:50 provisioningserver.rackdservices.tftp: [info] grubx64.efi requested by 10.1.10.82

  On a failed boot, the process then stops; the node does not request
  grub.cfg, as happens normally. Watching the console, I see several
  notices that read "error: Couldn't send network packet." (See attached
  screen shot.)

  At the grub> prompt, net_ls_addr shows the expected IP address, and
  net_ls_routes shows a routing table; however, net_bootps results in an
  error message stating "can't find command `net_bootps`" (see second
  attached screen shot.)

  Once the system is hung, typing "exit" at the "grub>" prompt causes
  the server to try the next boot option, which usually works (booting
  via another network interface, in the case of our servers).

  As noted, this problem occurs on a minority of boots. It can affect
  reboots after deployment, and if it occurs during deployment, it can
  prevent deployment because the server will hang at the "grub>" prompt.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1904588/+subscriptions