[Bug 1652348] Re: initrd dhcp fails / ignores valid response

Tue Jun 13 15:44:35 UTC 2017

Hello Paul, or anyone else affected,

Accepted klibc into trusty-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/klibc/2.0.3-0ubuntu1.14.04.3 in a
few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: klibc (Ubuntu Trusty)
       Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to klibc in Ubuntu.
https://bugs.launchpad.net/bugs/1652348

Title:
  initrd dhcp fails / ignores valid response

Status in klibc package in Ubuntu:
  Fix Released
Status in klibc source package in Trusty:
  Fix Committed
Status in klibc source package in Xenial:
  Fix Released
Status in klibc source package in Yakkety:
  Fix Committed
Status in klibc source package in Zesty:
  Fix Released

Bug description:
  [SRU justification]
  Changes to ordering of kernel enumeration of network interfaces, which may happen in any release, can regress network configuration from an initramfs.  Support for netbooting should not depend on interface order, it should work reliably on all systems.

  [Test case]
  Detailed reproducer described in 
  <https://bugs.launchpad.net/ubuntu/+source/klibc/+bug/1652348/comments/35>.

  [Regression potential]
  Moderate regression potential, because of a relatively large patch touching a not-widely-used but still critical piece of code.  Regression testing should include verifying that MAAS-booted cloud images still work as expected in a variety of environments.

  Between kernel versions 4.4.0-53 and 4.4.0-57 a bug has been (re?)introduced that is breaking dhcp booting in the initrd environment.  This is stopping instances that use iscsi storage from being able to connect.

  Over serial console it outputs:

  IP-Config: no response after 2 secs - giving up
  IP-Config: ens2f0 hardware address 90:e2:ba:d1:36:38 mtu 1500 DHCP RARP
  IP-Config: ens2f1 hardware address 90:e2:ba:d1:36:39 mtu 1500 DHCP RARP
  IP-Config: no response after 3 secs - giving up

  with increasing delays until it fails.  At which point a simple
  ipconfig -t dhcp -d "ens2f0"  works.  The console output is slightly
  garbled but should give you an idea:

  (initramfs) ipconfig -t dhcp -[  728.379793] ixgbe 0000:13:00.0 ens2f0: changing MTU from 1500 to 9000
  d "ens2f0"
  IP-Config: ens2f0 hardware address 90:e2:ba:d1:36:38 mtu 1500 DHCP RARP
  IP-Config: ens2f0 guessed broadcast address 10.0.1.255
  IP-Config: ens2f0 complete (dhcp from 169.254.169.254):
   addres[  728.980448] ixgbe 0000:13:00.0 ens2f0: detected SFP+: 3
  s: 10.0.1.56        broadcast: 10.0.1.255       netmask: 255.255.255.0
   gateway: 10.0.1.1   [  729.148410] ixgbe 0000:13:00.0 ens2f0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
        dns0     : 169.254.169.254  dns1   : 0.0.0.0
   rootserver: 169.254.169.254 rootpath:
   filename  : /ipxe.efi

  tcpdumps show that dhcp requests are being received from the host, and
  responses sent, but not accepted by the host.  When the ipconfig
  command is issued manually, an identical dhcp request and response
  happens, only this time it is accepted.  It doesn't appear to be that
  the messages are being sent and received incorrectly, just silently
  ignored by ipconfig.

  I was seeing this behaviour earlier this year, which I was able to fix
  by specifying "ip=dhcp" as a kernel parameter.  About a month ago that
  was identified as causing us other problems (long story) and we
  dropped it, at which point we discovered the original bug was no
  longer an issue.

  Putting "ip=dhcp" back on with this kernel no longer fixes the
  problem.

  I've compared the two initrds and effectively the only thing that has
  changed between the two is the kernel components.

  Ubuntu kernel bisect offending commit:
  # first bad commit: [fd4b5fa6e3487d15ede746f92601af008b2abbc0] mnt: Add a per mount namespace limit on the number of mounts

  Ubuntu kernel bisect offending commit submission:
  https://lkml.org/lkml/2016/10/5/308

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/klibc/+bug/1652348/+subscriptions