[Bug 1718568] Re: dhclient-script fails to wait for link-local address

Thu Oct 12 13:44:30 UTC 2017

Hi Jeff,

Thanks for reporting and debugging this!

I've tweaked your patch from comment 2 slightly, you can see my version
in the debdiffs attached.  I was concerned that just a "  " check may
not always work right, so I explicitly check for the ip output to
contain 'inet6' (but not 'tentative') which is really what the function
is supposed to be doing.

Since this needs to go into the development release, the SRUs to older
releases have to wait.  And since artful is currently in freeze, this
will need to wait for the b-series development to start before we can
add the fix, and then SRU the fixes back to artful and older.  I'll come
back to this once b-series development has started and get the process
rolling.

Until then, I built test packages here:
https://launchpad.net/~ddstreet/+archive/ubuntu/lp1718568

can you verify that fixes it on your system?

** Description changed:

+ [impact]
+ 
+ bug 1633479 made a change to isc-dhcp to wait for an interface's link-local ipv6 address to switch from 'tentative' to normal, because all link-local addresses briefly go through a 'tentative' state while the kernel is performing ipv6 link-local 'duplicate address detection' (DAD).  While in the 'tentative' state, dhclient can't take over the interface and send out
+ dhcpv6 requests; it must wait until DAD completes.
+ 
+ However, the change made in that bug does not account for the case where
+ the 'tentative' check is done before the interface has even set up a
+ link-local address; its case statement assumes if there is no
+ 'tentative' or 'dadfailed' string in the output, the link-local address
+ is ready to use.  When the address check finds no address at all, this
+ will return as successful, even though it shouldn't, and dhclient will
+ fail to get the dhcpv6 address.
+ 
+ [test case]
+ 
+ on a system that is configured for dhcpv6 on one or more of its
+ interfaces, repeatedly try to get the dhcpv6 address.  For interfaces
+ that are slower to actually set up their initial tentative link-local
+ address, they will occasionally fail, since the current code is a race
+ between the kernel adding the tentative link-local address, and the
+ dhclient-script.linux code checking the interface for a tentative
+ address.
+ 
+ with the patch to correct this, even interfaces slow to add their
+ tentative link-local address should correctly wait for the address to
+ get added, and then transition from tentative to normal, and then begin
+ the dhcpv6 process.
+ 
+ [regression potential]
+ 
+ errors in this function can cause dhclient to fail to get a ipv6 address
+ for an interface; regression would happen if this patch makes it fail
+ more than it already is failing, but would not cause other failures or
+ problems after getting an ipv6 address; this patch will affect only
+ startup-time.
+ 
+ [other info]
+ 
+ related bug 1633479
+ 
+ 
+ [original description]
+ 
+ 
  Summary:
  ========

  If a interface does not yet have a link-local address (as it may have
  just been brought up), dhclient -6 <ifname> will fail. The built-in
  "wait for link-local address" loop does not function properly, causing
  DHCP failure.

  Discussion:
  ===========

  In trying to configure isc-dhcp-client 4.3.5-3ubuntu1 for IPv6 on Ubuntu
  17.04, I was finding that on boot I was getting failures with the logged
  message "no link-local IPv6 address for <ifname>"

  I found that it took several seconds for the link-local address to be
  assigned when the interface came up (in this case, the ISP/modem-facing
  interface), and worked around it with a script that looks at

    /sbin/ifconfig $IFACE | /bin/fgrep -q 'scopeid 0x20'

  and loops for a fixed number of times for that to be successful.

  On looking at /sbin/dhclient-script it appears that it *tries* to do the
  same thing in

    # set the link up and wait for ipv6 link local dad to finish
    ipv6_link_up_and_dad()

  this code sets

    out=$(ip -6 -o address show dev "$dev" scope link)

  then checks it with a case statement inside of a loop for

          case " $out " in
              *\ dadfailed\ *)
                  error "$dev: ipv6 dad failed."
                  return 1;;
              *\ tentative\ *) :;;
              *) return 0;;
          esac

  If there is no link-local address, $out will be empty. The default case
  is taken, and the loop exits immediately:

  $ echo "'$out'" ; case " $out " in
  >     *\ dadfailed\ *)
  >         echo "dadfailed"
  >         ;;
  >     *\ tentative\ *)
  >         echo "tentative"
  >         ;;
  >     *)
  >         echo "default"
  > esac
  ''
  default

  As a result, there is no "wait for link-local address" and when there is
  no link-local address, dhclient fails later on.

  Possible Fix:
  =============

  Adding "the missing case" for "no address" case that continues the loop
  is one possible solution.

  .        case " $out " in
  .            *\ dadfailed\ *)
  .                error "$dev: ipv6 dad failed."
  .                return 1;;
  .            *\ tentative\ *) :;;
  +            "  ")
  +                :
  +                ;;
  .            *) return 0;;
  .        esac

  At least in my situation, this prevents the failure of dhclient due to
  the link-local address not being "ready" yet.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to isc-dhcp in Ubuntu.
https://bugs.launchpad.net/bugs/1718568

Title:
  dhclient-script fails to wait for link-local address

Status in isc-dhcp package in Ubuntu:
  In Progress
Status in isc-dhcp source package in Trusty:
  In Progress
Status in isc-dhcp source package in Xenial:
  In Progress
Status in isc-dhcp source package in Zesty:
  In Progress
Status in isc-dhcp source package in Artful:
  In Progress
Status in isc-dhcp source package in bb-series:
  In Progress
Status in isc-dhcp package in Debian:
  New

Bug description:
  [impact]

  bug 1633479 made a change to isc-dhcp to wait for an interface's link-local ipv6 address to switch from 'tentative' to normal, because all link-local addresses briefly go through a 'tentative' state while the kernel is performing ipv6 link-local 'duplicate address detection' (DAD).  While in the 'tentative' state, dhclient can't take over the interface and send out
  dhcpv6 requests; it must wait until DAD completes.

  However, the change made in that bug does not account for the case
  where the 'tentative' check is done before the interface has even set
  up a link-local address; its case statement assumes if there is no
  'tentative' or 'dadfailed' string in the output, the link-local
  address is ready to use.  When the address check finds no address at
  all, this will return as successful, even though it shouldn't, and
  dhclient will fail to get the dhcpv6 address.

  [test case]

  on a system that is configured for dhcpv6 on one or more of its
  interfaces, repeatedly try to get the dhcpv6 address.  For interfaces
  that are slower to actually set up their initial tentative link-local
  address, they will occasionally fail, since the current code is a race
  between the kernel adding the tentative link-local address, and the
  dhclient-script.linux code checking the interface for a tentative
  address.

  with the patch to correct this, even interfaces slow to add their
  tentative link-local address should correctly wait for the address to
  get added, and then transition from tentative to normal, and then
  begin the dhcpv6 process.

  [regression potential]

  errors in this function can cause dhclient to fail to get a ipv6
  address for an interface; regression would happen if this patch makes
  it fail more than it already is failing, but would not cause other
  failures or problems after getting an ipv6 address; this patch will
  affect only startup-time.

  [other info]

  related bug 1633479

  [original description]

  Summary:
  ========

  If a interface does not yet have a link-local address (as it may have
  just been brought up), dhclient -6 <ifname> will fail. The built-in
  "wait for link-local address" loop does not function properly, causing
  DHCP failure.

  Discussion:
  ===========

  In trying to configure isc-dhcp-client 4.3.5-3ubuntu1 for IPv6 on
  Ubuntu 17.04, I was finding that on boot I was getting failures with
  the logged message "no link-local IPv6 address for <ifname>"

  I found that it took several seconds for the link-local address to be
  assigned when the interface came up (in this case, the ISP/modem-
  facing interface), and worked around it with a script that looks at

    /sbin/ifconfig $IFACE | /bin/fgrep -q 'scopeid 0x20'

  and loops for a fixed number of times for that to be successful.

  On looking at /sbin/dhclient-script it appears that it *tries* to do
  the same thing in

    # set the link up and wait for ipv6 link local dad to finish
    ipv6_link_up_and_dad()

  this code sets

    out=$(ip -6 -o address show dev "$dev" scope link)

  then checks it with a case statement inside of a loop for

          case " $out " in
              *\ dadfailed\ *)
                  error "$dev: ipv6 dad failed."
                  return 1;;
              *\ tentative\ *) :;;
              *) return 0;;
          esac

  If there is no link-local address, $out will be empty. The default
  case is taken, and the loop exits immediately:

  $ echo "'$out'" ; case " $out " in
  >     *\ dadfailed\ *)
  >         echo "dadfailed"
  >         ;;
  >     *\ tentative\ *)
  >         echo "tentative"
  >         ;;
  >     *)
  >         echo "default"
  > esac
  ''
  default

  As a result, there is no "wait for link-local address" and when there
  is no link-local address, dhclient fails later on.

  Possible Fix:
  =============

  Adding "the missing case" for "no address" case that continues the
  loop is one possible solution.

  .        case " $out " in
  .            *\ dadfailed\ *)
  .                error "$dev: ipv6 dad failed."
  .                return 1;;
  .            *\ tentative\ *) :;;
  +            "  ")
  +                :
  +                ;;
  .            *) return 0;;
  .        esac

  At least in my situation, this prevents the failure of dhclient due to
  the link-local address not being "ready" yet.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/isc-dhcp/+bug/1718568/+subscriptions