[Bug 1783272] Re: upgrading systemd package restarts systemd-networkd and briefly takes down network interfaces

[Trent Lloyd]

** Attachment added: "netplan config"
   https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1783272/+attachment/5167024/+files/01-netcfg.yaml

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1783272

Title:
  upgrading systemd package restarts systemd-networkd and briefly takes
  down network interfaces

Status in systemd package in Ubuntu:
  New

Bug description:
  Upgrading the systemd package, which contains systemd-networkd,
  appears to restart networkd and subsequently reconfigure network
  interfaces causing a brief connectivity outage.

  This is a bionic system which has a network bridge as it's primary
  interface through netplan.

  You can see from the logs that the interface appears to have been briefly taken down
  > Jul 24 09:40:32 optane kernel: [ 1935.046068] br0: port 1(eno1) entered disabled state

  We also see logs of networkd restarting
  > 248 Jul 24 09:40:32 optane systemd[1]: Stopping Network Service...
  > 253 Jul 24 09:40:32 optane systemd[1]: Stopped Network Service.
  > 254 Jul 24 09:40:32 optane systemd[1]: Starting Network Service...

  Based on the ordering of the messages from Avahi, I also believe that
  the IP address was first removed, then it was taken down, brought back
  up, and the IP re-added. But I can not state that with 100% certainty.
  However if you just set an interface down manually (ip link set br0
  down), usually Avahi notes the interface is relevant before the
  address was removed.  We see the opposite here.  It's possible the
  ordering is just not entirely deterministic though.

  Jul 24 09:40:32 optane avahi-daemon[1611]: Withdrawing address record for 10.48.134.22 on br0.
  Jul 24 09:40:32 optane avahi-daemon[1611]: Interface br0.IPv4 no longer relevant for mDNS.

  The main reason I noticed this, is that the unofficial oracle-
  java8-installer package upgraded at the same time - and it's wget to
  download java failed due to "Network is unreachable" as it was
  upgraded as the same time. To be clear, I'm not suggesting necessarily
  that this specific package was affected is the bug, but it's the
  reason I noticed the restart and it did cause an upgrade failure which
  I resumed with "dpkg --configure -a".  However there are many other
  implications of the network interface being reconfigured.

  Generally it's probably just not ideal to have networkd restart and
  reconfigure the network interfaces - no matter the possible causes.
  But from a quick thought about realistic implications

   - it seems the bridge was not deleted/re-created which would be bad
  for systems using libvirt/lxd/etc where the VMs may drop off the
  bridge. The interface index didn't change so hopefully we're safe from
  this one ("Jul 24 09:40:32 optane systemd-networkd[17118]: br0: netdev
  exists, using existing without changing its parameters")

   - Another use case other than oracle-java8-installer that is highly
  likely to be impacted is daemons configured to bind to a specific IP
  address. By default, those binds will fail if the IP address doesn't
  exist. It's possible these two restarts will race and those services
  will fail to restart. An example where this could happen, is Apache2.
  It would probably be hard to reproduce but logically likely to occur
  in some small number of cases.

   - we potentially do want networkd to ideally restart to upgrade the
  code, but ideally it would "diff" the network interface config and not
  tear things down. I am using a 'switchport' match in my netplan
  config, I wonder if this is related?

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1783272/+subscriptions