[Bug 1886826] Re: high packet loss on 18.04 with systemd-networkd

[Luke Alexander]

Yeah, here is an example of the netplan config, though I have tried with
3 different servers, 2 with very similar configs as below and one with
bonded interfaces (all 3 have the same packet loss) - all 3 had the
same/equivalent configs and worked without packet loss when running
Ubuntu 16.04 - or when using netplan but with the renderer set to
NetworkManger instead of networkd

network:
  version: 2
  renderer: networkd
  ethernets:
    ens15f0:
      dhcp4: false
      dhcp6: false
      addresses:
        - 10.0.0.1/24
      gateway4: 10.0.0.254
      nameservers:
        addresses:
          - 8.8.8.8
          - 1.1.1.1
    ens15f1:
      dhcp4: false
      dhcp6: false
      addresses:
        - 192.168.1.12/16

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1886826

Title:
  high packet loss on 18.04 with systemd-networkd

Status in systemd package in Ubuntu:
  Incomplete

Bug description:
  Description:	Ubuntu 18.04.4 LTS
  Release:	18.04

  systemctl --version
  systemd 237
  +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid

  
  Our issue is that we have a k8s (1.18, kube-router CNI) cluster comprised of a number of Ubuntu 16.04 nodes, we are in the process of upgrading the nodes to Ubuntu 18.04 - however after upgrading the first node and adding it back to the cluster we observed high packet loss from other cluster nodes to the 18.04 node as well as ping error messages when running a continuous ping from the 18.04 node to another node in the cluster, eg:

  64 bytes from 10.8.11.1: icmp_seq=91 ttl=64 time=0.088 ms 
  64 bytes from 10.8.11.1: icmp_seq=92 ttl=64 time=0.076 ms 
  ping: sendmsg: No buffer space available 
  ping: sendmsg: No buffer space available 
  ping: sendmsg: No buffer space available

  Another observation is that one of our daemonset pods (promtail) could
  not start.

  After many days of troubleshooting various possibilities, changing
  network cable, NIC, trying a different server, checking/changing
  various sysctl values, I eventually tried switching from systemd-
  networkd to NetworkManager as the backend via netplan config. After
  rebooting the server with NetworkManager in place and re-adding the
  node back to the k8s cluster, the packet loss disappeared as did the
  ping errors and the daemonset pod (promtail) also started normally.

  I would like to use Ubuntu's default of systemd-networkd to handle our
  NIC/route config - but cannot until I've figured out what in systemd-
  networkd is breaking.

  Any help is much appreciated on this matter.

  Thanks!
  Luke

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1886826/+subscriptions