[Bug 2004179] Re: neutron-linuxbridge-agent flat network incompatibility with systemd-networkd

[Jan Graichen]

We're not yet on 24.04 and still have a few OpenStack upgrades to do
before we can try 24.04. For now, we're running 22.04 and do specify the
bridges that neutron-linuxbridge-agent would create directly in systemd-
networkd, which mostly works, like this:

    # /etc/systemd/network/10-brqe240c66b-da.netdev
    [NetDev]
    Name=brqe240c66b-da
    Kind=bridge
    MTUBytes=1500

    # /etc/systemd/network/10-brqe240c66b-da.network
    [Match]
    Name=brqe240c66b-da
    [Link]
    MTUBytes=1500
    [Network]
    LinkLocalAddressing=no
    ConfigureWithoutCarrier=yes

This might have some implications on security, since we're not
completely sure that we didn't miss any isolation or brtables/iptables
rule that neutron-linuxbridge-agent would or wouldn't set up, but in our
specific case, we deemed that as acceptable.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/2004179

Title:
  neutron-linuxbridge-agent flat network incompatibility with systemd-
  networkd

Status in systemd package in Ubuntu:
  Incomplete

Bug description:
  We are running an OpenStack installation from Ubuntu's Cloud Archive,
  and our computer hosts have their network configured with systemd-
  networkd. For example, a bond and several VLANs on top, including two
  VLANs used for OpenStack flat networks. We are using neutron-
  linuxbridge-agent, i.e. no OVS or OVN.

  Network interface overview:

       ┌──────┐ ┌──────┐ ┌──────┐
       │ eno2 │ │ eno3 │ │ eno4 │
       └───┬──┘ └───┬──┘ └───┬──┘
           │        │        │
       ┌───┴────────┴────────┴──┐
       │         bond0          │
       └─┬───────┬─────────┬────┘
         │       │         │
     ┌───┴──┐ ┌──┴───┐ ┌───┴────┐
     │ tnet │ │ stor │ │ public │
     └──────┘ └──────┘ └────────┘

  The "public" network is a flat provider network (VLAN) that shall be
  used by neutron-linuxbridge-agent and nova-compute to attach to
  virtual machines. "tnet" is used for VXLAN overlay networks, and
  "stor" for Ceph. The issue is with "public" only.

  The systemd-networkd configuration sets up all interfaces. We
  previously used netplan, but missing some needed options. It does use
  systemd-networkd too, so no relevant change to the problem.

  Since merging https://github.com/systemd/systemd/pull/17392, systemd-
  network will remove the master flag from the "public" interface, that
  neutron-linuxbridge-agent assigned to the bridge for the network, e.g.
  brq88363244-5f, breaking all network on the flat network.

  The fix was revered in bug #1929560, but the behavior will not be
  reverted in newer releases. Therefore, every reload of systemd-
  networkd will remove the physical network interface from neutrons
  bridge.

  Using "bridge_mappings" with nova-compute has never been finished,
  according to bug #1105488. nova-compute always requires the dynamic
  bridge, e.g. brq88363244-5f, to be present.

  @ddstreet asked for a new bug and the network configuration:

      # /etc/systemd/network/10-eno1.network:
      [..]

      # /etc/systemd/network/10-eno2.network:
      [Match]
      Name=eno2
      
      [Network]
      LinkLocalAddressing=no
      Bond=bond0

      # /etc/systemd/network/10-eno3.network:
      [Match]
      Name=eno3
      
      [Network]
      LinkLocalAddressing=no
      Bond=bond0

      # /etc/systemd/network/10-eno4.network:
      [Match]
      Name=eno4
      
      [Network]
      LinkLocalAddressing=no
      Bond=bond0

      # /etc/systemd/network/10-bond0.netdev:
      [NetDev]
      Name=bond0
      Kind=bond
      MTUBytes=9000

      [Bond]
      Mode=802.3ad
      LACPTransmitRate=fast
      MIIMonitorSec=100ms
      TransmitHashPolicy=layer3+4

      # /etc/systemd/network/10-bond0.network:
      [Match]
      Name=bond0

      [Link]
      MTUBytes=9000

      [Network]
      LinkLocalAddressing=no
      ConfigureWithoutCarrier=yes
      VLAN=tnet
      VLAN=stor
      VLAN=pulic

      # /etc/systemd/network/10-public.netdev:
      [NetDev]
      Name=public
      Kind=vlan
      MTUBytes=1500

      [VLAN]
      Id=82

      # /etc/systemd/network/10-public.network:
      [Match]
      Name=public

      [Link]
      MTUBytes=1500

      [Network]
      LinkLocalAddressing=no
      ConfigureWithoutCarrier=yes

      # /etc/systemd/network/10-tnet.netdev:
      [..]

      # /etc/systemd/network/10-tnet.network:
      [..]

      # /etc/systemd/network/10-stor.netdev:
      [..]

      # /etc/systemd/network/10-stor.network:
      [..]

  
  neutron-linuxbridge-agent is configured to use "public" as a flat network, e.g.:

      # /etc/neutron/plugins/ml2/linuxbridge_agent.ini
      [linux_bridge]
      physical_interface_mappings = public:public
      [..]
      
  When everything is started and a VM needs to be connected to the flat network, neutron-linuxbridge-agent creates a dynamic bridge for nova-compute, e.g. brq88363244-5f:

      14: public at bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master brqe240c66b-da state UP
      59: brqe240c66b-da: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP

  When systemd-networkd is restarted, it will see that "public" is up
  but has a master assigned. It will be unassigned and break the flat
  network:

      14: public at bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP
      59: brqe240c66b-da: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP

  What would be the recommended why to configure networking on compute
  hosts with Ubuntu OpenStack and flat networks? Not using systemd-
  networkd or netplan at all, but only /etc/network/interfaces?

  Is there any modern alternative?

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2004179/+subscriptions