[Bug 1826857] Re: offloading disabled when using linuxbrige or bonding

Junien Fridrick 1826857 at bugs.launchpad.net
Tue Jul 23 14:28:50 UTC 2019 

I don't think I tested with OVS out of the picture. However, none of the
physical interface was enslaved to an OVS bridge, so I'd say this is not
an ovs bug and can likely be repro'ed outside of OVS. I don't have time
to do this in the short term though.

** Changed in: openvswitch (Ubuntu)
       Status: Incomplete => New

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1826857

Title:
  offloading disabled when using linuxbrige or bonding

Status in Ubuntu Cloud Archive:
  New
Status in openvswitch package in Ubuntu:
  New

Bug description:
  Hi,

  I'm running Ubuntu 18.04 kernel 4.15.0-47-generic, openvswitch
  2.10.0-0ubuntu2~cloud0 set up by neutron (from openstack rocky) -
  VXLAN overlay.

  I've found out that as soon as a linuxbrige or bond is involved the
  network path of openvswitch, the offloading features of the NIC
  (checksumming, segmentation, VXLAN encap) get disabled, which reduces
  the throughput a lot.

  You can find the flows below. Examples of setup used :

  "Normal" setup :

  br-bond0 (192.168.11.11)
   bond0
    enp50s0f0
    enp113s0f0 (handling traffic)

  
  Only interface in the bond :

  br-bond0 (192.168.11.11)
   bond0
    enp50s0f0 (handling traffic)

  
  Interface directly in the bridge :

  br-bond0 (192.168.11.11)
   enp50s0f0

  With the setups above, there is NO offloading happening (tcpdump
  reports packet sizes of the MTU (~9k)). If the IP address is directly
  on the enp50s0f0 interface, then offloading works.

  For what it's worth, the NICs are Mellanox MT27800.

  Thanks !

  OVS flows :

  $ sudo ovs-vsctl show
  6f7e2040-e13b-4029-86b7-4f360aad2efe
      Manager "ptcp:6640:127.0.0.1"
          is_connected: true
      Bridge "br-physnet1"
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "phy-br-physnet1"
              Interface "phy-br-physnet1"
                  type: patch
                  options: {peer="int-br-physnet1"}
          Port "br-physnet1"
              Interface "br-physnet1"
                  type: internal
      Bridge br-int
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "sg-eecf32f7-da"
              tag: 2
              Interface "sg-eecf32f7-da"
                  type: internal
          Port "qvoc654e76e-de"
              tag: 2
              Interface "qvoc654e76e-de"
          Port "qr-84debef4-b7"
              tag: 2
              Interface "qr-84debef4-b7"
                  type: internal
          Port "qr-3e9ad9a7-02"
              tag: 3
              Interface "qr-3e9ad9a7-02"
                  type: internal
          Port "fg-997d43cb-27"
              tag: 5
              Interface "fg-997d43cb-27"
                  type: internal
          Port "qvoba76f8a7-50"
              tag: 2
              Interface "qvoba76f8a7-50"
          Port "qvod01a61c2-59"
              tag: 2
              Interface "qvod01a61c2-59"
          Port "enp50s0f0_60"
              tag: 2
              Interface "enp50s0f0_60"
          Port "tap15879e40-d9"
              tag: 2
              Interface "tap15879e40-d9"
          Port "int-br-physnet1"
              Interface "int-br-physnet1"
                  type: patch
                  options: {peer="phy-br-physnet1"}
          Port "qg-3f2f0aa4-fa"
              tag: 5
              Interface "qg-3f2f0aa4-fa"
                  type: internal
          Port "ha-a481b72f-6a"
              tag: 4
              Interface "ha-a481b72f-6a"
                  type: internal
          Port "enp50s0f0_62"
              tag: 2
              Interface "enp50s0f0_62"
          Port "sg-5e3c2880-79"
              tag: 3
              Interface "sg-5e3c2880-79"
                  type: internal
          Port "tap62452ada-90"
              tag: 3
              Interface "tap62452ada-90"
          Port br-int
              Interface br-int
                  type: internal
          Port "qvo75f1a5f8-e6"
              tag: 2
              Interface "qvo75f1a5f8-e6"
          Port "tapbc9a8481-c2"
              tag: 2
              Interface "tapbc9a8481-c2"
          Port "tapd7155195-25"
              tag: 1
              Interface "tapd7155195-25"
          Port patch-tun
              Interface patch-tun
                  type: patch
                  options: {peer=patch-int}
          Port "qvo1e23c34e-5f"
              tag: 2
              Interface "qvo1e23c34e-5f"
          Port "enp50s0f0_61"
              tag: 2
              Interface "enp50s0f0_61"
      Bridge br-ex
          Port br-ex
              Interface br-ex
                  type: internal
      Bridge br-tun
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port patch-int
              Interface patch-int
                  type: patch
                  options: {peer=patch-tun}
          Port br-tun
              Interface br-tun
                  type: internal
          Port "vxlan-c0a80a0b"
              Interface "vxlan-c0a80a0b"
                  type: vxlan
                  options: {df_default="true", in_key=flow, local_ip="192.168.11.11", out_key=flow, remote_ip="192.168.10.11"}
      ovs_version: "2.10.0"


  Features for all interfaces :
  $ for i in br-bond0 bond0 enp50s0f0 enp113s0f0; do sudo ethtool -k $i; echo "======"; done
  Features for br-bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
          tx-checksum-ipv4: off [fixed]
          tx-checksum-ip-generic: on
          tx-checksum-ipv6: off [fixed]
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: on
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: on
          tx-tcp-mangleid-segmentation: on
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off [fixed]
  rx-vlan-offload: off [fixed]
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: off [fixed]
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [requested on]
  tx-fcoe-segmentation: off [requested on]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: on
  tx-ipxip6-segmentation: on
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: on
  tx-esp-segmentation: on
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: off [fixed]
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: off [fixed]
  ======
  Features for bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
          tx-checksum-ipv4: off [fixed]
          tx-checksum-ip-generic: on
          tx-checksum-ipv6: off [fixed]
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [requested on]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: on
          tx-tcp-mangleid-segmentation: on
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: on
  tx-ipxip6-segmentation: on
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: off [fixed]
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: off [fixed]
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: off [fixed]
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: off [fixed]
  ======
  Features for enp50s0f0:
  rx-checksumming: on
  tx-checksumming: on
          tx-checksum-ipv4: on
          tx-checksum-ip-generic: off [fixed]
          tx-checksum-ipv6: on
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [fixed]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: off [fixed]
          tx-tcp-mangleid-segmentation: off
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off
  receive-hashing: on
  highdma: on [fixed]
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: off [fixed]
  netns-local: off [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: off [fixed]
  tx-ipxip6-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off
  rx-all: off
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: on [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: on
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: on
  ======
  Features for enp113s0f0:
  rx-checksumming: on
  tx-checksumming: on
          tx-checksum-ipv4: on
          tx-checksum-ip-generic: off [fixed]
          tx-checksum-ipv6: on
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [fixed]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: off [fixed]
          tx-tcp-mangleid-segmentation: off
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off
  receive-hashing: on
  highdma: on [fixed]
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: off [fixed]
  netns-local: off [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: off [fixed]
  tx-ipxip6-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off
  rx-all: off
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: on [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: on
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: on
  ======

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: openvswitch-switch 2.10.0-0ubuntu2~cloud0 [origin: Canonical]
  ProcVersionSignature: Ubuntu 4.15.0-47.50-generic 4.15.18
  Uname: Linux 4.15.0-47-generic x86_64
  ApportVersion: 2.20.9-0ubuntu7.6
  Architecture: amd64
  CrashDB:
   {
                  "impl": "launchpad",
                  "project": "cloud-archive",
                  "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
               }
  Date: Mon Apr 29 09:28:24 2019
  ProcEnviron:
   TERM=screen-256color
   PATH=(custom, no user)
   LANG=C.UTF-8
   SHELL=/bin/bash
  SourcePackage: openvswitch
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1826857/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list