[Bug 1826857] Re: offloading disabled when using linuxbrige or bonding

Launchpad Bug Tracker 1826857 at bugs.launchpad.net
Wed Aug 14 16:09:09 UTC 2019 

Status changed to 'Confirmed' because the bug affects multiple users.

** Changed in: openvswitch (Ubuntu)
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1826857

Title:
  offloading disabled when using linuxbrige or bonding

Status in Ubuntu Cloud Archive:
  New
Status in openvswitch package in Ubuntu:
  Confirmed

Bug description:
  Hi,

  I'm running Ubuntu 18.04 kernel 4.15.0-47-generic, openvswitch
  2.10.0-0ubuntu2~cloud0 set up by neutron (from openstack rocky) -
  VXLAN overlay.

  I've found out that as soon as a linuxbrige or bond is involved the
  network path of openvswitch, the offloading features of the NIC
  (checksumming, segmentation, VXLAN encap) get disabled, which reduces
  the throughput a lot.

  You can find the flows below. Examples of setup used :

  "Normal" setup :

  br-bond0 (192.168.11.11)
   bond0
    enp50s0f0
    enp113s0f0 (handling traffic)

  
  Only interface in the bond :

  br-bond0 (192.168.11.11)
   bond0
    enp50s0f0 (handling traffic)

  
  Interface directly in the bridge :

  br-bond0 (192.168.11.11)
   enp50s0f0

  With the setups above, there is NO offloading happening (tcpdump
  reports packet sizes of the MTU (~9k)). If the IP address is directly
  on the enp50s0f0 interface, then offloading works.

  For what it's worth, the NICs are Mellanox MT27800.

  Thanks !

  OVS flows :

  $ sudo ovs-vsctl show
  6f7e2040-e13b-4029-86b7-4f360aad2efe
      Manager "ptcp:6640:127.0.0.1"
          is_connected: true
      Bridge "br-physnet1"
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "phy-br-physnet1"
              Interface "phy-br-physnet1"
                  type: patch
                  options: {peer="int-br-physnet1"}
          Port "br-physnet1"
              Interface "br-physnet1"
                  type: internal
      Bridge br-int
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "sg-eecf32f7-da"
              tag: 2
              Interface "sg-eecf32f7-da"
                  type: internal
          Port "qvoc654e76e-de"
              tag: 2
              Interface "qvoc654e76e-de"
          Port "qr-84debef4-b7"
              tag: 2
              Interface "qr-84debef4-b7"
                  type: internal
          Port "qr-3e9ad9a7-02"
              tag: 3
              Interface "qr-3e9ad9a7-02"
                  type: internal
          Port "fg-997d43cb-27"
              tag: 5
              Interface "fg-997d43cb-27"
                  type: internal
          Port "qvoba76f8a7-50"
              tag: 2
              Interface "qvoba76f8a7-50"
          Port "qvod01a61c2-59"
              tag: 2
              Interface "qvod01a61c2-59"
          Port "enp50s0f0_60"
              tag: 2
              Interface "enp50s0f0_60"
          Port "tap15879e40-d9"
              tag: 2
              Interface "tap15879e40-d9"
          Port "int-br-physnet1"
              Interface "int-br-physnet1"
                  type: patch
                  options: {peer="phy-br-physnet1"}
          Port "qg-3f2f0aa4-fa"
              tag: 5
              Interface "qg-3f2f0aa4-fa"
                  type: internal
          Port "ha-a481b72f-6a"
              tag: 4
              Interface "ha-a481b72f-6a"
                  type: internal
          Port "enp50s0f0_62"
              tag: 2
              Interface "enp50s0f0_62"
          Port "sg-5e3c2880-79"
              tag: 3
              Interface "sg-5e3c2880-79"
                  type: internal
          Port "tap62452ada-90"
              tag: 3
              Interface "tap62452ada-90"
          Port br-int
              Interface br-int
                  type: internal
          Port "qvo75f1a5f8-e6"
              tag: 2
              Interface "qvo75f1a5f8-e6"
          Port "tapbc9a8481-c2"
              tag: 2
              Interface "tapbc9a8481-c2"
          Port "tapd7155195-25"
              tag: 1
              Interface "tapd7155195-25"
          Port patch-tun
              Interface patch-tun
                  type: patch
                  options: {peer=patch-int}
          Port "qvo1e23c34e-5f"
              tag: 2
              Interface "qvo1e23c34e-5f"
          Port "enp50s0f0_61"
              tag: 2
              Interface "enp50s0f0_61"
      Bridge br-ex
          Port br-ex
              Interface br-ex
                  type: internal
      Bridge br-tun
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port patch-int
              Interface patch-int
                  type: patch
                  options: {peer=patch-tun}
          Port br-tun
              Interface br-tun
                  type: internal
          Port "vxlan-c0a80a0b"
              Interface "vxlan-c0a80a0b"
                  type: vxlan
                  options: {df_default="true", in_key=flow, local_ip="192.168.11.11", out_key=flow, remote_ip="192.168.10.11"}
      ovs_version: "2.10.0"


  Features for all interfaces :
  $ for i in br-bond0 bond0 enp50s0f0 enp113s0f0; do sudo ethtool -k $i; echo "======"; done
  Features for br-bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
          tx-checksum-ipv4: off [fixed]
          tx-checksum-ip-generic: on
          tx-checksum-ipv6: off [fixed]
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: on
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: on
          tx-tcp-mangleid-segmentation: on
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off [fixed]
  rx-vlan-offload: off [fixed]
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: off [fixed]
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [requested on]
  tx-fcoe-segmentation: off [requested on]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: on
  tx-ipxip6-segmentation: on
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: on
  tx-esp-segmentation: on
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: off [fixed]
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: off [fixed]
  ======
  Features for bond0:
  rx-checksumming: off [fixed]
  tx-checksumming: on
          tx-checksum-ipv4: off [fixed]
          tx-checksum-ip-generic: on
          tx-checksum-ipv6: off [fixed]
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [requested on]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: on
          tx-tcp-mangleid-segmentation: on
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off [fixed]
  receive-hashing: off [fixed]
  highdma: on
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: on [fixed]
  netns-local: on [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: on
  tx-ipxip6-segmentation: on
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: off [fixed]
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off [fixed]
  rx-all: off [fixed]
  tx-vlan-stag-hw-insert: off [fixed]
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: off [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: off [fixed]
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: off [fixed]
  ======
  Features for enp50s0f0:
  rx-checksumming: on
  tx-checksumming: on
          tx-checksum-ipv4: on
          tx-checksum-ip-generic: off [fixed]
          tx-checksum-ipv6: on
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [fixed]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: off [fixed]
          tx-tcp-mangleid-segmentation: off
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off
  receive-hashing: on
  highdma: on [fixed]
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: off [fixed]
  netns-local: off [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: off [fixed]
  tx-ipxip6-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off
  rx-all: off
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: on [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: on
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: on
  ======
  Features for enp113s0f0:
  rx-checksumming: on
  tx-checksumming: on
          tx-checksum-ipv4: on
          tx-checksum-ip-generic: off [fixed]
          tx-checksum-ipv6: on
          tx-checksum-fcoe-crc: off [fixed]
          tx-checksum-sctp: off [fixed]
  scatter-gather: on
          tx-scatter-gather: on
          tx-scatter-gather-fraglist: off [fixed]
  tcp-segmentation-offload: on
          tx-tcp-segmentation: on
          tx-tcp-ecn-segmentation: off [fixed]
          tx-tcp-mangleid-segmentation: off
          tx-tcp6-segmentation: on
  udp-fragmentation-offload: off
  generic-segmentation-offload: on
  generic-receive-offload: on
  large-receive-offload: off
  rx-vlan-offload: on
  tx-vlan-offload: on
  ntuple-filters: off
  receive-hashing: on
  highdma: on [fixed]
  rx-vlan-filter: on
  vlan-challenged: off [fixed]
  tx-lockless: off [fixed]
  netns-local: off [fixed]
  tx-gso-robust: off [fixed]
  tx-fcoe-segmentation: off [fixed]
  tx-gre-segmentation: on
  tx-gre-csum-segmentation: on
  tx-ipxip4-segmentation: off [fixed]
  tx-ipxip6-segmentation: off [fixed]
  tx-udp_tnl-segmentation: on
  tx-udp_tnl-csum-segmentation: on
  tx-gso-partial: on
  tx-sctp-segmentation: off [fixed]
  tx-esp-segmentation: off [fixed]
  fcoe-mtu: off [fixed]
  tx-nocache-copy: off
  loopback: off [fixed]
  rx-fcs: off
  rx-all: off
  tx-vlan-stag-hw-insert: on
  rx-vlan-stag-hw-parse: off [fixed]
  rx-vlan-stag-filter: on [fixed]
  l2-fwd-offload: off [fixed]
  hw-tc-offload: on
  esp-hw-offload: off [fixed]
  esp-tx-csum-hw-offload: off [fixed]
  rx-udp_tunnel-port-offload: on
  ======

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: openvswitch-switch 2.10.0-0ubuntu2~cloud0 [origin: Canonical]
  ProcVersionSignature: Ubuntu 4.15.0-47.50-generic 4.15.18
  Uname: Linux 4.15.0-47-generic x86_64
  ApportVersion: 2.20.9-0ubuntu7.6
  Architecture: amd64
  CrashDB:
   {
                  "impl": "launchpad",
                  "project": "cloud-archive",
                  "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
               }
  Date: Mon Apr 29 09:28:24 2019
  ProcEnviron:
   TERM=screen-256color
   PATH=(custom, no user)
   LANG=C.UTF-8
   SHELL=/bin/bash
  SourcePackage: openvswitch
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1826857/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list