[Bug 1826857] Re: offloading disabled when using linuxbrige or bonding
James Page
james.page at ubuntu.com
Thu Jul 18 12:48:13 UTC 2019
@axino
Just to confirm whether you tested - if you take OVS out of the picture
and just test with bonded network interfaces, do you see offloading
still being disabled? Slightly tricky for the VXLAN test but wanted to
see if you had tested that combination.
I'm wondering whether this is an OVS issue or whether there is an issue
with hardware offloading with linux bonds more generally.
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1826857
Title:
offloading disabled when using linuxbrige or bonding
Status in Ubuntu Cloud Archive:
New
Status in openvswitch package in Ubuntu:
New
Bug description:
Hi,
I'm running Ubuntu 18.04 kernel 4.15.0-47-generic, openvswitch
2.10.0-0ubuntu2~cloud0 set up by neutron (from openstack rocky) -
VXLAN overlay.
I've found out that as soon as a linuxbrige or bond is involved the
network path of openvswitch, the offloading features of the NIC
(checksumming, segmentation, VXLAN encap) get disabled, which reduces
the throughput a lot.
You can find the flows below. Examples of setup used :
"Normal" setup :
br-bond0 (192.168.11.11)
bond0
enp50s0f0
enp113s0f0 (handling traffic)
Only interface in the bond :
br-bond0 (192.168.11.11)
bond0
enp50s0f0 (handling traffic)
Interface directly in the bridge :
br-bond0 (192.168.11.11)
enp50s0f0
With the setups above, there is NO offloading happening (tcpdump
reports packet sizes of the MTU (~9k)). If the IP address is directly
on the enp50s0f0 interface, then offloading works.
For what it's worth, the NICs are Mellanox MT27800.
Thanks !
OVS flows :
$ sudo ovs-vsctl show
6f7e2040-e13b-4029-86b7-4f360aad2efe
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge "br-physnet1"
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "phy-br-physnet1"
Interface "phy-br-physnet1"
type: patch
options: {peer="int-br-physnet1"}
Port "br-physnet1"
Interface "br-physnet1"
type: internal
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "sg-eecf32f7-da"
tag: 2
Interface "sg-eecf32f7-da"
type: internal
Port "qvoc654e76e-de"
tag: 2
Interface "qvoc654e76e-de"
Port "qr-84debef4-b7"
tag: 2
Interface "qr-84debef4-b7"
type: internal
Port "qr-3e9ad9a7-02"
tag: 3
Interface "qr-3e9ad9a7-02"
type: internal
Port "fg-997d43cb-27"
tag: 5
Interface "fg-997d43cb-27"
type: internal
Port "qvoba76f8a7-50"
tag: 2
Interface "qvoba76f8a7-50"
Port "qvod01a61c2-59"
tag: 2
Interface "qvod01a61c2-59"
Port "enp50s0f0_60"
tag: 2
Interface "enp50s0f0_60"
Port "tap15879e40-d9"
tag: 2
Interface "tap15879e40-d9"
Port "int-br-physnet1"
Interface "int-br-physnet1"
type: patch
options: {peer="phy-br-physnet1"}
Port "qg-3f2f0aa4-fa"
tag: 5
Interface "qg-3f2f0aa4-fa"
type: internal
Port "ha-a481b72f-6a"
tag: 4
Interface "ha-a481b72f-6a"
type: internal
Port "enp50s0f0_62"
tag: 2
Interface "enp50s0f0_62"
Port "sg-5e3c2880-79"
tag: 3
Interface "sg-5e3c2880-79"
type: internal
Port "tap62452ada-90"
tag: 3
Interface "tap62452ada-90"
Port br-int
Interface br-int
type: internal
Port "qvo75f1a5f8-e6"
tag: 2
Interface "qvo75f1a5f8-e6"
Port "tapbc9a8481-c2"
tag: 2
Interface "tapbc9a8481-c2"
Port "tapd7155195-25"
tag: 1
Interface "tapd7155195-25"
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "qvo1e23c34e-5f"
tag: 2
Interface "qvo1e23c34e-5f"
Port "enp50s0f0_61"
tag: 2
Interface "enp50s0f0_61"
Bridge br-ex
Port br-ex
Interface br-ex
type: internal
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Port "vxlan-c0a80a0b"
Interface "vxlan-c0a80a0b"
type: vxlan
options: {df_default="true", in_key=flow, local_ip="192.168.11.11", out_key=flow, remote_ip="192.168.10.11"}
ovs_version: "2.10.0"
Features for all interfaces :
$ for i in br-bond0 bond0 enp50s0f0 enp113s0f0; do sudo ethtool -k $i; echo "======"; done
Features for br-bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: on
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp-mangleid-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [requested on]
tx-fcoe-segmentation: off [requested on]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: on
tx-ipxip6-segmentation: on
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: on
tx-esp-segmentation: on
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: on
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
======
Features for bond0:
rx-checksumming: off [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on
tx-checksum-ipv6: off [fixed]
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [requested on]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp-mangleid-segmentation: on
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: on [fixed]
netns-local: on [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: on
tx-ipxip6-segmentation: on
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
======
Features for enp50s0f0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: on
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: on [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: on
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
======
Features for enp113s0f0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: on
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: on [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: on
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
======
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: openvswitch-switch 2.10.0-0ubuntu2~cloud0 [origin: Canonical]
ProcVersionSignature: Ubuntu 4.15.0-47.50-generic 4.15.18
Uname: Linux 4.15.0-47-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.6
Architecture: amd64
CrashDB:
{
"impl": "launchpad",
"project": "cloud-archive",
"bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
}
Date: Mon Apr 29 09:28:24 2019
ProcEnviron:
TERM=screen-256color
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
SourcePackage: openvswitch
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1826857/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list