[Bug 1629053] Re: ovs bridge breaking TCP between two virtio net devices when checksum offload on

Fri Sep 30 08:24:36 UTC 2016

Ok, after discussing on IRC a bit more info.

my "normal bridge" backed device is tx-checksumming: on by default and work
my "ovs-dpdk" backed devices are tx-checksumming: off by default and work
my "normal ovs" backed devices are tx-checksumming: on by default and work as well

In E/N/I my interfaces only have (ens3 is the bridge backed one, and ens4 the one with an OVS of some kind):
auto ens3
iface ens3 inet dhcp
auto ens4
iface ens4 inet dhcp
So all defaults.

Versions for me:
libvirt-bin                           2.1.0-1ubuntu6
qemu-kvm                              1:2.6.1+dfsg-0ubuntu4

Sharing my network definitions for those:

Normal one:
    <interface type='network'>
      <mac address='52:54:00:3e:4b:48'/>
      <source network='default' bridge='virbr0'/>
      <target dev='vnet1'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

OVS-DPDK one
    <interface type='vhostuser'>
      <mac address='52:54:00:99:f7:a4'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user-2' mode='client'/>
      <model type='virtio'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>

With OVS-DPDK-multiqueue
    <interface type='vhostuser'>
      <mac address='52:54:00:d1:7e:cc'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user-2' mode='client'/>
      <model type='virtio'/>
      <driver queues='4'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface

With "pure" OVS checksumming:
    <interface type='bridge'>
      <mac address='52:54:00:b4:b4:1c'/>
      <source network='ovsbr0' bridge='ovsbr0'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='1c486148-a9ea-4000-8cfa-5fc9ce591f52'/>
      </virtualport>
      <target dev='vnet3'/>
      <model type='virtio'/>
      <driver name='vhost' queues='4'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </interface>

The last should be your case, but with tx-checksumming: on it is working just nice.
I've run iperf, netperf and uperf over that connection  all good.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1629053

Title:
  ovs bridge breaking TCP between two virtio net devices when checksum
  offload on

Status in openvswitch package in Ubuntu:
  New

Bug description:
  == Comment: #0 - Gowrishankar Muthukrishnan - 2016-09-29 13:06:10 ==
  ---Problem Description---

  Two kvm guests, each booted with a vnic (virtio_net) in ovs bridge in host seem 
  to fail in establishing TCP when virtio turned on for tcp checksum offload.
  Where as, these guests are able to complete TCP when vnics are over linux
  bridge in host.

  tcpdump on destination guest that confirms SYN retransmission, in OVS
  case:

  12:21:56.588835 52:54:c1:a2:7c:63 > 52:54:c1:a2:7c:53, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 49483, offset 0, flags [DF], proto TCP (6), length 60)
      10.0.126.3.51944 > 10.0.126.2.22: Flags [S], cksum 0x1034 (incorrect -> 0x0482), seq 621922575, win 29200, options [mss 1460,sackOK,TS val 329796 ecr 0,nop,wscale 7], length 0
  12:21:57.588220 52:54:c1:a2:7c:63 > 52:54:c1:a2:7c:53, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 49484, offset 0, flags [DF], proto TCP (6), length 60)
      10.0.126.3.51944 > 10.0.126.2.22: Flags [S], cksum 0x1034 (incorrect -> 0x0388), seq 621922575, win 29200, options [mss 1460,sackOK,TS val 330046 ecr 0,nop,wscale 7], length 0

  tcpdump on destination guest that confirms SYN/SYN-ACK/ACK , in bridge
  case:

  12:34:53.140513 52:54:c1:a2:7c:64 > 52:54:c1:a2:7c:54, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 57464, offset 0, flags [DF], proto TCP (6), length 60)
      10.1.126.3.41722 > 10.1.126.2.22: Flags [S], cksum 0x1036 (incorrect -> 0xb8a4), seq 1286151909, win 29200, options [mss 1460,sackOK,TS val 523933 ecr 0,nop,wscale 7], length 0
  12:34:53.140729 52:54:c1:a2:7c:54 > 52:54:c1:a2:7c:64, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
      10.1.126.2.22 > 10.1.126.3.41722: Flags [S.], cksum 0x1036 (incorrect -> 0xe19a), seq 3214677586, ack 1286151910, win 28960, options [mss 1460,sackOK,TS val 524786 ecr 523933,nop,wscale 7], length 0
  12:34:53.141253 52:54:c1:a2:7c:64 > 52:54:c1:a2:7c:54, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl 64, id 57465, offset 0, flags [DF], proto TCP (6), length 52)
      10.1.126.3.41722 > 10.1.126.2.22: Flags [.], cksum 0x102e (incorrect -> 0x80a1), ack 1, win 229, options [nop,nop,TS val 523934 ecr 524786], length 0

  ---uname output---
  Linux ip9-114-219-126gate 4.4.0-9136-generic #55-Ubuntu SMP Fri Aug 26 05:56:24 UTC 2016 ppc64le ppc64le ppc64le GNU/Linux

  Machine Type = P8 LE 

  ---Steps to Reproduce---
   1. Create a ovs bridge "ovsbr0"
  2. Create a kvm guest (preferably same as host - Ubuntu 16.10) carrying a vnic
     (as shown below) in ovsbr0

     <interface type='bridge'>
        <mac address='52:54:c1:a2:7c:53'/>
        <source bridge='ovsbr0'/>
        <virtualport type='openvswitch'/>
        <model type='virtio'/>
     </interface>

     Similarly create another kvm guest but with vnic of different mac address.

  3. Assign a static IP over vnic inside each guest.
     For eg, if enp0s3 is vnic:
     sudo ifconfig enp0s3 10.0.126.2/24 up (in guest 1)
     sudo ifconfig enp0s3 10.0.126.3/24 up (in guest 2)

  4. Ping and ensure ARP completes.
     ping 10.0.126.3 (from guest 2)

  5. Check tcp scum offload ON as by default.
     ethtool -k enp0s3

     Features for enp0s3:
      rx-checksumming: on [fixed]
      tx-checksumming: on
  	tx-checksum-ipv4: off [fixed]
  	tx-checksum-ip-generic: on
  	tx-checksum-ipv6: off [fixed]
  	tx-checksum-fcoe-crc: off [fixed]
  	tx-checksum-sctp: off [fixed]

  6. Check if TCP establishes between two guests on above IPs, using nc (netcat).
     nc 10.0.126.2 22

     As you would not see "SSH-2.0-OpenSSH_7.3p1 Ubuntu-1" as response, this 
     confirms TCP over port did not establish.

     You may run tcpdump on each port while running nc to confirm invalid 
     checksum received in tcp (as shown in problem description).

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1629053/+subscriptions