[Bug 1568627] [NEW] OpenvSwitch with DPDK brings all VirtIO NICs down, software reboot also doesn't work.

Sun Apr 10 22:47:33 UTC 2016

Public bug reported:

Guys,

 Soon as we start OpenvSwitch with DPDK inside of a KVM guest, it brings
down all the VirtIO NICs! Including non-DPDK NIC VirtIO devices!

 The problem is easy to reproduce, I'm seeing it on two different hosts,
a Dell server and a Macbook Pro, both running Xenial on host and on VM.

 Also, if DPDK Openvswitch options are enabled on
/etc/default/openvswitch, soft reboot stops working! Very weird...

 Steps to reproduce:

 1- At the host, install Ubuntu KVM by running:

 sudo apt install ubuntu-virt

 2- Create two extra Libvirt networks, called "subscriber" and "internet", list this:

tmartins at blade:~$ virsh net-list
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 default              active     yes           yes
 internet             active     yes           yes
 subscriber           active     yes           yes

 3- Create a Xenial VM with 3 vNIC (Virtio), wire it against those 3 above netorks.

 4- Inside of the Xenial VM, you might see the PCI IDs like this:

00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:09.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0a.0 Ethernet controller: Red Hat, Inc Virtio network device

 5- Install OpenvSwitch with DPDK:

 sudo apt install openvswitch-switch-dpdk

 6- Configure DPDK:

 sudo vi /etc/dpdk/dpdk.conf  # Enable NR_2M_PAGES=64

 7- Select the PCI devices that you want to bind to DPDK drivers:

 sudo vi /etc/dpdk/interfaces

pci 0000:00:09.0 uio_pci_generic
pci 0000:00:0a.0 uio_pci_generic

 8- Reboot it

 NOTE: After rebooting it, you'll see by running "dpdk_nic_bind
--status", that the latest two VirtIO NIC cards are using DPDK-
compatible drivers. So far, so good...

 9- Configure OpenvSwitch to use DPDK

 run:

 update-alternatives --set ovs-vswitchd /usr/lib/openvswitch-switch-dpdk
/ovs-vswitchd-dpdk

 sudo vi /etc/default/openvswitch-switch

 Uncomment the latest line, like this:

DPDK_OPTS='--dpdk -c 0x1 -n 4'

 10- Hit the bug!

 Run:

 service openvswitch-switch stop

 service openvswitch-switch start   # BOOM!!!

 From this point, there are two problems:

1- All VirtIO NIC cards loses connectivity, not only the DPDK ones! But
also, vNICs with the regular Linux VirtIO drivers!

2- Soft reboot does not work.

 That's it! What am I missing here?

 Workaround:

 1- Disable OpenvSwitch DPDK options inside of /etc/default/openvswitch-
switch, just comment it, and force-reboot the VM. After this, VM
connectivity is restored (on its first vNIC) and soft reboot also works
again.

 I made a video about this! It is on Youtube:

 https://youtu.be/sSBqYUVsO8U

 I really appreciate any help!

Cheers!
Thiago

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: openvswitch-switch-dpdk 2.5.0-0ubuntu1
ProcVersionSignature: Ubuntu 4.4.0-18.34-generic 4.4.6
Uname: Linux 4.4.0-18-generic x86_64
ApportVersion: 2.20.1-0ubuntu1
Architecture: amd64
Date: Sun Apr 10 18:25:34 2016
InstallationDate: Installed on 2016-02-01 (69 days ago)
InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Alpha amd64 (20160111)
SourcePackage: openvswitch
UpgradeStatus: No upgrade log present (probably fresh install)

** Affects: openvswitch (Ubuntu)
     Importance: Undecided
         Status: New

** Tags: amd64 apport-bug xenial

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1568627

Title:
  OpenvSwitch with DPDK brings all VirtIO NICs down, software reboot
  also doesn't work.

Status in openvswitch package in Ubuntu:
  New

Bug description:
  Guys,

   Soon as we start OpenvSwitch with DPDK inside of a KVM guest, it
  brings down all the VirtIO NICs! Including non-DPDK NIC VirtIO
  devices!

   The problem is easy to reproduce, I'm seeing it on two different
  hosts, a Dell server and a Macbook Pro, both running Xenial on host
  and on VM.

   Also, if DPDK Openvswitch options are enabled on
  /etc/default/openvswitch, soft reboot stops working! Very weird...

   Steps to reproduce:

   1- At the host, install Ubuntu KVM by running:

   sudo apt install ubuntu-virt

   2- Create two extra Libvirt networks, called "subscriber" and "internet", list this:

  tmartins at blade:~$ virsh net-list
   Name                 State      Autostart     Persistent
  ----------------------------------------------------------
   default              active     yes           yes
   internet             active     yes           yes
   subscriber           active     yes           yes