[Bug 1568627] Re: OpenvSwitch with DPDK brings all VirtIO NICs down, software reboot also doesn't work.

[ChristianEhrhardt]

Hi Thiago,
as James said this is a known dpdk behaviour - not special to OpenVswitch.

Any other DPDK tool like l2fwd or such would have killed you just the same.
We documented it in the serverguide.
No release for 16.04 yet but you can take a look at http://bazaar.launchpad.net/~ubuntu-core-doc/serverguide/trunk/view/head:/serverguide/C/network-config.xml

Maybe you can bzr branch lp:serverguide and render it for yourself until it is released.
It contains all the pitfalls I found so far.

Most of the time we also added notes to the .conf files, but not yet in this case.
James - what do you think should we add a warning in /etc/default/openvswitch-switch ?

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1568627

Title:
  OpenvSwitch with DPDK brings all VirtIO NICs down, software reboot
  also doesn't work.

Status in openvswitch package in Ubuntu:
  New

Bug description:
  Guys,

   Soon as we start OpenvSwitch with DPDK inside of a KVM guest, it
  brings down all the VirtIO NICs! Including non-DPDK NIC VirtIO
  devices!

   The problem is easy to reproduce, I'm seeing it on two different
  hosts, a Dell server and a Macbook Pro, both running Xenial on host
  and on VM.

   Also, if DPDK Openvswitch options are enabled on
  /etc/default/openvswitch, soft reboot stops working! Very weird...

   Steps to reproduce:

   1- At the host, install Ubuntu KVM by running:

   sudo apt install ubuntu-virt

  
   2- Create two extra Libvirt networks, called "subscriber" and "internet", list this:

  tmartins at blade:~$ virsh net-list
   Name                 State      Autostart     Persistent
  ----------------------------------------------------------
   default              active     yes           yes
   internet             active     yes           yes
   subscriber           active     yes           yes

  
   3- Create a Xenial VM with 3 vNIC (Virtio), wire it against those 3 above netorks.

  
   4- Inside of the Xenial VM, you might see the PCI IDs like this:

  00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
  00:09.0 Ethernet controller: Red Hat, Inc Virtio network device
  00:0a.0 Ethernet controller: Red Hat, Inc Virtio network device

   5- Install OpenvSwitch with DPDK:

   sudo apt install openvswitch-switch-dpdk

  
   6- Configure DPDK:

   sudo vi /etc/dpdk/dpdk.conf  # Enable NR_2M_PAGES=64

  
   7- Select the PCI devices that you want to bind to DPDK drivers:

   sudo vi /etc/dpdk/interfaces

  pci 0000:00:09.0 uio_pci_generic
  pci 0000:00:0a.0 uio_pci_generic

   8- Reboot it

   NOTE: After rebooting it, you'll see by running "dpdk_nic_bind
  --status", that the latest two VirtIO NIC cards are using DPDK-
  compatible drivers. So far, so good...

   9- Configure OpenvSwitch to use DPDK

   run:

   update-alternatives --set ovs-vswitchd /usr/lib/openvswitch-switch-
  dpdk/ovs-vswitchd-dpdk

   sudo vi /etc/default/openvswitch-switch

   Uncomment the latest line, like this:

  DPDK_OPTS='--dpdk -c 0x1 -n 4'

   10- Hit the bug!

   Run:

   service openvswitch-switch stop

   service openvswitch-switch start   # BOOM!!!

   From this point, there are two problems:

  1- All VirtIO NIC cards loses connectivity, not only the DPDK ones!
  But also, vNICs with the regular Linux VirtIO drivers!

  2- Soft reboot does not work.

   That's it! What am I missing here?

   Workaround:

   1- Disable OpenvSwitch DPDK options inside of /etc/default
  /openvswitch-switch, just comment it, and force-reboot the VM. After
  this, VM connectivity is restored (on its first vNIC) and soft reboot
  also works again.

   I made a video about this! It is on Youtube:

   https://youtu.be/sSBqYUVsO8U

   I really appreciate any help!

  Cheers!
  Thiago

  ProblemType: Bug
  DistroRelease: Ubuntu 16.04
  Package: openvswitch-switch-dpdk 2.5.0-0ubuntu1
  ProcVersionSignature: Ubuntu 4.4.0-18.34-generic 4.4.6
  Uname: Linux 4.4.0-18-generic x86_64
  ApportVersion: 2.20.1-0ubuntu1
  Architecture: amd64
  Date: Sun Apr 10 18:25:34 2016
  InstallationDate: Installed on 2016-02-01 (69 days ago)
  InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Alpha amd64 (20160111)
  SourcePackage: openvswitch
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/1568627/+subscriptions