<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 7 March 2016 at 20:16, Martinx - ジェームズ <span dir="ltr"><<a href="mailto:thiagocmartinsc@gmail.com" target="_blank">thiagocmartinsc@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div class="h5"><div class="gmail_extra"><div class="gmail_quote">On 7 March 2016 at 17:28, Martinx - ジェームズ <span dir="ltr"><<a href="mailto:thiagocmartinsc@gmail.com" target="_blank">thiagocmartinsc@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div class="gmail_extra"><div class="gmail_quote">On 7 March 2016 at 16:40, Martinx - ジェームズ <span dir="ltr"><<a href="mailto:thiagocmartinsc@gmail.com" target="_blank">thiagocmartinsc@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hey guys,<div><br></div><div> I'm trying to create two OVS bridges powered by DPDK, on Xenial, but I'm facing problems...</div><div><br></div><div> First, there is no documentation about how to do this... So far, I found the following:</div><div><br></div><div><br></div><div><div> - This link doesn't how to bind real NIC:</div></div><div> <a href="https://software.intel.com/en-us/articles/using-open-vswitch-with-dpdk-on-ubuntu" target="_blank">https://software.intel.com/en-us/articles/using-open-vswitch-with-dpdk-on-ubuntu</a><br></div><div><br></div><div> - This one isn't specific to Ubuntu OVS + DPDK packages:</div><div> <a href="https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md" target="_blank">https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md</a></div><div><br></div><div> - Similar with above but, very old... Also not related to Ubuntu packages:</div><div> <a href="https://feiskyer.github.io/2015/03/01/open-vswitch-over-dpdk-on-ubuntu/" target="_blank">https://feiskyer.github.io/2015/03/01/open-vswitch-over-dpdk-on-ubuntu/</a></div><div><br></div><div><br></div><div> So, to begin with, I start with Intel.com guide, it worked! But without real NIC.</div><div><br></div><div> Then, I tried to bind it to real NIC, like this:</div><div><br></div><div> Added two NIC PCI id to "/etc/dpdk/interfaces" file:</div><div><br></div><div>---</div><div><div>pci 0000:01:00.0 uio-pci-generic</div><div>pci 0000:01:00.1 uio-pci-generic</div></div><div>---</div><div><br></div><div> Then, reboot, I can see both NICs using "DPDK-compatible driver" by running: "dpdk_nic_bind --status".</div><div><br></div><div> Trying to create new OVS Bridges with DPDK devices, failing:</div><div><br></div><div>---</div><div><div>root@xenial-1:~# ovs-vsctl add-br ovsbr0</div><div>root@xenial-1:~# ovs-vsctl set bridge ovsbr0 datapath_type=netdev</div><div>root@xenial-1:~# ovs-vsctl add-port ovsbr0 dpdk0 -- set Interface dpdk0 type=dpdk</div><div>ovs-vsctl: Error detected while setting up 'dpdk0'. See ovs-vswitchd log for details.</div></div><div>---</div><div><br></div><div>Log shows:</div><div><br></div><div>---</div><div><div>2016-03-07T19:34:07.532Z|00136|bridge|WARN|could not open network device dpdk0 (No such device)</div></div><div>---</div><div><br></div><div>---</div><div><div>root@xenial-1:~# ovs-vsctl show</div><div>2b84f2f7-6d3f-4d51-94a5-fe6d04f0a062</div><div> Bridge "ovsbr0"</div><div> Port "ovsbr0"</div><div> Interface "ovsbr0"</div><div> type: internal</div><div> Port "dpdk0"</div><div> Interface "dpdk0"</div><div> type: dpdk</div><div> error: "could not open network device dpdk0 (No such device)"</div><div> ovs_version: "2.5.0"</div></div><div>---<br></div><div><br></div><div> What am I missing?</div><div><br></div><div> I tried to use real NIC device name "em1", instead of "dpdk0", same results... Since "em1" does not even shows up on "ip link" list... DPDK toke it over...</div><div><br></div><div>Thanks!</div><div>Thiago</div></div>
</blockquote></div><br></div></div></div><div class="gmail_extra">Guys,</div><div class="gmail_extra"><br></div><div class="gmail_extra">I am also trying this:</div><div class="gmail_extra"><br></div><div class="gmail_extra"><span><div style="font-size:12.8px"> Added two NIC PCI id to "/etc/dpdk/interfaces" file:</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">---</div></span><div style="font-size:12.8px"><div>pci 0000:01:00.0 vfio-pci</div><div>pci 0000:01:00.1 vfio-pci</div></div></div><div class="gmail_extra">---</div><div class="gmail_extra"><br></div><div class="gmail_extra">According to: <a href="https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md" target="_blank">https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md</a> - but, it also doesn't work...</div><div class="gmail_extra"><br></div><div class="gmail_extra">When using "vfio-pci", the NIC doesn't even show up as a DPDK-compatible driver after running: "dpdk_nic_bind --status"...</div><div class="gmail_extra"><br></div><div class="gmail_extra">So, how to use "vfio-pci" with Ubuntu DPDK?</div><div class="gmail_extra"><br></div><div class="gmail_extra">Cheers!</div><div class="gmail_extra">Thiago</div></div>
</blockquote></div></div><div><br></div></div></div><div>After reading <a href="http://INSTALL.DPDK.md" target="_blank">INSTALL.DPDK.md</a> again, I realized that it only works with VFIO or IGB (UIO), Ubuntu DPDK package does not provides the "igb_uio" module, neither via DKMS. So, the only option is VFIO, right?</div><br><div>And, I am unable to use "vfio-pci" configured inside of "/etc/dpdk/interfaces" but, I managed to make it work manually.</div><div><br></div><div>I just filled a bug report against DPDK Ubuntu package related to this "vfio-pci" problem:</div><div><br></div><div>---</div><div>Usege of vfio-pci module via /etc/dpdk/interfaces doesn't work:<br></div><div><a href="https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1554214" target="_blank">https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1554214</a><br></div><div>---</div><div><br></div><div>So, after workaround the bug #1554214 (bug description have the instructions), I tried OpenvSwitch with DPDK + 10G NIC, again, still no success.</div><div><br></div><div>Same problem...</div><div><br></div><div>* DPDK using VFIO:</div><div><br></div><div>---</div><div><div>root@xenial-1:~# dpdk_nic_bind --status</div><div><br></div><div>Network devices using DPDK-compatible driver</div><div>============================================</div><div>0000:01:00.0 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' drv=vfio-pci unused=bnx2x</div><div>0000:01:00.1 'NetXtreme II BCM57800 1/10 Gigabit Ethernet' drv=vfio-pci unused=bnx2x</div></div><div>---</div><div><br></div><div>* OpenvSwitch with DPDK still failing:</div><div><br></div><div>---</div><div>service openvswitch-switch stop<br></div><div>service openvswitch-switch start<br></div><div><br></div><div>ps xfa | grep dpdk # I can see "ovs-vswit... --dpdk -c 0x3 -n 4 -- ...." running...<br></div><div><br></div><div><br></div><div><div>root@xenial-1:~# ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev</div><div>root@xenial-1:~# ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk</div><span class=""><div>ovs-vsctl: Error detected while setting up 'dpdk0'. See ovs-vswitchd log for details.</div></span></div><div>---</div><div><br></div><div>No luck... I can't wait to see it working in my lab!</div><div><br></div><div>Ubuntu rocks! Almost there... :-D</div><div><br></div><div>Best,</div><div>Thiago</div></div>
</blockquote></div><br></div><div class="gmail_extra">Hey guys,</div><div class="gmail_extra"><br></div><div class="gmail_extra"> I just found a minor issue with openvswitch-switch Init Script:</div><div class="gmail_extra"><br></div><div class="gmail_extra"> * Script "/etc/init.d/openvswitch-switch restart" ignores "DPDK_OPTS", while "service openvswitch-switch restart" works!<br></div><div class="gmail_extra"><br></div><div class="gmail_extra"> <a href="https://bugs.launchpad.net/ubuntu/+source/openvswitch-dpdk/+bug/1554275">https://bugs.launchpad.net/ubuntu/+source/openvswitch-dpdk/+bug/1554275</a><br></div><div class="gmail_extra"><br></div><div class="gmail_extra">-</div><div class="gmail_extra">Thiago</div></div>