[Bug 1559408] [NEW] Many PMD driver options are disabled, including "BNX2X"!
Thiago Martins
thiagocmartinsc at gmail.com
Sat Mar 19 07:00:22 UTC 2016
Public bug reported:
Guys,
After deep research on the following problem:
Xenial - OpenvSwitch with DPDK binding to 10G NIC, not working:
https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2016-March/016287.html
I realized that, going against the DPDK documentation:
http://dpdk.org/doc/guides/nics/bnx2x.html
The option "CONFIG_RTE_LIBRTE_BNX2X_PMD" is DISABLED by default! Not
enabled, as the doc points (CONFIG_RTE_LIBRTE_BNX2X_PMD (default y))...
Take a look:
---
cd ~/sources/dpdk
apt source dpdk
cd dpdk-2.2.0
$ grep PMD config/common_linuxapp | grep ^CONFIG | grep n
CONFIG_RTE_LIBRTE_MLX4_PMD=n
CONFIG_RTE_LIBRTE_MLX5_PMD=n
CONFIG_RTE_LIBRTE_BNX2X_PMD=n
CONFIG_RTE_LIBRTE_NFP_PMD=n
CONFIG_RTE_LIBRTE_PMD_SZEDATA2=n
CONFIG_RTE_LIBRTE_PMD_PCAP=n
CONFIG_RTE_LIBRTE_PMD_XENVIRT=n
CONFIG_RTE_LIBRTE_PMD_QAT=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=n
CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n
CONFIG_RTE_LIBRTE_PMD_AESNI_MB_DEBUG=n
CONFIG_RTE_TEST_PMD_RECORD_CORE_CYCLES=n
CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n
---
I strong believe that this is the main problem with OpenvSwitch + DPDK
on Ubuntu, that causes OVS to not find any "dpdkX" interfaces! Even if
DPDK-Compatible driver is there, being used!!!
So, I added the following line:
---
-e 's,(CONFIG_RTE_LIBRTE_BNX2X_PMD=).*,\1y,' \
---
To DPDK's debian/rules file, it now looks like this:
---
build-config:
dh_testdir
$(MAKE) O=$(DPDK_STATIC_DIR) T=$(DPDK_CONFIG) config
sed -ri -e 's,(RTE_MACHINE=).*,\1"default",' \
-e 's,(RTE_NEXT_ABI=).*,\1n,' \
-e 's,(RTE_APP_TEST=).*,\1n,' \
-e 's,(CONFIG_RTE_EAL_IGB_UIO=).*,\1n,' \
-e 's,(CONFIG_RTE_KNI_KMOD=).*,\1n,' \
-e 's,(CONFIG_RTE_BUILD_COMBINE_LIBS=).*,\1y,' \
-e 's,(CONFIG_RTE_LIBRTE_BNX2X_PMD=).*,\1y,' \
-e 's,(LIBRTE_PMD_PCAP=).*,\1y,' \
-e 's,(LIBRTE_PMD_XENVIRT=).*,\1y,' \
$(DPDK_STATIC_DIR)/.config
---
Note: There is a need to install:
sudo apt install zlib1g-dev
After that, now, I can see a very different behavior! Still not
working... lol
But some progress, look, the error on OVS log is very different now:
---
ovs-ctl[2170]: EAL: PCI device 0000:01:00.0 on NUMA socket 0
ovs-ctl[2170]: EAL: probe driver: 14e4:168a rte_bnx2x_pmd
ovs-vswitchd[2551]: EAL: PCI device 0000:01:00.0 on NUMA socket 0
ovs-ctl[2170]: EAL: PCI memory mapped at 0x7f6180000000
ovs-ctl[2170]: EAL: PCI memory mapped at 0x7f6180800000
ovs-ctl[2170]: EAL: PCI memory mapped at 0x7f6181000000
ovs-vswitchd[2551]: EAL: probe driver: 14e4:168a rte_bnx2x_pmd
ovs-ctl[2170]: ovs-vswitchd: /home/ubuntu/sources/dpdk/dpdk-2.2.0/drivers/net/bnx2x/bnx2x_ethdev.c:453: bnx2x_common_dev_init: Assertion `sc->firmware' failed.
ovs-vswitchd[2551]: EAL: PCI memory mapped at 0x7f6180000000
ovs-vswitchd[2551]: EAL: PCI memory mapped at 0x7f6180800000
ovs-vswitchd[2551]: EAL: PCI memory mapped at 0x7f6181000000
ovs-ctl[2170]: Aborted (core dumped)
---
So, I tried something different, changed /etc/dpdk/interfaces from
this:
---
pci 0000:01:00.0 uio_pci_generic
pci 0000:01:00.1 uio_pci_generic
---
To this:
---
pci 0000:01:00.0 vfio-pci
pci 0000:01:00.1 vfio-pci
---
And now, a completely different, error message (still not working):
---
ovs-vswitchd[2950]: EAL: PCI device 0000:01:00.0 on NUMA socket 0
ovs-vswitchd[2950]: EAL: probe driver: 14e4:168a rte_bnx2x_pmd
ovs-vswitchd[2950]: EAL: 0000:01:00.0 VFIO group is not viable!
ovs-vswitchd[2950]: EAL: Error - exiting with code: 1
ovs-vswitchd[2950]: Requested device 0000:01:00.0 cannot be used
---
It looks much better with VFIO! There is no more "core dumped"!
But still not working.
So, I'm opening this bug report, so you guys can enable the PMD drivers
for all the remaining drivers, like BNX2X, MLX4 & 5, XENVIRT, QAT, PCAP
and etc (maybe I am missing other DPDK options that we must enable! Lets
take a deep look into those disabled options)...
However, I'm curious about the usage of BNX2X when with
"uio_pci_generic", what is that firmware error followed by a core dump?
Very intriguing...
BTW, when with VFIO, I added the following options "iommu=pt
intel_iommu=on" to /etc/default/grub, otherwise, VFIO doesn't work.
And, after reading the VFIO OVS error message, "VFIO group is not
viable", I google for it and I found the following blog post about this:
[IOMMU] The error of "VFIO group is not viable":
http://danny270degree.blogspot.com.br/2015/12/iommu-error-of-vfio-group-is-not-viable.html
So, I think that I am almost there! It is just a matter of configuring
this VFIO group thing but, I have no idea about how to do that... :-P
Christian is using ixgbe driver, which have PMD enabled by default!
That's why his setup works (ixgbe), and mine doesn't (bnx2x), am I
right? Look:
---
.../dpdk-2.2.0$ grep -i ixgbe_pmd config/common_linuxapp
CONFIG_RTE_LIBRTE_IXGBE_PMD=y
---
I can't think about any other reason for this problem... PMD is
disabled for many drivers!
It is Friday night, long week... I'll play with this during weekend,
for sure! ^_^
Cheers!
Thiago
** Affects: dpdk (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to dpdk in Ubuntu.
https://bugs.launchpad.net/bugs/1559408
Title:
Many PMD driver options are disabled, including "BNX2X"!
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1559408/+subscriptions
More information about the Ubuntu-server-bugs
mailing list