[DC LoCo] Testing 12.04 LTS development branch on system with IOMMU and SR-IOV

[Ken Stailey]

Works!

1. install 12.04 LTS (no PPA's or other modifications)
2. configure igb via
/etc/modprobe.d/igb.conf contains:
options igb max_vfs=7
/etc/modprobe.d/blacklist-igbvf.conf contains:
blacklist igbvf
3. put those configuration files in the initrd
sudo update-initramfs -k all -t -u
4. reboot
5. lspci will display "Virtual Function" repeatedly.


Now to configure a KVM/Qemu VM guest to use a VF NIC via PCI pass-through.
________________________________
From: Ken Stailey <kstailey at yahoo.com>
To: DC Ubuntu <Ubuntu-us-dc at lists.ubuntu.com> 
Sent: Saturday, January 14, 2012 7:56 AM
Subject: Re: [DC LoCo] Testing 12.04 LTS development branch on system with IOMMU and SR-IOV



In case you were wondering, a simple explanation of what SR-IOV accomplishes is that it allows a single network card to appear as several network cards.

A separate technology "PCI pass-through" allows a virtual machine to access a physical PCI card directly.  In the case of a PCI network card it is much faster for a virtual machine to use a physical network card than to emulate one in software. 


If you know what an MMU is and what a south bridge is, you can think of the IOMMU as an MMU for the south bridge.  The IOMMU takes care of doing the memory remapping required to allow a virtual machine to directly access the PCI card.

By combining SR-IOV and PCI pass-though several virtual machines can directly access the same physical network card.  Otherwise you would need a VM host with a large number of network cards.