bad KVM network performance with 10GB

[Scholte van Mast, Rene]

Hello,
I have trouble with the network performance inside my virtual machines.

My KVM-Host machine is connected to a 10Gbit Network. All interfaces are configured to a mtu of 4132. On this host I have no problems and I can use the full bandwidth:

# iperf -c 10.10.80.100 -w 65536 -p 12345 -t 60 -P4
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-60.0 sec 18.8 GBytes 2.69 Gbits/sec
[ 5] 0.0-60.0 sec 15.0 GBytes 2.14 Gbits/sec
[ 6] 0.0-60.0 sec 19.3 GBytes 2.76 Gbits/sec
[ 3] 0.0-60.0 sec 15.1 GBytes 2.16 Gbits/sec
[SUM] 0.0-60.0 sec 68.1 GBytes 9.75 Gbits/sec


Inside a virtual machine don't reach this result:

# iperf -c 10.10.80.100 -w 65536 -p 12345 -t 60 -P 4
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 5.65 GBytes 808 Mbits/sec
[ 4] 0.0-60.0 sec 5.52 GBytes 790 Mbits/sec
[ 5] 0.0-60.0 sec 5.66 GBytes 811 Mbits/sec
[ 6] 0.0-60.0 sec 5.70 GBytes 816 Mbits/sec
[SUM] 0.0-60.0 sec 22.5 GBytes 3.23 Gbits/sec

I only can use 3,23Gbits of 10Gbits. I use the virtio driver for all of my vms, but I have also tried to use the e1000 nic device instead.

With starting the iperf performance test on multiple vms simultaneously I can use the full bandwidth of the kvm host's interface. But only one vm can't use the full bandwith. Is this a known limitation, or can I improve this performance?

Does anyone have an idea how I can improve my network performance? It's very important, because I want to use the network interface to boot all vms via AOE (ATA over Ethernet).

If I mount a harddisk via AOE inside a vm I get only this results:
Write |CPU |Rewrite |CPU |Read |CPU
102440 |10 |51343 |5 |104249 |3

On the KVM Host I get those results on a mouted AOE Device:
Write |CPU |Rewrite |CPU |Read |CPU
205597 |19 |139118 |11 |391316 |11

If I mount the AOE Device directly on the kvm-host and put a virtual harddisk-file in it I got the following results inside a vm using this harddisk-file:
Write |CPU |Rewrite |CPU |Read |CPU
175140 |12 |136113 |24 |599989 |29


Here more details about my environment:

KVM Version:
QEMU PC emulator version 0.12.3 (qemu-kvm-0.12.3), Copyright (c) 2003-2008 Fabrice Bellard
0.12.3+noroms-0ubuntu9

KVM Host Kernel:
2.6.32-22-server #36-Ubuntu SMP Thu Jun 3 20:38:33 UTC 2010 x86_64 GNU/Linux

KVM Host OS:
Distributor ID: Ubuntu
Description: Ubuntu 10.04 LTS
Release: 10.04
Codename: lucid

KVM Guest Kernel:
2.6.32-22-server #36-Ubuntu SMP Thu Jun 3 20:38:33 UTC 2010 x86_64 GNU/Linux

KVM Guest OS:
Distributor ID: Ubuntu
Description: Ubuntu 10.04 LTS
Release: 10.04
Codename: lucid

I would feel very happy if someone can help to solve my problem!

Best regards





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-devel-discuss/attachments/20100705/086642aa/attachment.html>