Some Xen performance data related to the xen-acpi-processor patchset

[Stefan Bader]

We got a request[1] to backport some patches which will disable the usual
frequency scaling drivers and add a new xen specific one that will make
callbacks into the hypervisor to enable p-state control (the hypervisor
has no acpi table parser but needs to make those decisions).

I have been playing with the proposed patches[2] on an Intel i7 and one AMD
Opteron 6128 based system. And finally I also got around to write some CPU
intensive test (adding numbers in a loop for each visible VCPU) that gathers the
runtime statistics.

= Intel i7 @2.6GHz =

On this host, the acpi cpufreq driver would load on the unpatched dom0 kernel.
Checking the idle power usage on an external power meter showed a slightly
higher (~4-5W) value running the Xen hypervisor/dom0 than running the same
kernel on bare metal. I suspect this is because under Xen the maximum c-state
used in idle is c1. Using deeper c-states seems to cause wakeup issues in some
cases, so they are not enabled by default. The idle power usage is the same with
or without the xen-acpi-processor driver.

Running the test shows that without the xen-acpi-processor driver, all CPUs
remain at the lowest frequency (p9). With the new driver, the hypervisor is
detecting the guests CPU usage and some CPUs go up into p0. Unfortunately it
was not possible to check turbo mode usage as turbostat fails when running
in dom0 (some msr values seem to go the wrong way). The runtime of the test
is slightly better with the patched kernel than before.

i7  (patched) (unpatched)
M:  221.218s  224.958s
S:  .618s     .589s

(M = mean runtime, S = standard deviation of 10 runs)

= AMD Opteron 6128 @2GHz =

On this host, the cpufreq driver did never load. Likely for the same reason
for which the xen-acpi-processor driver failed to load (APIC ID on AMD start
with 0x10 while under Xen this was always set to 0x00 as it is with Intel).
So all CPUs where always running at top speed which looks like there is not
much to be expected from the patchset.

However, looking at the idle power usage, making the xen hypervisor do frequency
scaling did improve that vastly. Without the idle system would draw about 128W
but when frequency scaling was done, it went down to 105W. That is a lot of
power saved when doing nothing.

Since the 6128 does not support Turbo Core (similar feature as on Intel) the
expectation on the test runtime was low. But surprisingly it is also better on
the patched dom0.

AMD (patched) (unpatched)
M:  294.357s  298.872s
S:  1.477s    1.820s

I cannot really explain that. Maybe some sort of opposite effect due to all
CPUs running full speed and some with load which causes some throttling in order
to prevent thermal problems...?
AMD also has bigger deviations in runtime. But then the AMD system is a real
8 core while the i7 is a 4 core with hyperthreading. And AMD has differs
in C1E.

= Summary =

Guest CPU performance seems to be better in all testing and fixing the APIC IDs
does help a lot for the idle power consumption on AMD based hosts. The changes
are limited to code paths used when running as dom0 and no bad effects have been
observed. So it looks like worth and eligible for SRU into Precise.

-Stefan

[1] http://bugs.launchpad.net/bugs/898112
[2] git://kernel.ubuntu.com/smb/ubuntu-precise.git xen-acpi
    (note this branch currently also has some debugging patches)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/kernel-team/attachments/20120516/998b2a70/attachment.sig>