Improving OOTB performance

Eero Tamminen oak at helsinkinet.fi
Fri May 18 22:23:19 UTC 2012


Hi,

(warning: long mail, better to browse to end before answering.)

On lauantai 19 toukokuu 2012, Len Ovens wrote:
> My experience with zram is that it is like very fast swap. CPU load is
> not a problem that I have found because memory seems to be the limit.
> Example is my netbook with 1Gig of ram and a 1.6ghz CPU. I start hitting
> the memory wall using CPU intensive synths (swappiness 0) yet the CPU
> load is only 33%

According to what?

Of the "top" output:
----
Tasks: 173 total,   1 running, 171 sleeping,   1 stopped,   0 zombie
Cpu(s):  0.7%us,  0.2%sy,  0.0%ni, 99.2%id,  0.0%wa,  0.0%hi,  0.0%si,  
----

You should be checking the load by checking how much your system
is idle from the "id" number in top output.

Otherwise one can accidentally e.g. ignore io-wait from the "wa" number.

(Space refreshes the top output if its default refresh interval is 
too slow.)


> and looking at what "ondemand" is doing the CPU is only
> running half speed (.8Ghz) anyway.

If your system doesn't have IO-waits and it's still slow, low
CPU speed is very suspicious.

Maybe kernel thinks zram swapping is IO and doesn't keep the HZ high
despite it actually using CPU?

Have you tried setting the CPU frequency scaling governor to "performance"
to see what effect that has on performance?

And how you're measuring CPU frequency?  CPU governor can be changing
the CPU frequency at very high frequency (several times a second), so
you need some kind of average of how much time is spent in each available
frequency.

E.g. mem-cpu-monitor from this repo:
	http://maemo.gitorious.org/maemo-tools/sp-memusage

Can provide that (it needs library from sp-measure repo which
is also under maemo-tools, debian packaging for them is in separate
"maemo-packaging" git branch, you need to merge "master" to them
before building Debian package).


> So zram is not a problem from that
> view. However, it is still swap and still slow enough the anything in
> the audio chain that gets swapped out (maybe just a sample file that has
> not been used for a bit) can stop all the audio till it swaps in again
> (from my own experience yes).

Are you sure it's swapped instead of e.g. code pages paging?

(swapping = both read & write, paging = disk reads; writes
can be slower on SSD, random seeks on rotating media.)


Are your applications doing a lot of disk read and writes which could
"poison" the kernel page cache and can cause additional swapping?

You could track that a bit with "vmstat 1".

If they don't need again the data they write to disk, they could
madvise() kernel that it doesn't need to cache them.  You can also
control that system wide to some extent by fiddling with the kernel
dirty writeback settings.


> So zram is not useful for audio as it is still not fast enough.

I wonder how much memory pressure your system has...

Gnome system-monitor gives fairly relevant memory usage information
(although it names its output columns so that one needs to guess a bit
map to what kernel SMAPS values they map to).


If you have some automated test-case which full effect to the system
on longer useage you want check out, this gets snapshots of
all possible system resources and its post-processing generates
very nice graphs out of that:
	http://maemo.gitorious.org/maemo-tools/sp-endurance

Here are short instructions on how to use it:
	http://harmattan-
dev.nokia.com/docs/library/html/guide/html/Developer_Library_Developing_for_Harmattan_Developer_tools_Performance_testing_tools_Using_sp-
endurance-postproc.html
    http://harmattan-
dev.nokia.com/docs/library/html/guide/html/Developer_Library_Developing_for_Harmattan_Developer_tools_Performance_testing_tools_Using_sp-
endurance.html


> Well designed audio apps should try to use memlock, but as I found out,
> some don't and it only takes one to put jackd on hold waiting for that
> sample. A missing word in audio is hard to hear and on stage may make no
> difference, but even a one second delay (with no sound at all BTW) is
> very noticeable and may even put that instrument out of sync with the
> rest of the band.

If they're locking their whole processes, that can be *very* wasteful[1]
usage of RAM.

[1] For example, typically processes use only few tens of kB from their,
by default 8MB, thread stacks.  If processes have lots of threads, that's
a lot of stupidly wasted memory.

And when memory is locked in RAM, you cannot analyze how much of it's
actually used (written/dirty) as you see everything program memory maps as
private...


> For the normal desktop experience xram is fantastic. The swapped out app
> just seems slow, not dead like with disk swap. So get more memory first
> then use zram.
> 
> These are my experiences as I have tried to test these tweaks on both of
> my (admittedly marginal) machines. I would love to hear other peoples
> experiences with audio applications.

Extensive container group (cgroups) config is a pain to setup, but with
that you can actually control which of the processes are swapped out or
even OOM killed, to prevent swapping of more important processes (in some 
other container group).

The main thing for getting good performance out of cgroups is to avoid
migrating processes between groups i.e. to have a fixed setup.


	- Eero



More information about the Ubuntu-Studio-devel mailing list