ZRAM, vm.swappiness and the future

[Leszek Lesner]

As some of you know there were some discussions on this mailinglist
regarding zram and vm.swappiness.
In this mail I want to write down all I know and learned about zram,
vm.swappiness and its cooperation.
This is intend as a help and a starting point for discussions for the
next release of Lubuntu which will be an LTS.

I have to thank Mélodie for the discussions about vm.swappiness which
made me dig deeper and deeper into it.
And I have to say I was wrong at least partly :P

So before talking about ZRAM a small comprehensive introduction what it is.
ZRAM is a compressed virtual swap device that lives in RAM. With this it
is possible to use and store more memory
that is physically available. It is used by Lubuntu in 13.10 by default.

What is vm.swappiness ? That is a far more complicated thing to explain.
Simply speaking it controls the balance between swapping out runtime
memory,
as opposed to dropping pages from the system page cache. It can have a
value from 0 to 100.
A lower value means that it will try not to swap out page cache but keep
it in memory.
A high value means try to more aggressively swap out unused page memory.

A discussion was started if Lubuntu should lower down the vm.swappiness
value or even set it down to 0
to improve system performance and responsiveness.
In my initial thought that I could give a universal answer on this topic
but I was wrong.
There a several scenarios we need to take a look at to see that there is
not an universal answer on this.

== The classical situation (before 13.10): ==
* RAM + SWAP (on HDD)

The RAM is backed up by a SWAP partition or SWAP file which is stored on
a harddrive or ssd.
As we all know disk i/o is a lot slower than RAM. On older machines and
slower disks this could lead to noticable
lags while swapping. But also on normal machines it is noticable. So for
this scenario it would be nice to tell the
kernel to avoid swapping as much as possible to avoid this bottleneck.
A low vm.swappiness or even 0 is a good setting here as the kernel keeps
filling the ram until it is almost full and then
starts swapping out which could make the system laggy for a few seconds
and while writing data to swap. So there is an unavoidable
bottleneck here but there is also the possibility that with enough ram
and a low memory footprint of the apps and desktop that
the user will not reach the point of swapping.

== The new modern situation (with 13.10¹): ==
* RAM + ZRAM

The RAM is backed up by a SWAP partition which is stored on a virtual
compressed ram device. Under Lubuntu
13.10 this device can use half of the physically available memory.
Writing and reading to and from RAM is a hell lot faster than to disk
(even to an SSD). This means there is no bootlneck
when the system swaps out to ZRAM with its default swappiness settings.
Setting the swappiness lower here would make a change but
a very minor only (noticable only via benchmarks). And lowering it here
might lead to a faster reaching of the second
bottleneck. So avoiding swapping and letting the RAM be filled and
swapping late especially when RAM is very full will slow down the
system and produces noticable lags. Those lags might be even harder in
this case because the pages need to be compressed into RAM and
when new pages need to be placed into RAM at the same time this might
lead to a long cpu 100% compressing decompressing moving loop.
Letting the swappiness at default (60) or even set it higher to 100
would help avoid this bottleneck as swapping early would avoid
RAM running full too fast.
Whats the downside ?
The CPU has to compress and decompress more often (don't worry even a
Pentium 3 is very fast at it) and this causes more wake up times
for the CPU. (perhaps a downside for those who want to save every little
watt of there notebook battery)

== The combination of both situations : ==
* RAM + ZRAM + SWAP (on HDD)

The RAM is backed up by ZRAM and a SWAP partition/file. This is an usual
setting if you have very low RAM (like 256/512MB or 1 GB).
By default ZRAM is as Swap device has a higher priority than the swap
partition/file on hd. The swapping mechanism now works very clevely
and swaps out first and by default early to zram first until its full.
Then it will start swapping to hdd.
This will lead us to our bottleneck nr.1 again but in contrast to the
classical situation (scenario 1) it will get there a lot later :)
So is there a possibility to avoid this bottleneck by changing the
swappiness ? No not really.
Swappiness low would lead too faster filling in memory which will lead
(especially on low memory systems) lead to bootlneck 2 which tends
to make the system a lot laggier than bottleneck 1.
Setting swappiness higher here would be also only work until ZRAM is
filled and than would lead to bottleneck 1 again.
So overall I would recommend the default swappiness value here which
tends to keep the system in a good balance between bottleneck 1 and 2.

So all in all when you see this scenarios I have to say that the default
value of swappiness 60 makes the most sense as the default value
of lubuntu. Changing it for the next release is something I would not
recommend.

Seeing this one could ask wouldn't it be good to directly compress any
data written to RAM ?
And yes this is a possibility in the future with zcache (which is
basically a backend for cleancache which is a whole framework around it).
Zcache compresses every page cache written to RAM which will lead to
more available free memory.
And yes it is even possible to combine ZCache (Page Cache[Disk Cache and
so on]) and ZRAM (Application memory) which
could lead to even more available free RAM and might even make it
possible to completely get rid of SWAP partitions/files on hdd.
In the making is also frontswap which is basically speaking zram but
without the necessity to give it a fixed size. So it can dynamically
grow.

So the future looks very promising when it comes to low ram management
and we are on a good way.