Revisiting default initramfs compression

[Heinrich Schuchardt]

On 12/8/21 09:12, Julian Andres Klode wrote:
> Hi all,
> 
> some time ago, the default compressor for initramfs was changed
> from lz4 -9 to zstd -19. This caused significant problems:
> 
> - it is very slow
> - it uses a lot of memory
> 
> The former is a problem for everyone, the latter means that
> zstd just crashes on a Pi Zero.
> 
> This is an analysis of what we have in terms of time spent,
> memory spent, and file size achieved, and where we can
> go from here.
> 
> # Comparison of different compression levels
> 
> ## Desktop (ThinkPad T480s, jammy)
> 
> level    usertime   elapsed memory fileSize
> lz4         9.65s    11.09s    12M      64M
> -1          5.69s	  6.99s    24M      57M
> -6         12.59s     8.58s    99M      47M
> -12        19.85s    10.82s   249M      41M
> -19        71.29s    26.95s   519M      35M
> 
> -> I believe that somewhere around -12 is a decent
>     compromise between size and speed.
> 
> ## Pi 4 (arm64, focal)
> 
> Times have been measured for mkinitramfs only. A full
> update-initramfs call spends much more time copying
> some firmware bits to boot partition with flash-kernel
> 
> level    usertime   elapsed memory fileSize
> lz4        21.10s    64.85s    21M      29M
> -1         13.73s    44.55s    21M      27M
> -6         26.07s    49.09s    91M      24M
> -12        48.18s    54.67s   203M      22M
> -19       130.07s    92.80s   350M      20M
> 
> -> 6 is essentially free if the Pi 4 is idle. Nice.
> -> -6 is still 20% of total RAM of a Pi 0
> -> There's no meaningful difference between -6 and -12
>     in terms of time elapsed. -6 uses 116% CPU, -12 uses
>     145% CPU.
> 
> ## Adaptive compression
> 
> zstd also supports adaptive compression, compressing as hard as
> it can while not impacting I/O speed. So hardware with slow I/O
> like a Pi would compress harder to avoid idling.
> 
> This is somewhat suboptimal with recent update-initramfs though,
> as it first writes the cpio archive to the disk and then compresses
> it rather than doing it in a pipe where that would be more
> meaningful.
> 
> Question: Does zstd --adapt adapt to memory available?
> 
> # Way(s) forward
> 
> To remedy the issue the proposal is to build with
> 
> - zstd -1 on hardware with 512 MB or less memory
> - zstd between -1 and -19 on other hardware
> - zstd -19 during image building
> 
> Finding the right level between -1 and -19 is hard. The more
> cores you have, the less penalty you pay for higher level.
> 
> Going for adaptive compression would remove the guess work, but
> will result in larger images on faster machines. Maybe that's
> fine, though - they probably have more space on /boot anyway?
> 
> If we want to aim for 5% of total memory, we should probably
> aim for something like:
> 
> -1  on <= 512MB
> -6  on <= 2 GB (or --adapt=min=1,max=6)
> -12 on the rest (or --adapt=min=12)

The memory should only be one factor to be taken into account. The 
SiFive Unmatched and the RPi 4 boards both have 8 GiB but slow CPUs. It 
would be advisable to use a lower compression level on these.

Could the post install script of initramfs-tools measure the compression 
speed and adjust the compression level?

Best regards

Heinrich

> 
> It's clear that in all cases, zstd -1 is at least better than the
> lz4 -9 we used before; both in terms of space used, and time spent.
> 
> # Concerns
> 
> Lowering the compression level will reduce the boot speed by fractions
> of a second on hardware with fast I/O.
>