Reducing initramfs size and speed up the generation

[Glenn Washburn]

Hi,

I recently came across the thread last month of the same name as the
subject and would like to bring up for discussion some ideas orthagonal
to all the ones presented there.

Some observations:
 1. I have observed that /lib/firmware in a sample initrd extracted on
    my system is about 25% of the uncompressed archive.
 2. In that same archive, 13% of the archive was occupied by files not
    in /lib/firmware nor /lib/modules
 3. I have currently many kernels installed on my system (really more
    than I reasonably need), which means many initrds with duplicate
    firmware files.
 4. GRUB supports concatenating initrds at boot time.
 5. GRUB can generate initrds from file paths (only regular files with
    0777 permissions).

So the first idea is that the data from (1) and (2) is replicated
across initrds and can ideally be shared amongst all initrds. This
could be implemented by having initramfs-tools generate a common
/boot/initrd-firmware and /boot/initrd-common files which will contain
the firmware files and base initrd system files to be shared amongst all
kernel specific initrds. Then GRUB will at boot time combine the
initrds together with a command like:

  initrd /boot/initrd.img-kver-generic /boot/initrd-firmware \
         /boot/initrd-common

This behavior, while until recently undocumented, has been available
for at least a decade, which might be a concern for older GRUB builds
using a config generated in this way.

The benefits to this are that the firmware and base initrds only need be
generated once regardless of number of kernels installed. And their
generation is decoupled from kernel upgrades/installs and each other.
So the firmware initrd only needs to be regenerated when the firmware
package is upgraded, and that need not trigger the base initrd to be
regenerated. Likewise, upgrading cryptsetup (or any other dependency of
the base initrd) need not cause the firmware initrd to be regenerated.
This approach could also be used with the early init microcode parts of
the initrd.

And even more radical approach to the firmware is to use point (5) and
have GRUB generate /lib/firmware at boot time. This would mean that the
firmware is always up to date with what is on the system to be booted
and both the disk space and generation time could be saved. This could
also be applied to the kernel modules to have them be generated on the
fly as well, saving the bulk of space and time used by initrd
generation. GRUB could also do on the fly decompression of on disk
compressed firmware modules that older kernels might not support
(although sounds like this might not be much of an issue if zstd is
back ported to the older kernel).

The caveat to this second approach is that the way GRUB currently builds
an initrd at run time is by passing a special argument to initrd for
*each* file path in the initrd. So for my sample initrd above, that
would be at least 921 arguments to initrd, one for each firmware file,
plus the other initrds. I've not looked to see if this could be a
problem, but it might. A better approach would be to allow GRUB to take
a directory and recursively graft that tree into a point in the initrd
tree (much like -graft-points of genisoimage).

Also, this assumes that files that initrds are generated from must be
accessible to GRUB. This might not be true in certain situations due to
lack of GRUB support, eg. the firmware files are EXT4 encrypted files,
but I suspect its the overwhelmingly common case.

Another issue I see with these two approaches is they rely on GRUB,
which might not be the bootloader of choice for Ubuntu on some
architectures and the bootloaders there may not support these features.
I think its reasonable to consider these approaches just for platforms
supporting GRUB.

Glenn