Reducing initramfs size and speed up the generation

[TJ]

On 13/07/2023 23:30, Dimitri John Ledkov wrote:
> journald stores a copy of dmesg subject to its own retention and rotation
> policy, even when dmesg is cleared. Thus it should be reliable to have
> journal output for dmesg for multiple boots.
> 
> The issue however is that drivers and kernels can change on disk, and the
> next kernel ABI may need different or more firmware files. examples:
> 1) on desktop installs hwe-22.04 rolls major versions.
> 2) each Nvidia SRU update needs a new firmware file name

I did do some basic investigation on this issue and played with a few 
ideas; trying to find something simple.

At the time I considered the most reliable would be a file 
"firmware.ids" along the lines of pci.ids/usb.ids included in each 
kernel image binary package that links hardware ID to firmware as in:

firmware-file # comment
<tab>	PCI # bus name
<tab>	<tab>	Vendor
<tab>	<tab>	<tab>	Device1
<tab>	<tab>	<tab>	Device2
<tab>	USB
<tab>	<tab>	idVendor
<tab>	<tab>	<tab>	idProduct1
<tab>	<tab>	<tab>	idProduct2
<tab>	<tab>	<tab>	idProduct3

firmware-file # comment
...

Being able to create it at package build-time would have some advantages 
but I haven't given much thought to potential problems including 
build-time complications.

The challenge for "firmware.ids" is there is (currently) no automated 
way to generate it and would require either or both of kernel driver or 
linux-firmware to provide a machine-readable entry. The best I could 
come up with at the time was an additional line in linux-firmware WHENCE 
possibly of the form:

alias: Vendor:Device Vendor:Device ...

But that has a lot of issues, not least being that the list would need 
to be comprehensive and rely on human knowledge. A combination of what 
is loaded plus WHENCE:alias might work though.

The list of firmware files in a kernel module would be much more 
difficult to link to hardware IDs even though that is the real source of 
truth since there is no declarative link between them.

There is a sneaky way to side-step the need for a list: use the 
currently loaded list PLUS any firmware files newly declared by an 
included (in initrd.img) module that the current module doesn't declare.

That would mean an additional call to modinfo for each included kernel 
module and a diff which could be generated at kernel-install time via a 
script in /etc/kernel/install.d/ that generates the

/var/lib/firmware/${DMI_HASH}/${KERNEL_VERSION}.firmware

described in my reply to Benjamin.