[Bug 2059976] Re: unmkinitramfs is very slow on initrd from ubuntu 23.10+, solution proposed

[Benjamin Drung]

The result is promising. I tested on my Ryzen 5700G desktop with the
mantic initrd:

```
$ ls /boot/initrd.img-6.5.0-26-generic -l
-rw-r--r-- 1 root root 106445830 Mär 18 18:30 /boot/initrd.img-6.5.0-26-generic
$ time lsinitramfs /boot/initrd.img-6.5.0-26-generic > /dev/null

real	0m13,960s
user	0m17,944s
sys	0m5,098s
$ time ./unmkinitramfs-turbo -l /boot/initrd.img > /dev/null

real	0m1,824s
user	0m2,099s
sys	0m0,915s
$ time cargo run > /dev/null

real	0m0,190s
user	0m0,037s
sys	0m0,147s
$ cargo build --release
$ time target/release/init-cpio > /dev/null

real	0m0,155s
user	0m0,027s
sys	0m0,121s
```

Cutting the time to list the content from roughly 14 second down to 160
ms (1% of the time or factor of 90). Compared to your unmkinitramfs-
turbo it is still over 11 times faster.

Okay, I still have to add some consistency check (check the magic), but
I can still optimize the code. But before doing that, I need to clean up
the code to publish it somewhere.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to initramfs-tools in Ubuntu.
https://bugs.launchpad.net/bugs/2059976

Title:
  unmkinitramfs is very slow on initrd from ubuntu 23.10+, solution
  proposed

Status in initramfs-tools package in Ubuntu:
  New

Bug description:
  Since 23.10 the initrd consists of 4 embedded cpio:
  - amd microcode
  - intel microcode
  - firmware + kernel modules
  - rest (compressed)

  Previously it was:
  - amd microcode
  - intel microcode
  - rest (compressed)

  There is no archive size header for cpio, so unmkinitramfs has to
  parse headers of each entry in uncompressed archives.

  > time unmkinitramfs 23.04/{initrd,unpack}

  real    0m3.625s
  user    0m1.595s
  sys     0m3.467s

  > time unmkinitramfs 23.10/{initrd,unpack}

  real    0m15.982s
  user    0m20.174s
  sys     0m6.995s

  > for dir in 23.04/unpack/early*; do printf '%-20s %s\n' "$dir" "$(find "$dir" | wc -l)"; done
  23.04/unpack/early   5
  23.04/unpack/early2  6

  > for dir in 23.10/unpack/early*; do printf '%-20s %s\n' "$dir" "$(find "$dir" | wc -l)"; done
  23.10/unpack/early   5
  23.10/unpack/early2  6
  23.10/unpack/early3  2874

  
  Here https://github.com/slowpeek/unmkinitramfs-turbo I've made it a lot faster with two changes:
  - bash instead of sh for bash's "read -N"
  - xxd dump of initrd instead of raw binary data

  
  I wonder, if the developers are interested in accepting my improvements into the code base?

  
  > time unmkinitramfs-classic-turbo 23.10/{initrd,unpack}

  real    0m2.209s
  user    0m2.161s
  sys     0m1.341s

  > time unmkinitramfs-turbo 23.10/{initrd,unpack}
   early .. +
  early2 .. +
  early3 .. +
    main .. +

  real    0m1.973s
  user    0m2.094s
  sys     0m1.244s

  
  All tools in the repo, even the classic one, feature -s/--scan option. With it, the tools only dump offset + size for embedded archives, nothing is unpacked. It can be used to measure parsing speed.

  In the repo there are three tools:
  - unmkinitramfs-classic: the original tool
  - unmkinitramfs-classic-turbo: the improved tool with minimal changes
  - unmkinitramfs-turbo: the improved tool with many changes to my tastes, extra dependency on "file" and some additional features (-ss option, decompression with 7za)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/2059976/+subscriptions