[Bug 1996489] Re: The unminimize command fails to reinstall packags with missing files in the `share` and `locale` directories

[John Chittum]

Thank you for reporting the issue and giving suggestions in great
detail! Also very timely as myself and another script maintainer were
just having conversations about the minimize/unminimize flow.

I've added `livecd-rootfs` package for tracking, as the script lives
within those build scripts. We'll take some time to dig into the
suggestions and issues and come back with more info.

** Also affects: livecd-rootfs (Ubuntu)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to livecd-rootfs in Ubuntu.
https://bugs.launchpad.net/bugs/1996489

Title:
  The unminimize command fails to reinstall packags with missing files
  in the `share` and `locale` directories

Status in cloud-images:
  New
Status in livecd-rootfs package in Ubuntu:
  New

Bug description:
  I am building a custom Docker image based on:

      https://github.com/microsoft/vscode-dev-
  containers/tree/main/containers/ubuntu

  I added a line that runs `yes | unminmize':

  https://github.com/nomirose/devcontainer/commit/1ba51c651c06b9ce71c73e7bf016939d989cfa2c#diff-4c1f94864a9642897e2fa6c1d532830b6d1a2ba9d6b6f2f149178dde32cf0e77R10-R11

  Here's from build log :

  https://github.com/nomirose/devcontainer/actions/runs/3451129967/jobs/5760151170#step:6:1565

  And here are the errors from the `uminmize` command:

      #9 47.54 Reinstalling packages with system documentation in /usr/share/doc/ ..
      #9 48.76 dpkg-query: error: --search needs at least one file name pattern argument
      #9 48.76
      #9 48.76 Use --help for help about querying packages.
      #9 48.80 Reading package lists...
      #9 49.49 Building dependency tree...
      #9 49.62 Reading state information...
      #9 49.76 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
      #9 49.76 Restoring system translations...
      #9 50.97 dpkg-query: error: --search needs at least one file name pattern argument
      #9 50.97
      #9 50.97 Use --help for help about querying packages.
      #9 51.01 Reading package lists...
      #9 51.71 Building dependency tree...
      #9 51.85 Reading state information...
      #9 52.00 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
      #9 53.18 Documentation has been restored successfully.
      #9 DONE 54.3s

  Specifically, the error is:

      dpkg-query: error: --search needs at least one file name pattern
  argument

  I copped `/usr/local/sbin/unminimize` to my local directory to see if
  I could debug what was doing on.

  When looking for packages with missing files in the `/usr/bin/man`
  directory, the script does this:

      dpkg -S /usr/share/man/ |sed 's|, |\n|g;s|: [^:]*$||' |
  DEBIAN_FRONTEND=noninteractive xargs apt-get install --reinstall -y

  However, for the `/usr/share/docs/` the script does this:

      dpkg --verify --verify-format rpm | awk '/..5......   \/usr\/share\/doc/ {print $2}' | sed 's|/[^/]*$||' | sort |uniq \
          | xargs dpkg -S | sed 's|, |\n|g;s|: [^:]*$||' | uniq | DEBIAN_FRONTEND=noninteractive xargs apt-get install --reinstall -y

  The script runs the same command for the `/usr/share/locale/`
  directory.

  In both cases, no packages ever make it to the `xargs dpkg -S`
  command, so the script throws an error. However, because the error
  occurs in a pipe, it is lost, and the script continues despite the
  `set -e` at the start (which is a bug in itself).

  However, with a little experimentation, I found that the original
  command for `/usr/local/bin` works perfectly for the
  `/usr/share/docs/` and `/usr/share/locale/` directories.

  Why not run the same command for all three directories? You could even
  wrap it in a function.

  However, you could improve performance by combining all three into a
  single command.

  The `unminimize` script notes:

      # Reinstallation takes place in two steps because a single dpkg --verified
      # command generates very long parameter list for "xargs dpkg -S" and may go
      # over ARG_MAX. Since many packages have man pages the second download
      # handles a much smaller amount of packages.

  I understand the concern about hitting `ARG_MAX`, but I think a better
  solution to that problem would be to use `xargs -n` to set a limit on
  the maximum number of arguments. This change would allow xargs to run
  `dpkg -S` as many times as need (to avoid hitting ARG_MAX).

  Doing it this way would also speed up the script.

  There are many packages with missing files in multiple minmized
  directories, meaning that as it stands, even if the bug was fixed,
  some packages would be re-installed twice, maybe even three times.

  (I have seen this for myself because I essentially wrote a shorter
  version of the script for my own use.)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/1996489/+subscriptions