[Bug 1876034] Re: apt update crashes with "KeyError: 'suite'" in cnf-update-db if index compression is enabled

[Steve Langasek]

The upload in the hirsute queue shows multiple changes outside of the
fix for compressed files in db/creator.py, including extensive changes
to the debian/ directory.  Have these been included by mistake?

** Changed in: command-not-found (Ubuntu Hirsute)
       Status: New => Incomplete

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to command-not-found in Ubuntu.
https://bugs.launchpad.net/bugs/1876034

Title:
  apt update crashes with "KeyError: 'suite'" in cnf-update-db if index
  compression is enabled

Status in command-not-found package in Ubuntu:
  Fix Released
Status in command-not-found source package in Focal:
  Fix Committed
Status in command-not-found source package in Hirsute:
  Incomplete
Status in command-not-found source package in Impish:
  Fix Released

Bug description:
  [Impact]
  On all Ubuntu releases since at least 19.10, enabling compressed APT package list storage with command-not-found installed leads to this crash in the cnf-update-db hook called by apt update:

  Traceback (most recent call last):
    File "/usr/lib/cnf-update-db", line 26, in <module>
      col.create(db)
    File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 93, in create
      self._fill_commands(con)
    File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 127, in _fill_commands
      self._parse_single_commands_file(con, fp)
    File "/usr/lib/python3/dist-packages/CommandNotFound/db/creator.py", line 165, in _parse_single_commands_file
      suite=tagf.section["suite"]
  KeyError: 'suite'

  (To enable compressed APT list storage, add Acquire::GzipIndexes
  "true"; to an apt.conf fragment.)

  This appears to be caused by apt downloading an lz4-compressed command
  list for command-not-found and cnf-update-db passing the
  /var/lib/apt/lists/*Commands-*.lz4 files to db/creator.py, which does
  not know about the compression and attempts to parse them as plain
  text.

  
  [Test plan]
  Enable the option, purge old lists, and update to populate lists directory with compressed files; this will run cnf-update-db for you.

  [Where problems could occur]
  Replacing the open() call with subprocess.Popen for apt-helper seems like a minimal change. If apt-helper has problems, we check the return code and fail, keeping the old db in place.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/command-not-found/+bug/1876034/+subscriptions