[Bug 1815494] Re: May remove autoremovable non-kernel packages matching pattern from APT::VersionedKernelPackages

Fri Feb 22 10:17:02 UTC 2019

Verified on unattended-upgrades/1.5ubuntu3.18.10.2 for Cosmic:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-cosmic/cosmic/amd64/u/unattended-upgrades/20190221_173137_1ecf7@/log.gz :
...
Running ./test_remove_unused.py with python3
Initial blacklisted packages: 
Initial whitelisted packages: 
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=lucid-security
APT::VersionedKernelPackages is not set
Checking: test-package ([<Origin component:'main' archive:'lucid-security' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
pkgs that look like they should be upgraded: test-package

Fetched 0 B in 0s (0 B/s)                                                       
fetch.run() result: 0
<apt_pkg.AcquireItem object:Status: 2 Complete: 1 Local: 1 IsTrusted: 1 FileSize: 624 DestFile:'/tmp/autopkgtest.5oXOHy/build.r1N/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb' DescURI: 'http://archive.ubuntu.com/ubuntu/test-package_2.0_all.deb' ID:0 ErrorText: ''>
check_conffile_prompt(/tmp/autopkgtest.5oXOHy/build.r1N/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb)
found pkg: test-package
blacklist: []
whitelist: []
Packages that will be upgraded: test-package
Writing dpkg log to ./root.unused-deps/var/log/unattended-upgrades/unattended-upgrades-dpkg.log
applying set ['test-package']
left to upgrade set()
All upgrades installed
marking linux-image-4.05.0-1021-kvm for removal
marking old-unused-dependency for removal
marking any-old-unused-modules for removal
marking test-package-dependency for removal
Packages that were successfully auto-removed: any-old-unused-modules linux-image-4.05.0-1021-kvm old-unused-dependency test-package-dependency
Packages that are kept back: 
InstCount=0 DelCount=0 BrokenCount=0
...
Initial blacklisted packages: 
Initial whitelisted packages: 
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=lucid-security
Using (^linux-image-[0-9]+\.[0-9\.]+-.*|^.*-modules-[0-9]+\.[0-9\.]+-.*|^linux-headers-[0-9]+\.[0-9\.]+-.*) regexp to find kernel packages
Using (^linux-image-4\.18\.0\-15\-generic$|^.*-modules-4\.18\.0\-15\-generic$|^linux-headers-4\.18\.0\-15\-generic$) regexp to find running kernel packages
Checking: test-package ([<Origin component:'main' archive:'lucid-security' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
pkgs that look like they should be upgraded: test-package

Fetched 0 B in 0s (0 B/s)                                                       
fetch.run() result: 0
<apt_pkg.AcquireItem object:Status: 2 Complete: 1 Local: 1 IsTrusted: 1 FileSize: 624 DestFile:'/tmp/autopkgtest.5oXOHy/build.r1N/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb' DescURI: 'http://archive.ubuntu.com/ubuntu/test-package_2.0_all.deb' ID:0 ErrorText: ''>
check_conffile_prompt(/tmp/autopkgtest.5oXOHy/build.r1N/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb)
found pkg: test-package
blacklist: []
whitelist: []
Removing unused kernel packages: linux-image-4.05.0-1021-kvm
marking linux-image-4.05.0-1021-kvm for removal
Keeping auto-removable linux-image-4.05.0-1021-kvm package(s) because it would also remove the following packages which should be kept in this step: test-package
Packages that were successfully auto-removed: 
Packages that are kept back: linux-image-4.05.0-1021-kvm
Packages that will be upgraded: test-package
Writing dpkg log to ./root.unused-deps/var/log/unattended-upgrades/unattended-upgrades-dpkg.log
applying set ['test-package']
left to upgrade set()
All upgrades installed
marking test-package-dependency for removal
Packages that were successfully auto-removed: test-package-dependency
Packages that are kept back: 
InstCount=0 DelCount=0 BrokenCount=0
Extracting content from ./root.unused-deps/var/log/unattended-upgrades/unattended-upgrades-dpkg.log since 2019-02-21 17:12:00
/tmp/autopkgtest.5oXOHy/build.r1N/src/test/unattended_upgrade.py:1163: ResourceWarning: unclosed file <_io.TextIOWrapper name=12 mode='w' encoding='UTF-8'>
  options.verbose or options.debug)
/tmp/autopkgtest.5oXOHy/build.r1N/src/test/unattended_upgrade.py:1163: ResourceWarning: unclosed file <_io.TextIOWrapper name=11 mode='r' encoding='UTF-8'>
  options.verbose or options.debug)
/tmp/autopkgtest.5oXOHy/build.r1N/src/test/unattended_upgrade.py:1513: ResourceWarning: unclosed file <_io.TextIOWrapper name=12 mode='w' encoding='UTF-8'>
  res, error = cache_commit(cache, logfile_dpkg, verbose)
/tmp/autopkgtest.5oXOHy/build.r1N/src/test/unattended_upgrade.py:1513: ResourceWarning: unclosed file <_io.TextIOWrapper name=11 mode='r' encoding='UTF-8'>
  res, error = cache_commit(cache, logfile_dpkg, verbose)
.
----------------------------------------------------------------------
Ran 2 tests in 2.761s
...

** Tags removed: verification-needed-cosmic
** Tags added: verification-done-cosmic

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1815494

Title:
  May remove autoremovable non-kernel packages matching pattern from
  APT::VersionedKernelPackages

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in update-manager package in Ubuntu:
  New
Status in unattended-upgrades source package in Bionic:
  Fix Committed
Status in unattended-upgrades source package in Cosmic:
  Fix Committed

Bug description:
  [Impact]

  * Non-kernel related autoremovable packages may be removed by unattended-upgrades due to their name matching generic patterns like '.*-modules' which should be applied to versioned kernel packages only.
  * The fix corrects the way those patterns are handled.

  [Test Case]

  * test/test_remove_unused.py checks for correct pattern usage now.

  [Regression Potential]

  * Unattended-upgrades may stop removing autoremovable kernels, but the
  tests also contain cases covering this and u-u's kernel autoremoval
  still works.

  [Original Bug Text]

  Unattended-upgrades and Update Manager use the patterns from the
  APT::VersionedKernelPackages list directly for finding kernel packages
  to remove while APT uses patterns by attaching version and flavor to
  them.

  As a result in APT's script ".*-modules" becomes
  "^.*-modules-4\.15\.0-45-generic$":

  ...
          for package in $(apt-config dump --no-empty --format '%v%n' 'APT::VersionedKernelPackages'); do
                  for kernel in $kernels; do
                          echo "   \"^${package}-${kernel}$\";"
    done
  ...

  In unattended-upgrades and update-manager ".*-modules" is used
  directly for matching and may false  identify autoremovable packages
  as kernel-related ones and remove them (of just offer the removal in
  case of update-manager):

  ...
  Removing unused kernel packages: extra-cmake-modules
  marking extra-cmake-modules for removal
  (Reading database ... 31149 files and directories currently installed.)
  Removing extra-cmake-modules (5.44.0-0ubuntu1) ...
  Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
  Packages that were successfully auto-removed: extra-cmake-modules
  ...

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1815494/+subscriptions