[Bug 1815494] Re: May remove autoremovable non-kernel packages matching pattern from APT::VersionedKernelPackages

Tue Mar 12 17:49:49 UTC 2019

Verified with unattended-upgrades/1.1ubuntu1.18.04.7~16.04.2 on Xenial:

https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac
/autopkgtest-xenial/xenial/amd64/u/unattended-
upgrades/20190228_150449_11313@/log.gz :

Running ./test_remove_unused.py with python3
Initial blacklisted packages: 
Initial whitelisted packages: 
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=lucid-security
APT::VersionedKernelPackages is not set
Checking: test-package ([<Origin component:'main' archive:'lucid-security' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
pkgs that look like they should be upgraded: test-package

Fetched 0 B in 0s (0 B/s)                                                       
fetch.run() result: 0
<apt_pkg.AcquireItem object:Status: 2 Complete: 1 Local: 1 IsTrusted: 1 FileSize: 624 DestFile:'/tmp/autopkgtest.nX7fyI/build.uPt/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb' DescURI: 'http://archive.ubuntu.com/ubuntu/test-package_2.0_all.deb' ID:0 ErrorText: ''>
check_conffile_prompt(/tmp/autopkgtest.nX7fyI/build.uPt/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb)
found pkg: test-package
blacklist: []
whitelist: []
Packages that will be upgraded: test-package
Writing dpkg log to ./root.unused-deps/var/log/unattended-upgrades/unattended-upgrades-dpkg.log
applying set ['test-package']
left to upgrade set()
All upgrades installed
marking linux-image-4.05.0-1021-kvm for removal
marking test-package-dependency for removal
marking old-unused-dependency for removal
marking any-old-unused-modules for removal
Packages that were successfully auto-removed: any-old-unused-modules linux-image-4.05.0-1021-kvm old-unused-dependency test-package-dependency
Packages that are kept back: 
InstCount=0 DelCount=0 BrokenCount=0
Extracting content from ./root.unused-deps/var/log/unattended-upgrades/unattended-upgrades-dpkg.log since 2019-02-28 14:26:03
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1129: ResourceWarning: unclosed file <_io.TextIOWrapper name=13 mode='w' encoding='UTF-8'>
  options.verbose or options.debug)
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1129: ResourceWarning: unclosed file <_io.TextIOWrapper name=12 mode='r' encoding='UTF-8'>
  options.verbose or options.debug)
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1474: ResourceWarning: unclosed file <_io.TextIOWrapper name=13 mode='w' encoding='UTF-8'>
  res, error = cache_commit(cache, logfile_dpkg, verbose)
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1474: ResourceWarning: unclosed file <_io.TextIOWrapper name=12 mode='r' encoding='UTF-8'>
  res, error = cache_commit(cache, logfile_dpkg, verbose)
.Initial blacklisted packages: 
Initial whitelisted packages: 
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=lucid-security
Using (^linux-image-[0-9]+\.[0-9\.]+-.*|^.*-modules-[0-9]+\.[0-9\.]+-.*|^linux-headers-[0-9]+\.[0-9\.]+-.*) regexp to find kernel packages
Using (^linux-image-4\.4\.0\-142\-generic$|^.*-modules-4\.4\.0\-142\-generic$|^linux-headers-4\.4\.0\-142\-generic$) regexp to find running kernel packages
Checking: test-package ([<Origin component:'main' archive:'lucid-security' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
pkgs that look like they should be upgraded: test-package

Fetched 0 B in 0s (0 B/s)                                                       
fetch.run() result: 0
<apt_pkg.AcquireItem object:Status: 2 Complete: 1 Local: 1 IsTrusted: 1 FileSize: 624 DestFile:'/tmp/autopkgtest.nX7fyI/build.uPt/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb' DescURI: 'http://archive.ubuntu.com/ubuntu/test-package_2.0_all.deb' ID:0 ErrorText: ''>
check_conffile_prompt(/tmp/autopkgtest.nX7fyI/build.uPt/src/test/root.unused-deps/var/cache/apt/archives/test-package_2.0_all.deb)
found pkg: test-package
blacklist: []
whitelist: []
Removing unused kernel packages: linux-image-4.05.0-1021-kvm
marking linux-image-4.05.0-1021-kvm for removal
Keeping auto-removable linux-image-4.05.0-1021-kvm package(s) because it would also remove the following packages which should be kept in this step: test-package
Packages that were successfully auto-removed: 
Packages that are kept back: linux-image-4.05.0-1021-kvm
Packages that will be upgraded: test-package
Writing dpkg log to ./root.unused-deps/var/log/unattended-upgrades/unattended-upgrades-dpkg.log
applying set ['test-package']
left to upgrade set()
All upgrades installed
marking test-package-dependency for removal
Packages that were successfully auto-removed: test-package-dependency
Packages that are kept back: 
InstCount=0 DelCount=0 BrokenCount=0
Extracting content from ./root.unused-deps/var/log/unattended-upgrades/unattended-upgrades-dpkg.log since 2019-02-28 14:26:04
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1129: ResourceWarning: unclosed file <_io.TextIOWrapper name=17 mode='w' encoding='UTF-8'>
  options.verbose or options.debug)
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1129: ResourceWarning: unclosed file <_io.TextIOWrapper name=16 mode='r' encoding='UTF-8'>
  options.verbose or options.debug)
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1474: ResourceWarning: unclosed file <_io.TextIOWrapper name=17 mode='w' encoding='UTF-8'>
  res, error = cache_commit(cache, logfile_dpkg, verbose)
/tmp/autopkgtest.nX7fyI/build.uPt/src/test/unattended_upgrade.py:1474: ResourceWarning: unclosed file <_io.TextIOWrapper name=16 mode='r' encoding='UTF-8'>
  res, error = cache_commit(cache, logfile_dpkg, verbose)
.
----------------------------------------------------------------------
Ran 2 tests in 1.185s

OK

** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1815494

Title:
  May remove autoremovable non-kernel packages matching pattern from
  APT::VersionedKernelPackages

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in update-manager package in Ubuntu:
  New
Status in unattended-upgrades source package in Xenial:
  Fix Committed
Status in unattended-upgrades source package in Bionic:
  Fix Released
Status in unattended-upgrades source package in Cosmic:
  Fix Released

Bug description:
  [Impact]

  * Non-kernel related autoremovable packages may be removed by unattended-upgrades due to their name matching generic patterns like '.*-modules' which should be applied to versioned kernel packages only.
  * The fix corrects the way those patterns are handled.

  [Test Case]

  * test/test_remove_unused.py checks for correct pattern usage now.

  [Regression Potential]

  * Unattended-upgrades may stop removing autoremovable kernels, but the
  tests also contain cases covering this and u-u's kernel autoremoval
  still works.

  [Original Bug Text]

  Unattended-upgrades and Update Manager use the patterns from the
  APT::VersionedKernelPackages list directly for finding kernel packages
  to remove while APT uses patterns by attaching version and flavor to
  them.

  As a result in APT's script ".*-modules" becomes
  "^.*-modules-4\.15\.0-45-generic$":

  ...
          for package in $(apt-config dump --no-empty --format '%v%n' 'APT::VersionedKernelPackages'); do
                  for kernel in $kernels; do
                          echo "   \"^${package}-${kernel}$\";"
    done
  ...

  In unattended-upgrades and update-manager ".*-modules" is used
  directly for matching and may false  identify autoremovable packages
  as kernel-related ones and remove them (of just offer the removal in
  case of update-manager):

  ...
  Removing unused kernel packages: extra-cmake-modules
  marking extra-cmake-modules for removal
  (Reading database ... 31149 files and directories currently installed.)
  Removing extra-cmake-modules (5.44.0-0ubuntu1) ...
  Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
  Packages that were successfully auto-removed: extra-cmake-modules
  ...

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1815494/+subscriptions