[Bug 1824804] Re: Unattended upgrades falls back to adjust all upgradable packages in attempt to install held packages

[Balint Reczey]

Xenial was not affected by this bug due to the fix of LP: #1821101 was
not released to Xenial.

The fix for LP: #1821101 however will be released including the fix for
the regression for this bug thus the verification of this regression fix
is needed.

Verified 1.1ubuntu1.18.04.7~16.04.3 on Xenial:

Original version:
root at x-lp-1824804:~# dpkg -l unattended-upgrades  | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                Version                    Architecture Description
+++-===================-==========================-============-===========================================
ii  unattended-upgrades 1.1ubuntu1.18.04.7~16.04.2 all          automatic installation of security upgrades
root at x-lp-1824804:~# unattended-upgrade --verbose --debug
Initial blacklisted packages: 
Initial whitelisted packages: 
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial
...
Checking: libpam-systemd ([<Origin component:'main' archive:'xenial-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'xenial-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
package libpam-systemd upgradable but fails to be marked for upgrade (E:Unable to correct problems, you have held broken packages.)
Checking: libsystemd0 ([<Origin component:'main' archive:'xenial-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'xenial-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
package libsystemd0 upgradable but fails to be marked for upgrade (E:Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.)
Checking: systemd ([<Origin component:'main' archive:'xenial-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'xenial-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
pkg systemd is on hold
sanity check failed
pkgs that look like they should be upgraded: 
Fetched 0 B in 0s (0 B/s)                                                                                         
fetch.run() result: 0
blacklist: []

Fixed version:
root at x-lp-1824804:~# dpkg -l unattended-upgrades  | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                Version                    Architecture Description
+++-===================-==========================-============-===========================================
ii  unattended-upgrades 1.1ubuntu1.18.04.7~16.04.3 all          automatic installation of security upgrades
root at x-lp-1824804:~# unattended-upgrade --verbose --debug
Initial blacklisted packages: 
Initial whitelisted packages: 
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial
...
Checking: libpam-systemd ([<Origin component:'main' archive:'xenial-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'xenial-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
package libpam-systemd upgradable but fails to be marked for upgrade (E:Unable to correct problems, you have held broken packages.)
falling back to adjusting libpam-systemd's dependencies recursively
adjusting candidate version: libaudit1=1:2.4.5-1ubuntu2
adjusting candidate version: init-system-helpers=1.29ubuntu1
adjusting candidate version: libpam-modules=1.1.8-3.2ubuntu2
...

** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1824804

Title:
  Unattended upgrades falls back to adjust all upgradable packages in
  attempt to install held packages

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Xenial:
  Fix Committed
Status in unattended-upgrades source package in Bionic:
  Fix Committed
Status in unattended-upgrades source package in Cosmic:
  Fix Committed
Status in unattended-upgrades source package in Disco:
  Fix Committed

Bug description:
  [Impact]

   * The fix for LP: #1821101 allowed installing packages for which
  APT's resolver could not find solution for without adjusting other
  packages, but also tries to adjust many packages to install packages
  which are held back.

  [Test Case]

   * Set up a system where systemd (or a package that can't be upgraded
  without also upgrading other packages) need to be upgraded to the
  version in -security.

   * Mark systemd on hold
     apt-mark hold systemd

   * Observe u-u falling back to adjusting all upgradable packages in
  unfixed versions and adjusting only a few in fixed versions:

  # unattended-upgrade --verbose --debug
  ...

  adjusting candidate version: libnss-systemd=237-3ubuntu10.19
  falling back to marking libnss-systemd, then adjusting changes
  package libnss-systemd upgradable but fails to be marked for upgrade (E:Unable to correct problems, you have held broken packages.)
  falling back to adjusting all packages
  adjusting candidate version: 2ping=4.1-1
  ...

    Fixed:
  ...
  adjusting candidate version: libnss-systemd=237-3ubuntu10.19
  falling back to adjusting libnss-systemd's dependencies recursively
  adjusting candidate version: libnss-systemd=237-3ubuntu10.19
  adjusting candidate version: libkmod2=24-1ubuntu3
  adjusting candidate version: mount=2.31.1-0.4ubuntu3
  adjusting candidate version: libmount1=2.31.1-0.4ubuntu3
  ...

  [Regression Potential]

   * The less extensive fallback may not find solutions for installing
  packages with special relationships that the full fallback could have
  found, but I don't know about such special cases in the archive and
  the autopkgtest cover a fair set of popular packages. Keeping back
  those packages seems to be better than spending a lot of CPU time on
  finding a few solutions - which still was not enough to find _all_
  solutions.

  [Other Info]

  Originally reported at: https://bugs.launchpad.net/ubuntu/+source
  /unattended-upgrades/+bug/1396787/comments/21

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1824804/+subscriptions