[Bug 1781176] Re: Blacklisted packages are included in the "upgradable origin", while they should not

Fri Apr 5 16:19:40 UTC 2019

Verified with 1.1ubuntu1.18.04.7~16.04.2 on Xenial:

...
2019-04-05 16:17:32,704 INFO Initial blacklisted packages: ebtables
2019-04-05 16:17:32,705 INFO Initial whitelisted packages: 
2019-04-05 16:17:32,705 INFO Starting unattended upgrades script
2019-04-05 16:17:32,705 INFO Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial
...
2019-04-05 16:17:33,813 DEBUG Checking: busybox-static ([<Origin component:'main' archive:'xenial-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>, <Origin component:'main' archive:'xenial-security' origin:'Ubuntu' label:'Ubuntu' site:'security.ubuntu.com' isTrusted:True>])
2019-04-05 16:17:33,960 DEBUG Checking: ebtables ([<Origin component:'main' archive:'xenial-updates' origin:'Ubuntu' label:'Ubuntu' site:'archive.ubuntu.com' isTrusted:True>])
2019-04-05 16:17:33,962 DEBUG adjusting candidate version: ebtables=2.0.10.4-3.4ubuntu1
2019-04-05 16:17:34,660 DEBUG pkgs that look like they should be upgraded: busybox-static
2019-04-05 16:17:34,712 DEBUG fetch.run() result: 0
...
2019-04-05 16:17:37,879 INFO All upgrades installed

>From root at x-uu-verify.lxd Fri Apr 05 16:17:38 2019
Return-path: <root at x-uu-verify.lxd>
Envelope-to: root at x-uu-verify.lxd
Delivery-date: Fri, 05 Apr 2019 16:17:38 +0000
Received: from root by x-uu-verify.lxd with local (Exim 4.86_2)
	(envelope-from <root at x-uu-verify.lxd>)
	id 1hCRX8-0004lu-Pd
	for root at x-uu-verify.lxd; Fri, 05 Apr 2019 16:17:38 +0000
Subject: [reboot required] unattended-upgrades result for x-uu-verify: True
To: root at x-uu-verify.lxd
Auto-Submitted: auto-generated
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
From: root <root at x-uu-verify.lxd>
Message-Id: <E1hCRX8-0004lu-Pd at x-uu-verify.lxd>
Date: Fri, 05 Apr 2019 16:17:38 +0000

Unattended upgrade returned: True

Warning: A reboot is required to complete this upgrade.

Packages that were upgraded:
 busybox-static=20

Package installation log:
Log started: 2019-04-05  16:17:35
Preparing to unpack .../busybox-static_1%3a1.22.0-15ubuntu1.4_amd64.deb ...
Unpacking busybox-static (1:1.22.0-15ubuntu1.4) over (1:1.22.0-15ubuntu1) .=
..
Processing triggers for man-db (2.7.5-1) ...
Setting up busybox-static (1:1.22.0-15ubuntu1.4) ...
Log ended: 2019-04-05  16:17:37

Unattended-upgrades log:
...

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1781176

Title:
  Blacklisted packages are included in the "upgradable origin", while
  they should not

Status in unattended-upgrades:
  Fix Released
Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Xenial:
  Fix Committed
Status in unattended-upgrades source package in Bionic:
  Fix Released

Bug description:
  [Impact]

   * Reports from u-u incorrectly list packages from non-upgradable
  origins as "Packages with upgradable origin but kept back"

   * Listing the packages incorrectly is a result of
  is_pkgname_in_blacklist() having a side effect and removing the side
  effect is part of fixing LP: #1396787 which fix is also being SRU-d.

   * The fix is removing the side effect of is_pkgname_in_blacklist()

  [Test Case]

   * There is a build-time test in test/test_blacklisted_wrong_origin.py
   * To reproduce the original problem set up a system where all security updates are installed but ebtables (from bionic-updates) is not updated:
  $ sudo unattended-upgrade  --verbose
  Initial blacklisted packages:
  Initial whitelisted packages:
  Starting unattended upgrades script
  Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
  No packages found that can be upgraded unattended and no pending auto-removals
  $ sudo apt upgrade
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  Calculating upgrade... Done
  The following packages will be upgraded:
    apt apt-utils ebtables initramfs-tools initramfs-tools-bin initramfs-tools-core libapt-inst2.0 libapt-pkg5.0
    liblxc-common liblxc1 libpython3-stdlib lxcfs lxd lxd-client netplan.io networkd-dispatcher nplan
    python-apt-common python3 python3-apt python3-minimal python3-update-manager snapd squashfs-tools
    unattended-upgrades update-manager-core update-notifier-common
  27 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
  Need to get 24.1 MB of archives.
  After this operation, 1454 kB of additional disk space will be used.
  Do you want to continue? [Y/n] n
  Abort.

  * blacklist ebtables, set up emails from u-u, then run u-u again:
  $ sudo echo 'Unattended-Upgrade::Package-Blacklist {"ebtables";};' > /etc/apt/apt.conf.d/51unattended-upgrades-blacklist-ebtables
  $ sudo echo 'Unattended-Upgrade::Mail "root";' > /etc/apt/apt.conf.d/51unattended-upgrades-mail
  $ sudo unattended-upgrade  --verbose
  Initial blacklisted packages: ebtables
  Initial whitelisted packages:
  Starting unattended upgrades script
  Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic
  Packages that will be upgraded:

  * Observe ebtables listed as being kept back and having upgradable origin with buggy u-u:
  $ sudo cat /var/mail/mail
  ...
  Packages with upgradable origin but kept back:
   ebtables=20
  ...

  * Upgrade u-u to a fixed version and run it, observing ebtables to be
  not listed as having upgradable origin

  [Regression Potential]

   * Regressions may make packages incorrectly missing from u-u's
  report, but the autopkgtests also cover that to some extent.

  [Other Info]

   * Original report: https://github.com/mvo5/unattended-
  upgrades/issues/116

To manage notifications about this bug go to:
https://bugs.launchpad.net/unattended-upgrades/+bug/1781176/+subscriptions