[Bug 1396787] Re: checking trust of archives eats a lot of cpu

[Balint Reczey]

Verified with 1.1ubuntu1.18.04.7~16.04.2, measured times are similar to
.1, verified previously.

https://objectstorage.prodstack4-5.canonical.com
/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-xenial/xenial/amd64/u/unattended-upgrades/20190228_150449_11313@/log.gz

...
Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial
Packages that will be upgraded: 
4.62user 0.16system 0:04.83elapsed 99%CPU (0avgtext+0avgdata 78264maxresident)k
0inputs+124024outputs (0major+38540minor)pagefaults 0swaps
new packages marked as manually installed (should be none)
...


** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1396787

Title:
  checking trust of archives eats a lot of cpu

Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Xenial:
  Fix Committed
Status in unattended-upgrades source package in Bionic:
  Fix Released

Bug description:
  [Impact]

   * Unattended-upgrades consumes tens of seconds or even minutes of CPU
  time to verify the origin of the packages

   * Using excessive amount of CPU is unpleasant for desktop/laptop
  users and also wastes computation time on servers/cloud instances.

   * Unattended-upgrades' algorithm for checking and adjusting package
  origins is redesigned to visit and adjust less packages.

  [Test Case]

   * The added upgrade-all-security autopkgtest measure the time u-u needs for upgrading security updates on the tested release starting with no security updates applied to the point where all security updates are applied but all packages are left upgradable from <release>-updates. The test also measures the time needed for --dry-run to find no updates to be installed unattended.
  * Please run autopkgtests and look for the to time results:
  ...
  All upgrades installed
  44.41user 3.06system 0:48.35elapsed 98%CPU (0avgtext+0avgdata 164872maxresident)k
  208inputs+192376outputs (0major+642657minor)pagefaults 0swaps
  ...
  No packages found that can be upgraded unattended and no pending auto-removals
  2.83user 0.11system 0:02.98elapsed 98%CPU (0avgtext+0avgdata 79308maxresident)k

  
  [Regression Potential] 

   * Due to algorithm redesign there is a risk that packages from
  allowed origins are not upgraded. There were unit tests for testing
  the selection of the right packages to upgrade already, but a new
  autopkgtest is also introduce to verify u-u's behavior on current
  real-life security-updates.

  
  [Original bug text]

  (System: Ubuntu 14.04, up to date packages)

  I noticed that unattended-upgrades spends a significant amount of time
  in phases where it runs at 100% cpu. On a slower machine (core 2 t7200
  2GHz) this goes on for minutes rather than seconds. This interferes
  with using the machine for other tasks.

  Using the --debug option to unattended-upgrades shows that the program
  outputs a lot of lines like the following during these 100% cpu
  phases:

  matching 'a'='trusty-updates' against '<Origin component:'universe'
  archive:'trusty-updates' origin:'Ubuntu' label:'Ubuntu'
  site:'de.archive.ubuntu.com' isTrusted:True>

  From this output I guess the operation executed is not so complicated
  that it should require so much cpu power. ??

  ProblemType: Bug
  DistroRelease: Ubuntu 14.04
  Package: unattended-upgrades 0.82.1ubuntu2
  ProcVersionSignature: Ubuntu 3.13.0-40.69-generic 3.13.11.10
  Uname: Linux 3.13.0-40-generic x86_64
  ApportVersion: 2.14.1-0ubuntu3.5
  Architecture: amd64
  Date: Wed Nov 26 21:53:57 2014
  InstallationDate: Installed on 2014-08-28 (90 days ago)
  InstallationMedia: Kubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140416.1)
  PackageArchitecture: all
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=de_DE.UTF-8
   SHELL=/bin/bash
  SourcePackage: unattended-upgrades
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1396787/+subscriptions