[Bug 1823872] Re: Fixing fsfreeze-hook can break unattended upgrades

Balint Reczey balint.reczey at canonical.com
Fri May 3 14:44:46 UTC 2019 

Verified 1.1ubuntu1.18.04.7~16.04.3 on Xenial:

root at x-lp-1823872:~# cat acpid.equivs 
Source: acpid
Section: misc
Priority: optional
Standards-Version: 3.9.2

Package: acpid
File: /etc/acpi/events/powerbtn/powerbtn 
 # /etc/acpi/events/powerbtn
 # This is called when the user presses the power button and calls
 # /etc/acpi/powerbtn.sh for further processing.
 .
 # Optionally you can specify the placeholder %e. It will pass
 # through the whole kernel event message to the program you've
 # specified.
 .
 # We need to react on "button power.*" and "button/power.*" because
 # of kernel changes.
 .
 event=button[ /]power
 action=/etc/acpi/powerbtn.sh
root at x-lp-1823872:~# equivs-build acpid.equivs 
...
root at x-lp-1823872:~# dpkg -i acpid_1.0_all.deb 
dpkg: warning: downgrading acpid from 1:2.0.26-1ubuntu2 to 1.0
(Reading database ... 32148 files and directories currently installed.)
Preparing to unpack acpid_1.0_all.deb ...
Unpacking acpid (1.0) over (1:2.0.26-1ubuntu2) ...
Setting up acpid (1.0) ...
Processing triggers for man-db (2.7.5-1) ...
root at x-lp-1823872:~# dpkg -l unattended-upgrades | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                Version                    Architecture Description
+++-===================-==========================-============-===========================================
ii  unattended-upgrades 1.1ubuntu1.18.04.7~16.04.2 all          automatic installation of security upgrades
root at x-lp-1823872:~# unattended-upgrade --dry-run 
Traceback (most recent call last):
  File "/usr/bin/unattended-upgrade", line 1998, in <module>
    sys.exit(main(options))
  File "/usr/bin/unattended-upgrade", line 1714, in main
    if conffile_prompt(item.destfile):
  File "/usr/bin/unattended-upgrade", line 929, in conffile_prompt
    with open(prefix + conf_file, 'rb') as fp:
IsADirectoryError: [Errno 21] Is a directory: '/etc/acpi/events/powerbtn'
root at x-lp-1823872:~# apt install -y -qq unattended-upgrades 
The following package was automatically installed and is no longer required:
  libfreetype6
Use 'apt autoremove' to remove it.
Suggested packages:
  bsd-mailx default-mta | mail-transport-agent needrestart
The following packages will be upgraded:
  unattended-upgrades
1 upgraded, 0 newly installed, 0 to remove and 9 not upgraded.
Need to get 41.4 kB of archives.
After this operation, 8,192 B of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 32136 files and directories currently installed.)
Preparing to unpack .../unattended-upgrades_1.1ubuntu1.18.04.7~16.04.3_all.deb ...
Unpacking unattended-upgrades (1.1ubuntu1.18.04.7~16.04.3) over (1.1ubuntu1.18.04.7~16.04.2) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for ureadahead (0.100.0-19.1) ...
Processing triggers for systemd (229-4ubuntu21.21) ...
Setting up unattended-upgrades (1.1ubuntu1.18.04.7~16.04.3) ...
root at x-lp-1823872:~# unattended-upgrade --dry-run --verbose
Initial blacklisted packages: 
Initial whitelisted packages: 
Starting unattended upgrades script
Allowed origins are: o=Ubuntu,a=xenial, o=Ubuntu,a=xenial-security, o=UbuntuESM,a=xenial
Option --dry-run given, *not* performing real actions
Packages that will be upgraded: acpid
Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
/usr/bin/dpkg --status-fd 9 --unpack --auto-deconfigure /var/cache/apt/archives/acpid_1%3a2.0.26-1ubuntu2_amd64.deb 
/usr/bin/dpkg --status-fd 11 --configure acpid:amd64 
/usr/bin/dpkg --status-fd 13 --configure --pending 
All upgrades installed
#

** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1823872

Title:
  Fixing fsfreeze-hook can break unattended upgrades

Status in qemu package in Ubuntu:
  Invalid
Status in unattended-upgrades package in Ubuntu:
  Fix Released
Status in unattended-upgrades source package in Trusty:
  Confirmed
Status in unattended-upgrades source package in Xenial:
  Fix Committed
Status in unattended-upgrades source package in Bionic:
  Fix Committed
Status in unattended-upgrades source package in Cosmic:
  Fix Committed
Status in unattended-upgrades source package in Disco:
  Fix Committed

Bug description:
  [Impact]

   * If an update has a new conffile at a path that in a former version was
     a directory like
      old: /a/b/c
      new: a/b
     Here b is the new file name and was a directory in the old version.
     Then unattended upgrades breaks on installing such a package.

   * a recent qemu update has such a case and due to that triggered the
     issue in >=Bionic

   * The fix is to harden unattended upgrades to be able to handle the case
     without aborting.

  [Test Case]

  Get a qemu guest e.g. of Bionic before the update to 1:2.11+dfsg-1ubuntu7.12
  That can be done with:
    $ time uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=bionic
    $ uvt-kvm create --password ubuntu bionic-testuu arch=amd64 release=bionic label=daily

  Log in and apt update & upgrade all packages, then Install the release level qemu in there.
    $ uvt-kvm ssh bionic-testuu
    $ sudo apt update
    $ sudo apt dist-upgrade
    $ sudo apt install unattended-upgrades
    $ sudo apt install qemu-guest-agent=1:2.11+dfsg-1ubuntu7

  All before was preparation, now force the unattended upgrade to trigger the bug.
    $ sudo unattended-upgrade -d

  With the bug you'll find some error like:
  found pkg: qemu-guest-agent
  conffile line: /etc/init.d/qemu-guest-agent f61a64ac1e48993023018fd1cff85191
  current md5: f61a64ac1e48993023018fd1cff85191
  conffile line: /etc/qemu/fsfreeze-hook/fsfreeze-hook 15f6ff42cbc5550a07ee21c2a471d905
  /etc/qemu/fsfreeze-hook/fsfreeze-hook not in package conffiles /etc/init.d/qemu-guest-agent
  /etc/qemu/fsfreeze-hook
  found conffile /etc/qemu/fsfreeze-hook in new pkg but on dpkg status
  Traceback (most recent call last):
    File "/usr/bin/unattended-upgrade", line 2057, in <module>
      sys.exit(main(options))
    File "/usr/bin/unattended-upgrade", line 1773, in main
      if conffile_prompt(item.destfile):
    File "/usr/bin/unattended-upgrade", line 988, in conffile_prompt
      with open(prefix + conf_file, 'rb') as fp:
  IsADirectoryError: [Errno 21] Is a directory: '/etc/qemu/fsfreeze-hook'

  [Regression Potential]

   * The fix is trying to detect moved conffiles by looking for /etc/foo/foo when the new package ships /etc/foo and /etc/foo is not a known conffile and also checking the renames in the opposite direction.
  The potential regression is breaking the logic for detecting changed conffiles and either holding back a package for no reason or trying to install a package with a modified conffile on the system and aborting the upgrade in the middle due to the appearing conffile prompt. To avoid such regressions the test_conffile.py tests are extended to cover rename scenarios.

  [Other Info]

   * n/a

  ---

  As reported on https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1820291/comments/20
  We fixed an issue and we added workrounds since basic mv_conffile coudn't handle it and did all sort of upgrade tests.

  That all worked fine and moved the conffile.

  It was now reported that due to some pre-checks that unattended
  upgrades might do this might do some checks on its own.

  Next step:
  - check unattended upgrades through this change
  - check if it only affects cases were the former config was modified (minority) or the default file layout (majority)

  --- original report copied ---

  just wanted to add: This bug also crashes unattended-upgrade and thus
  prevents security updates on 18.04:

  root at mailin1:~# unattended-upgrade
  Traceback (most recent call last):
    File "/usr/bin/unattended-upgrade", line 1998, in <module>
      sys.exit(main(options))
    File "/usr/bin/unattended-upgrade", line 1714, in main
      if conffile_prompt(item.destfile):
    File "/usr/bin/unattended-upgrade", line 929, in conffile_prompt
      with open(prefix + conf_file, 'rb') as fp:
  IsADirectoryError: [Errno 21] Is a directory: '/etc/qemu/fsfreeze-hook'

  Basically, unattended-upgrade wants to compare old/new conffiles and
  doesn't like it when the old conffile turns out to be a directory...

  This prevents security updates to be installed. The unattended-upgrade
  logs do not contain the trace, only the messages:

  2019-04-05 13:24:24,851 INFO Initial blacklisted packages:
  2019-04-05 13:24:24,853 INFO Initial whitelisted packages:
  2019-04-05 13:24:24,853 INFO Starting unattended upgrades script
  2019-04-05 13:24:24,854 INFO Allowed origins are: o=Ubuntu,a=bionic, o=Ubuntu,a=bionic-security, o=UbuntuESM,a=bionic'

  And that's it.

  You can't really get out of this without manually running 'apt-get
  install qemu-guest-agent' because at this point, unattended-upgrade
  can't update itself anymore.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1823872/+subscriptions

More information about the foundations-bugs mailing list