[Bug 1803137] Re: Unattended-upgrades may keep running after unmounting local filesystems in InstallOnShutdown mode
Balint Reczey
balint.reczey at canonical.com
Tue Nov 27 22:12:47 UTC 2018
Verified with unattended-upgrades 1.5ubuntu3.18.10.0:
oot at cc-uu-onshutdown:~# service unattended-upgrades status
● unattended-upgrades.service - Unattended Upgrades Shutdown
Loaded: loaded (/lib/systemd/system/unattended-upgrades.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2018-11-27 22:05:20 UTC; 6s ago
Docs: man:unattended-upgrade(8)
Main PID: 2198 (unattended-upgr)
Tasks: 2 (limit: 4915)
Memory: 8.1M
CGroup: /system.slice/unattended-upgrades.service
└─2198 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal
Nov 27 22:05:20 cc-uu-onshutdown systemd[1]: Started Unattended Upgrades Shutdown.
root at cc-uu-onshutdown:~# vi /etc/apt/sources.list
root at cc-uu-onshutdown:~# echo 'Unattended-Upgrade::InstallOnShutdown "true";' > /etc/apt/apt.conf.d/51unattended-upgrades-on-shutdown
root at cc-uu-onshutdown:~# dbus-send --system --print-reply --dest=org.freedesktop.login1 /org/freedesktop/login1 "org.freedesktop.login1.Manager.Reboot" boolean:false
method return time=1543356364.788282 sender=:1.4 -> destination=:1.14 serial=32 reply_serial=2
root at cc-uu-onshutdown:~# logout
rbalint at yogi:~/tmp$ lxc shell cc-uu-onshutdown
mesg: ttyname failed: No such device
root at cc-uu-onshutdown:~# uptime
22:06:50 up 0 min, 0 users, load average: 0.39, 0.44, 0.36
root at cc-uu-onshutdown:~# journalctl -l | grep -B5 -A5 Unatt
Nov 27 22:05:20 cc-uu-onshutdown systemd[1]: snapd.seeded.service: Failed to reset devices.list: Operation not permitted
Nov 27 22:05:20 cc-uu-onshutdown systemd[1]: systemd-user-sessions.service: Failed to reset devices.list: Operation not permitted
Nov 27 22:05:20 cc-uu-onshutdown systemd[1]: systemd-sysctl.service: Failed to reset devices.list: Operation not permitted
Nov 27 22:05:20 cc-uu-onshutdown systemd[1]: snap.mount: Failed to reset devices.list: Operation not permitted
Nov 27 22:05:20 cc-uu-onshutdown systemd[1]: unattended-upgrades.service: Failed to reset devices.list: Operation not permitted
Nov 27 22:05:20 cc-uu-onshutdown systemd[1]: Started Unattended Upgrades Shutdown.
Nov 27 22:06:06 cc-uu-onshutdown systemd-logind[190]: System is rebooting.
Nov 27 22:06:06 cc-uu-onshutdown systemd[1]: Stopping Availability of block devices...
Nov 27 22:06:06 cc-uu-onshutdown systemd[1]: Removed slice system-getty.slice.
Nov 27 22:06:06 cc-uu-onshutdown systemd[1]: Stopping Session c1 of user root.
Nov 27 22:06:06 cc-uu-onshutdown su[1160]: pam_unix(su:session): session closed for user root
--
...
root at cc-uu-onshutdown:~# cat /var/log/unattended-upgrades/unattended-upgrades
unattended-upgrades-dpkg.log unattended-upgrades-shutdown.log unattended-upgrades.log
root at cc-uu-onshutdown:~# cat /var/log/unattended-upgrades/unattended-upgrades-shutdown.log
2018-11-27 22:06:04,792 WARNING - Running unattended-upgrades in shutdown mode
2018-11-27 22:06:04,804 WARNING - Unattended-upgrade in progress during shutdown, please don't turn off the computer
2018-11-27 22:06:06,457 WARNING - Unattended-upgrade in progress during shutdown, please don't turn off the computer
2018-11-27 22:06:06,473 INFO - All upgrades installed
root at cc-uu-onshutdown:~# cat /var/log/unattended-upgrades/unattended-upgrades.log
2018-11-27 22:06:05,014 WARNING System is on battery power, stopping
root at cc-uu-onshutdown:~# vi /etc/apt/apt.conf.d/50
50command-not-found 50unattended-upgrades
root at cc-uu-onshutdown:~# vi /etc/apt/apt.conf.d/50unattended-upgrades
root at cc-uu-onshutdown:~# dbus-send --system --print-reply --dest=org.freedesktop.login1 /org/freedesktop/login1 "org.freedesktop.login1.Manager.Reboot" boolean:false
method return time=1543356568.325178 sender=:1.4 -> destination=:1.13 serial=32 reply_serial=2
root at cc-uu-onshutdown:~# dpkg -l unattended-upgrades
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-======================-================-================-==================================================
ii unattended-upgrades 1.5ubuntu3.18.10 all automatic installation of security upgrades
root at cc-uu-onshutdown:~# dpkg -l unattended-upgrades | cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===================-==================-============-===========================================
ii unattended-upgrades 1.5ubuntu3.18.10.0 all automatic installation of security upgrades
root at cc-uu-onshutdown:~#
logout
Session terminated, terminating shell... ...terminated.
rbalint at yogi:~/tmp$ lxc shell cc-uu-onshutdown
mesg: ttyname failed: No such device
root at cc-uu-onshutdown:~# cat /var/log/unattended-upgrades/unattended-upgrades.log
2018-11-27 22:06:05,014 WARNING System is on battery power, stopping
2018-11-27 22:09:28,550 INFO Initial blacklisted packages:
2018-11-27 22:09:28,550 INFO Initial whitelisted packages:
2018-11-27 22:09:28,550 INFO Starting unattended upgrades script
2018-11-27 22:09:28,550 INFO Allowed origins are: o=Ubuntu,a=cosmic, o=Ubuntu,a=cosmic-security, o=UbuntuESM,a=cosmic
2018-11-27 22:09:32,528 INFO Packages that will be upgraded: git git-man libnss-systemd libpam-systemd libsystemd0 libudev1 systemd systemd-sysv udev
2018-11-27 22:09:32,528 INFO Writing dpkg log to /var/log/unattended-upgrades/unattended-upgrades-dpkg.log
2018-11-27 22:09:55,195 INFO All upgrades installed
root at cc-uu-onshutdown:~#
** Tags removed: verification-needed-cosmic
** Tags added: verification-done-cosmic
** Tags removed: verification-needed
** Tags added: verification-done
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to unattended-upgrades in Ubuntu.
https://bugs.launchpad.net/bugs/1803137
Title:
Unattended-upgrades may keep running after unmounting local
filesystems in InstallOnShutdown mode
Status in unattended-upgrades package in Ubuntu:
Fix Released
Status in unattended-upgrades source package in Cosmic:
Fix Committed
Bug description:
[Impact]
* Unattended-upgrades may keep running during shutdown even beyond unmounting local filesystems potentially leaving a broken installation behind.
* The fix reverts the original fix of LP: #1778219 and applies a different one starting unattended-upgrades-shutdown _before_ the shutdown transaction starts/
[Test Case]
* Run unattended-upgrades in InstallOnShutdown mode and observe it
being finished installing a few packages _before_ the shutdown
transaction starts:
$ lxc launch ubuntu:18.10 cc-uu-onshutdown
Creating cc-uu-onshutdown
Starting cc-uu-onshutdown
$ lxc shell cc-uu-onshutdown
mesg: ttyname failed: No such device
root at cc-uu-onshutdown:~# apt update -qq
33 packages can be upgraded. Run 'apt list --upgradable' to see them.
root at cc-uu-onshutdown:~# echo 'Unattended-Upgrade::InstallOnShutdown "true";' > /etc/apt/apt.conf.d/51unattended-upgrades-on-shutdown
root at cc-uu-onshutdown:~# dbus-send --system --print-reply --dest=org.freedesktop.login1 /org/freedesktop/login1 "org.freedesktop.login1.Manager.Reboot" boolean:false
method return time=1542112922.046290 sender=:1.4 -> destination=:1.13 serial=27 reply_serial=2
Session terminated, terminating shell...$
$ lxc shell cc-uu-onshutdown
mesg: ttyname failed: No such device
root at cc-uu-onshutdown:~# journalctl -l
...
Nov 13 12:50:10 cc-uu-shutdown systemd[1]: Started Unattended Upgrades Shutdown.
Nov 13 12:51:17 cc-uu-shutdown systemd-logind[228]: System is rebooting.
...
root at cc-uu-shutdown:~# cat /var/log/unattended-upgrades/unattended-upgrades-shutdown.log
...
2018-11-13 12:51:13,835 WARNING - Running unattended-upgrades in shutdown mode
2018-11-13 12:51:13,852 WARNING - Unattended-upgrade in progress during shutdown, please don't turn off the computer
2018-11-13 12:51:15,482 WARNING - Unattended-upgrade in progress during shutdown, please don't turn off the computer
2018-11-13 12:51:17,151 WARNING - Unattended-upgrade in progress during shutdown, please don't turn off the computer
2018-11-13 12:51:17,166 INFO - All upgrades installed
[Regression Potential]
* The change reverts the behavior of unattended-upgrades-shutdown to
be close to the behavior observed in Ubuntu 18.04, blocking the
shutdown process reliably until is unattended-upgrades finished, but
it starts _before_ shutdown triggered by the PrepareForShutdown()
signal. Due to the shutdown not fully starting yet users may not get
visual notification of unattended-upgrades running.
In my testing on Bionic when a logged-in user shuts down the system
when InstallOnShutdown is configured the user is dropped out to the
login manager and nothing shows that u-u is running behind the screens
for 30s, when the inhibition timer expires and u-u is starting to
gracefully stop and the usual text appears after the login manager
exits and plymouth shutdown screen is shown. The fix would be the
login manager also monitoring PrepareForShutdown() and exiting (LP:
#1803581).
On Xenial (with Unity) starting shutdown from the graphical session
does not log the user out nor show any progress on the shutdown until
the inhibition timer expires which is confusing. Users should be
logged out on PrepareForShutdown() (LP: #1803581).
* The reversion of unattended-upgrades.service was not complete in
u-u 1.7 and needed a further fix in 1.7ubuntu1 (which fix is already
included in 1.5ubuntu4). This part may be source of potential
regressions.
* The fix itself rewrites big part of unattended-upgrades-shutdown
and this rewrite could cause regressions in running unattended-
upgrades in InstallOnShutdown mode, but also this mode was extensively
tested.
* The fix includes increasing logind's InhibitDelayMaxSec to 30s to
give u-u enough time to gracefully stop in normal mode and install
packages in InstallOnShutdown mode. The delay is global, thus any
other program holding the lock can delay shutdown or sleep by 30s
instead of the original 5s default. This regression is hard to avoid
and the 30s was chosen to minimize the regression potential while
still giving more than 5s to u-u to finish actions. Bugs reporting
increased delay to sleep or shutdown should be monitored to catch
other programs misbehaving with this new default.
[Other Info]
This is a regression of introduced in LP: #1778219 and can be observed
in a cosmic lxd container easily:
$ lxc launch ubuntu:18.10 cc-uu-onshutdown
Creating cc-uu-onshutdown
Starting cc-uu-onshutdown
$ lxc shell cc-uu-onshutdown
mesg: ttyname failed: No such device
root at cc-uu-onshutdown:~# apt update -qq
33 packages can be upgraded. Run 'apt list --upgradable' to see them.
root at cc-uu-onshutdown:~# echo 'Unattended-Upgrade::InstallOnShutdown "true";' > /etc/apt/apt.conf.d/51unattended-upgrades-on-shutdown
root at cc-uu-onshutdown:~# dbus-send --system --print-reply --dest=org.freedesktop.login1 /org/freedesktop/login1 "org.freedesktop.login1.Manager.Reboot" boolean:false
method return time=1542112922.046290 sender=:1.4 -> destination=:1.13 serial=27 reply_serial=2
Session terminated, terminating shell...$
$ lxc shell cc-uu-onshutdown
mesg: ttyname failed: No such device
root at cc-uu-onshutdown:~# journalctl -l
...
Nov 13 12:42:02 cc-uu-onshutdown systemd[1]: Stopped target Local File Systems.
Nov 13 12:42:02 cc-uu-onshutdown systemd[1]: unattended-upgrades.service: Failed to reset devices.list: Operation not permitted
Nov 13 12:42:02 cc-uu-onshutdown systemd[1]: Starting Unattended Upgrades Shutdown...
..
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1803137/+subscriptions
More information about the foundations-bugs
mailing list