[Bug 1702823] Re: Systemd fails to serialize tasks correctly on daemon-reload

ChristianEhrhardt 1702823 at bugs.launchpad.net
Fri Sep 8 07:45:22 UTC 2017 

# Before upgrade
./test.sh 
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 9 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
              accounts-daemon.service : 
                          atd.service : 
                         cron.service : 
                         dbus.service : 
                       iscsid.service : 
                       polkit.service : 
                      rsyslog.service : 
                        snapd.service : 
                          ssh.service : 
             systemd-journald.service : 
               systemd-logind.service : 
             systemd-resolved.service : 
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 9 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)

# Confirmed still failing

# Upgrade install
root at zesty-test:~# vim /etc/apt/sources.list
root at zesty-test:~# apt update
Hit:1 http://archive.ubuntu.com/ubuntu zesty InRelease
Hit:2 http://archive.ubuntu.com/ubuntu zesty-updates InRelease                                       
Hit:3 http://archive.ubuntu.com/ubuntu zesty-backports InRelease                                     
Get:4 http://archive.ubuntu.com/ubuntu zesty-proposed InRelease [240 kB]       
Hit:5 http://security.ubuntu.com/ubuntu zesty-security InRelease                    
Get:6 http://archive.ubuntu.com/ubuntu zesty-proposed/universe Sources [15.7 kB]    
Get:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main Sources [12.4 kB]
Get:8 http://archive.ubuntu.com/ubuntu zesty-proposed/main amd64 Packages [22.2 kB]
Get:9 http://archive.ubuntu.com/ubuntu zesty-proposed/main Translation-en [13.7 kB]
Get:10 http://archive.ubuntu.com/ubuntu zesty-proposed/universe amd64 Packages [27.4 kB]
Get:11 http://archive.ubuntu.com/ubuntu zesty-proposed/universe Translation-en [16.8 kB]
Fetched 349 kB in 0s (565 kB/s)          
Reading package lists... Done
Building dependency tree       
Reading state information... Done
29 packages can be upgraded. Run 'apt list --upgradable' to see them.
root at zesty-test:~# apt upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  apt apt-transport-https apt-utils base-files cloud-init libapt-inst2.0 libapt-pkg5.0 libgnutls-openssl27 libgnutls28-dev libgnutls30
  libgnutlsxx28 libnss-resolve libpam-systemd libpq-dev libpq5 libsystemd0 libudev-dev libudev1 linux-libc-dev openssh-client openssh-server
  openssh-sftp-server python3-update-manager snapd systemd systemd-sysv udev unattended-upgrades update-manager-core
29 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 17.7 MB of archives.
After this operation, 43.0 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.
root at zesty-test:~# apt install systemd systemd-sysv
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libnss-resolve libpam-systemd libsystemd0
Suggested packages:
  systemd-ui systemd-container
The following packages will be upgraded:
  libnss-resolve libpam-systemd libsystemd0 systemd systemd-sysv
5 upgraded, 0 newly installed, 0 to remove and 24 not upgraded.
Need to get 2816 kB of archives.
After this operation, 4096 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu zesty-proposed/main amd64 libnss-resolve amd64 232-21ubuntu6 [114 kB]
Get:2 http://archive.ubuntu.com/ubuntu zesty-proposed/main amd64 systemd-sysv amd64 232-21ubuntu6 [13.1 kB]
Get:3 http://archive.ubuntu.com/ubuntu zesty-proposed/main amd64 libpam-systemd amd64 232-21ubuntu6 [116 kB]
Get:4 http://archive.ubuntu.com/ubuntu zesty-proposed/main amd64 systemd amd64 232-21ubuntu6 [2363 kB]
Get:5 http://archive.ubuntu.com/ubuntu zesty-proposed/main amd64 libsystemd0 amd64 232-21ubuntu6 [211 kB]
Fetched 2816 kB in 1s (2576 kB/s)   
(Reading database ... 54213 files and directories currently installed.)
Preparing to unpack .../libnss-resolve_232-21ubuntu6_amd64.deb ...
Unpacking libnss-resolve:amd64 (232-21ubuntu6) over (232-21ubuntu5) ...
Preparing to unpack .../systemd-sysv_232-21ubuntu6_amd64.deb ...
Unpacking systemd-sysv (232-21ubuntu6) over (232-21ubuntu5) ...
Preparing to unpack .../libpam-systemd_232-21ubuntu6_amd64.deb ...
Unpacking libpam-systemd:amd64 (232-21ubuntu6) over (232-21ubuntu5) ...
Preparing to unpack .../systemd_232-21ubuntu6_amd64.deb ...
Unpacking systemd (232-21ubuntu6) over (232-21ubuntu5) ...
Preparing to unpack .../libsystemd0_232-21ubuntu6_amd64.deb ...
Unpacking libsystemd0:amd64 (232-21ubuntu6) over (232-21ubuntu5) ...
Setting up libsystemd0:amd64 (232-21ubuntu6) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for libc-bin (2.24-9ubuntu2.2) ...
Setting up systemd (232-21ubuntu6) ...
Installing new version of config file /etc/systemd/resolved.conf ...
addgroup: The group `systemd-journal' already exists as a system group. Exiting.
Setting up libnss-resolve:amd64 (232-21ubuntu6) ...
Processing triggers for man-db (2.7.6.1-2) ...
Processing triggers for dbus (1.10.10-1ubuntu2) ...
Setting up systemd-sysv (232-21ubuntu6) ...
Setting up libpam-systemd:amd64 (232-21ubuntu6) ...
Processing triggers for libc-bin (2.24-9ubuntu2.2) ...

# Good after Upgrade
root at zesty-test:~# ./test.sh                
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 9 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 9 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 9 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)


Thanks for your work xnox!
Seems to work well now.

** Tags removed: verification-needed verification-needed-zesty
** Tags added: verification-done verification-done-zesty

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1702823

Title:
  Systemd fails to serialize tasks correctly on daemon-reload

Status in systemd:
  Fix Released
Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Zesty:
  Fix Committed
Status in systemd source package in Artful:
  Fix Released
Status in systemd package in Debian:
  Fix Released

Bug description:
  [Impact]
  A unit with multiple Exec stanzas will loose execution state (which stanzas, out of multiple, have executed or not) upon systemd daemon-reload during the execution of these stanzas. This results in lost data/state, and units potentially failing to start/restart in unpredictable manner

  [Testcase]
  Setup a race of a unit with multiple exec stanzas (e.g. echo, sleep, echo) and execute daemon-reload whilst that is in progress. The execution state should persist across daemon-reload with all exec stanzas completing, in order, without duplicates.

  [Original Bug report]

  Hi,
  I was initially tracking down a libvirt bug [1], but happened to realize it is actually a general systemd issue.
  From there I got to file, discuss and test [2].

  For now I file the bug to be ablte to do my tests correctly, but if it
  really fixes the issue I'd like to suggest to fix it in the release -
  so better file upfront and do even my test patches right.

  [1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867379
  [2]: https://github.com/systemd/systemd/issues/6299.

  [Regression Potential]
  Minimal, this is a fix for incorrect upstream behaviour which resolves a racy condition. The code changes are upstream reviewed cherry-pick.

To manage notifications about this bug go to:
https://bugs.launchpad.net/systemd/+bug/1702823/+subscriptions

More information about the foundations-bugs mailing list