[Bug 1702823] Re: Systemd fails to serialize tasks correctly on daemon-reload

[Dimitri John Ledkov]

Reproducing the bug on zesty:
$ dpkg-query -W systemd
systemd	232-21ubuntu5
$ bash test.sh 
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                        acpid.service :     Tasks: 1 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                 lvm2-lvmetad.service :     Tasks: 1 (limit: 4915)
                        lxcfs.service :     Tasks: 3 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 6 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-networkd.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
            systemd-timesyncd.service :     Tasks: 2 (limit: 4915)
                systemd-udevd.service :     Tasks: 1
              accounts-daemon.service : 
                        acpid.service : 
                          atd.service : 
                         cron.service : 
                         dbus.service : 
                       iscsid.service : 
                 lvm2-lvmetad.service : 
                        lxcfs.service : 
                       polkit.service : 
                      rsyslog.service : 
                        snapd.service : 
                          ssh.service : 
             systemd-journald.service : 
               systemd-logind.service : 
             systemd-networkd.service : 
             systemd-resolved.service : 
            systemd-timesyncd.service : 
                systemd-udevd.service : 
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                        acpid.service :     Tasks: 1 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                 lvm2-lvmetad.service :     Tasks: 1 (limit: 4915)
                        lxcfs.service :     Tasks: 3 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 6 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-networkd.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
            systemd-timesyncd.service :     Tasks: 2 (limit: 4915)
                systemd-udevd.service :     Tasks: 1

Upgrading
$ dpkg-query -W systemd
systemd	232-21ubuntu7
$ bash test.sh 
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                        acpid.service :     Tasks: 1 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                 lvm2-lvmetad.service :     Tasks: 1 (limit: 4915)
                        lxcfs.service :     Tasks: 3 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 5 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-networkd.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
            systemd-timesyncd.service :     Tasks: 2 (limit: 4915)
                systemd-udevd.service :     Tasks: 1
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                        acpid.service :     Tasks: 1 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                 lvm2-lvmetad.service :     Tasks: 1 (limit: 4915)
                        lxcfs.service :     Tasks: 3 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 5 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-networkd.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
            systemd-timesyncd.service :     Tasks: 2 (limit: 4915)
                systemd-udevd.service :     Tasks: 1
              accounts-daemon.service :     Tasks: 3 (limit: 4915)
                        acpid.service :     Tasks: 1 (limit: 4915)
                          atd.service :     Tasks: 1 (limit: 4915)
                         cron.service :     Tasks: 1 (limit: 4915)
                         dbus.service :     Tasks: 1 (limit: 4915)
                       iscsid.service :     Tasks: 2 (limit: 4915)
                 lvm2-lvmetad.service :     Tasks: 1 (limit: 4915)
                        lxcfs.service :     Tasks: 3 (limit: 4915)
                       polkit.service :     Tasks: 3 (limit: 4915)
                      rsyslog.service :     Tasks: 4 (limit: 4915)
                        snapd.service :     Tasks: 5 (limit: 4915)
                          ssh.service :     Tasks: 1 (limit: 4915)
             systemd-journald.service :     Tasks: 1 (limit: 4915)
               systemd-logind.service :     Tasks: 1 (limit: 4915)
             systemd-networkd.service :     Tasks: 1 (limit: 4915)
             systemd-resolved.service :     Tasks: 1 (limit: 4915)
            systemd-timesyncd.service :     Tasks: 2 (limit: 4915)
                systemd-udevd.service :     Tasks: 1

Looking much better.

** Tags removed: verification-needed verification-needed-zesty
** Tags added: verification-done verification-done-zesty

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1702823

Title:
  Systemd fails to serialize tasks correctly on daemon-reload

Status in systemd:
  Fix Released
Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Zesty:
  Fix Committed
Status in systemd source package in Artful:
  Fix Released
Status in systemd package in Debian:
  Fix Released

Bug description:
  [Impact]
  A unit with multiple Exec stanzas will loose execution state (which stanzas, out of multiple, have executed or not) upon systemd daemon-reload during the execution of these stanzas. This results in lost data/state, and units potentially failing to start/restart in unpredictable manner

  [Testcase]
  Setup a race of a unit with multiple exec stanzas (e.g. echo, sleep, echo) and execute daemon-reload whilst that is in progress. The execution state should persist across daemon-reload with all exec stanzas completing, in order, without duplicates.

  [Original Bug report]

  Hi,
  I was initially tracking down a libvirt bug [1], but happened to realize it is actually a general systemd issue.
  From there I got to file, discuss and test [2].

  For now I file the bug to be ablte to do my tests correctly, but if it
  really fixes the issue I'd like to suggest to fix it in the release -
  so better file upfront and do even my test patches right.

  [1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867379
  [2]: https://github.com/systemd/systemd/issues/6299.

  [Regression Potential]
  Minimal, this is a fix for incorrect upstream behaviour which resolves a racy condition. The code changes are upstream reviewed cherry-pick.

To manage notifications about this bug go to:
https://bugs.launchpad.net/systemd/+bug/1702823/+subscriptions