[Bug 1839290] Re: systemd doesn't restart a service after crashes

Wed May 6 11:32:31 UTC 2020

xenial:

ubuntu at lp1839290-x:~$ dpkg -l systemd|grep systemd
ii  systemd        229-4ubuntu21.27 amd64        system and service manager
ubuntu at lp1839290-x:~$ systemctl cat foo
# /etc/systemd/system/foo.service
# /etc/systemd/system/foo.service
[Unit]
PartOf=bar.service
[Service]
ExecStart=/bin/sleep infinity
RestartSec=0s
Restart=always

ubuntu at lp1839290-x:~$ systemctl cat bar
# /etc/systemd/system/bar.service
# /etc/systemd/system/bar.service
[Unit]
BindsTo=foo.service
#Same with or without After=, the window is very small to make a difference.
After=foo.service
[Service]
ExecStart=/bin/sleep infinity

ubuntu at lp1839290-x:~$ sudo systemctl start foo
ubuntu at lp1839290-x:~$ sudo systemctl start bar
ubuntu at lp1839290-x:~$ systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:28:14 UTC; 5s ago
 Main PID: 1268 (sleep)
    Tasks: 1
   Memory: 80.0K
      CPU: 1ms
   CGroup: /system.slice/foo.service
           └─1268 /bin/sleep infinity

May 06 11:28:14 lp1839290-x systemd[1]: Started foo.service.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:28:17 UTC; 3s ago
 Main PID: 1273 (sleep)
    Tasks: 1
   Memory: 84.0K
      CPU: 1ms
   CGroup: /system.slice/bar.service
           └─1273 /bin/sleep infinity

May 06 11:28:17 lp1839290-x systemd[1]: Started bar.service.
ubuntu at lp1839290-x:~$ sudo kill -6 1268
ubuntu at lp1839290-x:~$ systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabled)
   Active: failed (Result: resources) since Wed 2020-05-06 11:28:29 UTC; 929ms ago
  Process: 1268 ExecStart=/bin/sleep infinity (code=dumped, signal=ABRT)
 Main PID: 1268 (code=dumped, signal=ABRT)

May 06 11:28:29 lp1839290-x systemd[1]: foo.service: Unit entered failed state.
May 06 11:28:29 lp1839290-x systemd[1]: foo.service: Failed with result 'core-dump'.
May 06 11:28:29 lp1839290-x systemd[1]: foo.service: Service has no hold-off time, scheduling restart.
May 06 11:28:29 lp1839290-x systemd[1]: foo.service: Failed to schedule restart job: Transaction is destructive.
May 06 11:28:29 lp1839290-x systemd[1]: foo.service: Unit entered failed state.
May 06 11:28:29 lp1839290-x systemd[1]: foo.service: Failed with result 'resources'.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabled)
   Active: inactive (dead) since Wed 2020-05-06 11:28:29 UTC; 931ms ago
  Process: 1273 ExecStart=/bin/sleep infinity (code=killed, signal=TERM)
 Main PID: 1273 (code=killed, signal=TERM)

May 06 11:28:17 lp1839290-x systemd[1]: Started bar.service.
May 06 11:28:29 lp1839290-x systemd[1]: Stopping bar.service...
May 06 11:28:29 lp1839290-x systemd[1]: Stopped bar.service.

ubuntu at lp1839290-x:~$ dpkg -l systemd|grep systemd
ii  systemd        229-4ubuntu21.28 amd64        system and service manager
ubuntu at lp1839290-x:~$ sudo systemctl start foo
ubuntu at lp1839290-x:~$ sudo systemctl start bar
ubuntu at lp1839290-x:~$ systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabl
   Active: active (running) since Wed 2020-05-06 11:31:03 UTC; 6s ago
 Main PID: 1238 (sleep)
    Tasks: 1
   Memory: 84.0K
      CPU: 1ms
   CGroup: /system.slice/foo.service
           └─1238 /bin/sleep infinity

May 06 11:31:03 lp1839290-x systemd[1]: Started foo.service.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabl
   Active: active (running) since Wed 2020-05-06 11:31:05 UTC; 3s ago
 Main PID: 1243 (sleep)
    Tasks: 1
   Memory: 84.0K
      CPU: 1ms
   CGroup: /system.slice/bar.service
           └─1243 /bin/sleep infinity

May 06 11:31:05 lp1839290-x systemd[1]: Started bar.service.

ubuntu at lp1839290-x:~$ sudo kill -6 1238
ubuntu at lp1839290-x:~$ systemctl status foo bar
● foo.service
   Loaded: loaded (/etc/systemd/system/foo.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:31:32 UTC; 1s ago
 Main PID: 1257 (sleep)
    Tasks: 1
   Memory: 80.0K
      CPU: 1ms
   CGroup: /system.slice/foo.service
           └─1257 /bin/sleep infinity

May 06 11:31:32 lp1839290-x systemd[1]: Started foo.service.

● bar.service
   Loaded: loaded (/etc/systemd/system/bar.service; static; vendor preset: enabled)
   Active: active (running) since Wed 2020-05-06 11:31:32 UTC; 1s ago
 Main PID: 1258 (sleep)
    Tasks: 1
   Memory: 80.0K
      CPU: 1ms
   CGroup: /system.slice/bar.service
           └─1258 /bin/sleep infinity

May 06 11:31:32 lp1839290-x systemd[1]: Started bar.service.
ubuntu at lp1839290-x:~$ journalctl -b -u foo
-- Logs begin at Wed 2020-05-06 11:30:16 UTC, end at Wed 2020-05-06 11:31:35 UTC. --
May 06 11:31:03 lp1839290-x systemd[1]: Started foo.service.
May 06 11:31:32 lp1839290-x systemd[1]: foo.service: Main process exited, code=dumped, status=6/ABRT
May 06 11:31:32 lp1839290-x systemd[1]: foo.service: Unit entered failed state.
May 06 11:31:32 lp1839290-x systemd[1]: foo.service: Failed with result 'core-dump'.
May 06 11:31:32 lp1839290-x systemd[1]: foo.service: Service has no hold-off time, scheduling restart.
May 06 11:31:32 lp1839290-x systemd[1]: Stopped foo.service.
May 06 11:31:32 lp1839290-x systemd[1]: Started foo.service.
ubuntu at lp1839290-x:~$ journalctl -b -u bar
-- Logs begin at Wed 2020-05-06 11:30:16 UTC, end at Wed 2020-05-06 11:31:35 UTC. --
May 06 11:31:05 lp1839290-x systemd[1]: Started bar.service.
May 06 11:31:32 lp1839290-x systemd[1]: Stopping bar.service...
May 06 11:31:32 lp1839290-x systemd[1]: Stopped bar.service.
May 06 11:31:32 lp1839290-x systemd[1]: Started bar.service.

** Tags removed: verification-needed verification-needed-xenial
** Tags added: verification-done verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1839290

Title:
  systemd doesn't restart a service after crashes

Status in systemd:
  Fix Released
Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Xenial:
  Fix Committed
Status in systemd source package in Bionic:
  Fix Committed
Status in systemd source package in Disco:
  Fix Released

Bug description:
  [impact]

  when a service configured to restart crashes, under certain
  configurations, that service is not correctly restarted.

  [test case]

  see comment 8

  [regression potential]

  this changes the job mode of manager-triggered restarts to 'replace'
  any existing queued job(s), instead of failing if there are queued
  job(s).  thus any regressions would occur when a service fails, that
  is configured to restart on failure.

  [scope]

  This is needed only for Xenial and Bionic.

  this is fixed with commit 03ff2dc71ecb09272d728d458498b44f7f132f51
  which is included already in Eoan.

  Fixed in Disco with the backported v240 patch from
  https://github.com/systemd/systemd/commit/677b4cc753f183731fc54fcb68ad46f806c394bc

  [other info]

  original description:
  ---

  Affected versions of OS and systemd:
  $ cat /etc/issue
  Ubuntu 16.04.6 LTS \n \l
  $ systemd --version
  systemd 229
  +PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN

  Affected packages:
  systemd 229-4ubuntu21.22 and previous versions.

  Expected behaviour you didn't see:
  Scheduling restart of failed service.
  A process crashed by sigabrt and didn't restart.

  Description:
  The bug was reported to a systemd upstream repository: https://github.com/systemd/systemd/issues/11456
  The bug was fixed and accepted to the master branch: https://github.com/systemd/systemd/pull/11467/files

  Action:
  Include this patch to Ubuntu 16.04 and other version of Ubuntu which are supported.

To manage notifications about this bug go to:
https://bugs.launchpad.net/systemd/+bug/1839290/+subscriptions