[Bug 1821343] Re: slapd process failure is not detected by systemd

[Heitor R. Alves de Siqueira]

I've been doing some research on how we could implement a proper systemd
unit file for slapd, and came up with interesting results.

There are a number of options that we can pass to the slapd service that will give us lots of troubles on an "idiomatic" service file. To give some examples:
- The PID file can be directly obtained from /etc/default/slapd if set, or it
  has to be parsed from /etc/ldap/slapd.d or /etc/ldap/slapd.conf. The /etc/ldap
  files don't follow the /etc/default syntax, so we can't just pass them with
  "EnvironmentFile=" in the systemd service. We would have to parse those files
  as is currently done by the init script.
- slapd has the "-f" and "-F" flags that can be used to specify a config file or
  directory, respectively. It does not accept both at the same time, but
  SLAPD_CONF in /etc/default/slapd allows either a file or directory. The init
  script will figure out which of those is set, and pass the correct flag
  accordingly. I don't see a simple way of doing this in systemd without either
  polluting the SLAPD_CONF option with the necessary flag, or delegating it to a
  helper script which would check whether the option meant a file or a
  directory.
- The slapd.conf file is used by other related tools (see slapd.conf(5)), which
  prevents us from pulling some of the options to another centralized file
  without also updating these tools

I hoped that the upstream package would have some insight on the proper
way of writing the service file, but unfortunately it seems that
upstream Openldap devs are very much against the idea of shipping them
(be they sysv-init or systemd files). There are some discussions on the
openldap lists, one of which deals specifically with adding systemd
support for slapd [0]. From that discussion, it's clear that upstream
openldap is not ready to provide the necessary systemd files out-of-the-
box, even when approached with relevant patches.

I also checked how CentOS implement their slapd.service file since they don't seem to run into these problems. They don't use an autogenerated unit file, and have a proper one instead [1]. From what I've checked, this is easier for them for two reasons:
1) The CentOS configs for slapd are much simpler than what we have in Debian/Ubuntu (their defaults don't include most of the options that we do)
2) They use helper scripts for the funky parsing we have in the sysv-init script ("ExecStartPre=/usr/libexec/openldap/check-config.sh", which also invokes some other helper scripts)

On Debian/Ubuntu, the autogenerated unit file for slapd uses the sysv-init script as a "shim" to manage the service, which makes it possible to keep the whole config-parsing insanity consistent between sysv-init and systemd.
It's not the most elegant solution, but for now I believe it makes sense to include an override file for the "RemainAfterExit=" option until we have a native systemd unit file for slapd (similar to what was done for apache2 in bug 1488962 mentioned by Ryan Tandy).

[0] https://www.openldap.org/its/index.cgi/Incoming?id=8707
[1] https://git.centos.org/blob/rpms!openldap.git/c7/SOURCES!slapd.service

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1821343

Title:
  slapd process failure is not detected by systemd

Status in openldap package in Ubuntu:
  Confirmed
Status in openldap source package in Xenial:
  Confirmed
Status in openldap source package in Bionic:
  Confirmed
Status in openldap source package in Cosmic:
  Confirmed

Bug description:
  [Impact]
  Systemd service reports slapd as active, even though it may have failed

  [Description]
  The slapd package for OpenLDAP is shipped with a SysV-style init script (/etc/init.d/slapd). Systemd automatically converts this to a systemd service by generating the unit file using the systemd-sysv-generator(8) utility. The generated unit file contains Type=forking and RemainAfterExit=yes directives.

  If the slapd daemon process exits due to some failure (e.g., it
  receives a SIGTERM or SIGKILL), the failure is not detected properly
  by systemd. The service is still reported as active even though the
  child (daemon) process has exited with a signal.

  We can easily fix this by including a proper systemd service file for
  slapd in the openldap package. Since the init.d script already does
  most of the necessary work (parsing configs, setting up PID files,
  etc.), we don't need anything complicated for the systemd unit file.
  Just making sure that RemainAfterExit is set to "no" makes the systemd
  service behave in the expected way.

  [Test Case]
  1) Deploy a disco container
  $ lxc launch images:ubuntu/disco disco

  2) Install slapd
  ubuntu at disco:~$ sudo apt update && sudo apt install slapd -y

  3) Verify that slapd is running with the auto-generated service
  ubuntu at disco:~$ systemctl status slapd
  ● slapd.service - LSB: OpenLDAP standalone server (Lightweight Directory Access Protocol)
     Loaded: loaded (/etc/init.d/slapd; generated)
     Active: active (running) since Fri 2019-03-22 11:51:22 UTC; 40min ago
       Docs: man:systemd-sysv-generator(8)
    Process: 1103 ExecStart=/etc/init.d/slapd start (code=exited, status=0/SUCCESS)
      Tasks: 3 (limit: 4915)
     Memory: 712.6M
     CGroup: /system.slice/slapd.service
             └─1109 /usr/sbin/slapd -h ldap:/// ldapi:/// -g openldap -u openldap -F /etc/ldap/slapd.d

  4) SIGKILL the slapd process (PID is displayed in systemctl status output)
  ubuntu at disco:~$ sudo kill -9 1109

  5) Check if systemd service lists slapd as still active, even though it was terminated
  ubuntu at disco:~$ systemctl status slapd
  ● slapd.service - LSB: OpenLDAP standalone server (Lightweight Directory Access Protocol)
     Loaded: loaded (/etc/init.d/slapd; generated)
     Active: active (exited) since Fri 2019-03-22 11:51:22 UTC; 42min ago
       Docs: man:systemd-sysv-generator(8)
    Process: 1103 ExecStart=/etc/init.d/slapd start (code=exited, status=0/SUCCESS)

  [Regression Potential]
  The regression potential for this fix should be very low, if we keep the new systemd unit file close to the one generated by systemd-sysv-generator(8). The only significant change would be the RemainAfterExit directive, and this should make the slapd service behave like a "normal" forking service. Nonetheless, we'll perform scripted test runs to make sure no regressions arise.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1821343/+subscriptions