[Bug 1688575] Re: Segmentation fault on a slave slapd (sync replication with kerberos authentication)

Fri May 18 18:15:42 UTC 2018

** Description changed:

  [Impact]
  Concurrent SASL authentications could trigger a segfault. This was observed by the bug reporter during replication from a master to a slave, and can be reproduced with a test program.

  The fix is applied upstream, see comment #13.

  [Test Case]
  * Create a fresh xenial VM or container and login. Update the apt repositories:
  sudo apt update

  * Create a local directory and cd into it:
  mkdir test && cd test

  * Download the test attachments from this bug: Makefile, sasltest.c and testscript:
  wget https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1688575/+attachment/5139678/+files/Makefile https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1688575/+attachment/5139679/+files/sasltest.c https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1688575/+attachment/5139680/+files/testscript

  * Execute the testscript with sudo once. It shall fail at the very end with a core dump:
  sudo sh ./testscript
  (...)
  sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
  Aborted (core dumped)

  * Export this var:
  export LDAPSASL_SECPROPS=none

- 
  * Run the actual test script a few more times to confirm the crasH:
- $ ./sasltest 
+ $ ./sasltest
  rc = -6 (Unknown authentication method)
  sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
  rc = -6 (Unknown authentication method)
  sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
  rc = -6 (Unknown authentication method)
  sasltest: sasltest.c:70: bind_thread: Assertion `rc == LDAP_SUCCESS' failed.
  Aborted (core dumped)

  * Install the updated packages from proposed

  * Run ./sasltest again. Make sure the LDAPSASL_SECPROPS var is still exported:
- $ echo $LDAPSASL_SECPROPS 
+ $ echo $LDAPSASL_SECPROPS
  none

  $ ./sasltest
  $

  This time the test completes without crashing.

+ [Regression Potential]
+ This is SASL authentication, and with kerberos nonetheless (in the case of the bug reporter). I suspect not many people deploy this due to its complexity. The ones that have such a setup, however, tend to know what they are doing, so if they say the problem is fixed for them, I believe it is.

- [Regression Potential]
- 
-  * discussion of how regressions are most likely to manifest as a result
- of this change.
- 
-  * It is assumed that any SRU candidate patch is well-tested before
-    upload and has a low overall risk of regression, but it's important
-    to make the effort to think about what ''could'' happen in the
-    event of a regression.
- 
-  * This both shows the SRU team that the risks have been considered,
-    and provides guidance to testers in regression-testing the SRU.
+ About this particular change, it's committed upstream and also in
+ debian, and @rtandy was kind enough to provide a sample test script that
+ exhibits the problem.

  [Other Info]
+ Since the fix is applied upstream and in Debian, there shouldn't be additional surprises here.

-  * Anything else you think is useful to include
-  * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board
-  * and address these questions in advance

  [Original description]

  I have a slapd problem on a freshly installed 16.04 machine:

  slapd[17107]: segfault at 1a ip 00007f3c12c79f55 sp 00007f3c03c2d080
  error 4 in libsasl2.so.2.0.25[7f3c12c72000+19000]

  I'm using the server as Slave LDAP-Server and sync replication with kerberos authentication.
  The service either starts and runs successfully or it fails with segmentation fault or 100% CPU.
  Maybe an useful info, I'm replicating two databases. When I deactivate syncrepl for one of them (doesn't matter which one) the problem is not occuring.

  Linux xxx 4.4.0-75-generic #96-Ubuntu SMP Thu Apr 20 09:56:33 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  slapd 2.4.42+dfsg-2ubuntu3.1
  libsasl2-2:amd64 2.1.26.dfsg1-14build1
  libsasl2-modules:amd64 2.1.26.dfsg1-14build1
  libsasl2-modules-gssapi-mit:amd64 2.1.26.dfsg1-14build1

  GDB debug:

  Starting program: /usr/sbin/slapd -h "ldap:/// ldaps:/// ldapi:///" -u openldap -g openldap -f /etc/ldap/slapd.conf -d 256
  [Thread debugging using libthread_db enabled]
  Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
  590c82ab @(#) $OpenLDAP: slapd  (Ubuntu) (May 11 2016 16:12:05) $
   buildd at lgw01-10:/build/openldap-mF7Kfq/openldap-2.4.42+dfsg/debian/build/servers/slapd
  590c82ab slapd starting
  [New Thread 0x7f2e96b7b700 (LWP 42139)]
  [New Thread 0x7f2e9637a700 (LWP 42140)]
  [New Thread 0x7f2e95b79700 (LWP 42141)]
  [New Thread 0x7f2e95378700 (LWP 42142)]
  [New Thread 0x7f2e94b77700 (LWP 42143)]
  590c82ba slap_client_connect: URI=ldap://xxx ldap_sasl_interactive_bind_s failed (-6)
  590c82ba do_syncrepl: rid=132 rc -6 retrying (9 retries left)

  Thread 4 "slapd" received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 0x7f2e95b79700 (LWP 42141)]
  0x00007f2ea53035b5 in sasl_client_add_plugin () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2

  (gdb) thr apply all bt

  Thread 6 (Thread 0x7f2e94b77700 (LWP 42143)):
  #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  #1  0x00007f2ea59463f3 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #2  0x00007f2ea487c6ba in start_thread (arg=0x7f2e94b77700) at pthread_create.c:333
  #3  0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

  Thread 5 (Thread 0x7f2e95378700 (LWP 42142)):
  #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  #1  0x00007f2ea59463f3 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #2  0x00007f2ea487c6ba in start_thread (arg=0x7f2e95378700) at pthread_create.c:333
  #3  0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

  Thread 4 (Thread 0x7f2e95b79700 (LWP 42141)):
  #0  0x00007f2ea53035b5 in sasl_client_add_plugin () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
  #1  0x00007f2ea530f250 in ?? () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
  #2  0x00007f2ea5303d69 in sasl_client_init () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2
  #3  0x00007f2ea594da6c in ldap_int_sasl_init () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #4  0x00007f2ea594db2c in ldap_int_sasl_open () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #5  0x00007f2ea594e2d4 in ldap_int_sasl_bind () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #6  0x00007f2ea5951828 in ldap_sasl_interactive_bind () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #7  0x00007f2ea5951a4e in ldap_sasl_interactive_bind_s () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #8  0x0000561fbc556db4 in slap_client_connect (ldp=0x561fbe1e9f68, sb=0x561fbe1e9d40) at ../../../../servers/slapd/config.c:2063
  #9  0x0000561fbc5c699d in do_syncrep1 (si=0x561fbe1e9d10, op=0x7f2e95b787b0) at ../../../../servers/slapd/syncrepl.c:618
  #10 do_syncrepl (ctx=<optimized out>, arg=0x561fbe1e5620) at ../../../../servers/slapd/syncrepl.c:1548
  #11 0x00007f2ea59463a2 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #12 0x00007f2ea487c6ba in start_thread (arg=0x7f2e95b79700) at pthread_create.c:333
  #13 0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

  Thread 3 (Thread 0x7f2e9637a700 (LWP 42140)):
  ---Type <return> to continue, or q <return> to quit---
  #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  #1  0x00007f2ea59463f3 in ?? () from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2
  #2  0x00007f2ea487c6ba in start_thread (arg=0x7f2e9637a700) at pthread_create.c:333
  #3  0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

  Thread 2 (Thread 0x7f2e96b7b700 (LWP 42139)):
  #0  0x00007f2ea45b2e23 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84
  #1  0x0000561fbc55a8f0 in slapd_daemon_task (ptr=<optimized out>) at ../../../../servers/slapd/daemon.c:2539
  #2  0x00007f2ea487c6ba in start_thread (arg=0x7f2e96b7b700) at pthread_create.c:333
  #3  0x00007f2ea45b282d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

  Thread 1 (Thread 0x7f2ea5d96740 (LWP 42138)):
  #0  0x00007f2ea487d98d in pthread_join (threadid=139838073845504, thread_return=0x0) at pthread_join.c:90
  #1  0x0000561fbc55cc81 in slapd_daemon () at ../../../../servers/slapd/daemon.c:2932
  #2  0x0000561fbc543bea in main (argc=11, argv=<optimized out>) at ../../../../servers/slapd/main.c:1017
  (gdb)

-- 
You received this bug notification because you are a member of Ubuntu
Server, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1688575

Title:
  Segmentation fault on a slave slapd (sync replication with kerberos
  authentication)

To manage notifications about this bug go to:
https://bugs.launchpad.net/openldap/+bug/1688575/+subscriptions