[Bug 1185908] Re: slapd: slapcat output truncated every now and then

[Sebastien Bacher]

@Ryan: thanks, I've sponsored the raring and precise ones (current
stable and LTS), the SRU and verification teams are quite busy so we try
to avoid the non-current-non-LTS SRUS, I'm going to skip the quantal one

** Changed in: openldap (Ubuntu Precise)
       Status: Triaged => In Progress

** Changed in: openldap (Ubuntu Quantal)
       Status: Triaged => Won't Fix

** Changed in: openldap (Ubuntu Raring)
       Status: Triaged => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1185908

Title:
  slapd: slapcat output truncated every now and then

Status in “openldap” package in Ubuntu:
  Fix Released
Status in “openldap” source package in Precise:
  In Progress
Status in “openldap” source package in Quantal:
  Won't Fix
Status in “openldap” source package in Raring:
  In Progress
Status in “openldap” package in Debian:
  Fix Released

Bug description:
  SRU justification:

  [Impact]

  The slapd tools (slapcat, slapadd, et al) don't retry after failing to
  acquire a BDB read lock, and on a busy LDAP server can sometimes
  return incomplete data. This could result in data loss, for example
  when slapcat is used to take a hot backup.

  Debian bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=673038
  OpenLDAP thread: http://www.openldap.org/lists/openldap-technical/201301/msg00195.html
  OpenLDAP ITS:
   - http://www.openldap.org/its/index.cgi?findid=6365
   - http://www.openldap.org/its/index.cgi?findid=7503

  The impact is limited to slapd servers with a write load high enough
  to generate lock contention. It's been present at least since 2.4.17
  and maybe longer, and at least some people got used to working around
  it, e.g. [1], but not everyone is aware that the problem exists.

  The fix is minimal and has been tested in OpenLDAP upstream and Debian
  wheezy.

  [1] https://github.com/elmar/ldap-git-backup/blob/master/README.mdown
  #safe-ldif

  [Test Case]

  # apt-get install ldap-utils slapd
  <configure admin password>
  # ldapadd -D cn=admin,dc=nodomain -w adminpw <<end
  dn: cn=test,dc=nodomain
  objectClass: organizationalRole
  objectClass: simpleSecurityObject
  cn: test
  userPassword: test

  end
  # while true; do slapcat | wc -l; done

  and in another terminal...

  $ while true; do ldappasswd -H ldap:// -D cn=admin,dc=nodomain -w
  adminpw cn=test,dc=nodomain; done

  In the first terminal, note that the output from wc is usually 41 but
  sometimes smaller. It should be the same line count every time.

  [Regression Potential]

  The regression risk should be small. The change is minimal, was
  authored by upstream, and has been accepted and released in Debian
  wheezy. I admit to not being familiar enough with the code to comment
  in detail on what regressions might be possible. If the fix were
  faulty wrt locking, I would hope for it to turn up during verification
  since the test case involves inducing a heavy write load on the
  server.

  original description:

  Debian #673038 was fixed in wheezy but the fix has never been merged
  to Ubuntu. I verified the existence of this bug in precise, quantal,
  raring, and saucy using more or less the procedure from
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=673038#111:

  # apt-get install ldap-utils slapd
  <configure admin password>
  # ldapadd -D cn=admin,dc=nodomain -w adminpw <<end
  dn: cn=test,dc=nodomain
  objectClass: organizationalRole
  objectClass: simpleSecurityObject
  cn: test
  userPassword: test

  end
  # while true; do slapcat | wc -l; done

  and in another terminal...

  $ while true; do ldappasswd -H ldap:// -D cn=admin,dc=nodomain -w
  adminpw cn=test,dc=nodomain; done

  In the first terminal, note that the output from wc is usually 41 but
  sometimes smaller. It should be the same line count every time.

  I'm building and testing patched packages now and will post debdiffs
  shortly.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/openldap/+bug/1185908/+subscriptions