[Bug 2121543] [NEW] [SRU] Poor performance of libnss-db on large db files

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Dariusz Gadomski (dgadomski):

libdnss-db opens and closes the DB file each time. This is normally not
an issue as the DB files are small.

However, in a user environment, this takes considerably longer because
the DB files have entries to the tune of 20k which severely impacts the
performance of lookups.

----

[ Impact ]

The libnss-db library, which processes Berkeley DB files, closes
and re-opens the DB file for each entry in the file that it currently scans.
This results in very poor performance and noticiable botteneck on the
application side. In a user environment that has 20k entries in the DB,
this results in about 40 times worse performance (compared to keeping
the file open).

[ Test Plan ]

The test plan sets up some DB files and queries them to see if the
open(2) and close(2) calls are repeatedly called.

1. update /etc/default/libnss-db
DBS = passwd group shadow
VAR_DB = /var/lib/misc
ETC = ${VAR_DB}/etc
AWK = awk
MAKEDB = makedb --quiet
 
2. update /etc/nsswitch.conf
passwd:         files systemd db
group:          files systemd db
shadow:         files systemd db
 
3. create required files
mkdir -p /var/lib/misc/etc
touch /var/lib/misc/etc/{passwd,group,shadow}
cp /etc/login.defs /var/lib/misc/etc
 
4. update /var/lib/misc/etc/login.defs so that UIDs are less likely to conflict
UID_MIN                  5000
 
5. create test users in berkelydb file
cd /var/lib/misc
for i in $(seq 0001 1000); do useradd -P /var/lib/misc test$i; done
make
 
6. verify users in berkeleydb is recognized
id test1000
 
7. verify that 'getent passwd' and 'getenv group' calls correctly retrieve the entr fom 
verify that there's not as many open/close calls as there are entries in the DB files

[ Where problems could occur ]

To my understanding the historical reason for doing close-open for each line is to
minimise possible resource usage and reduce the possibility of DB files getting stale
if DB files get updated externally. Potential risks revolve around those two.

In the former case, this could result in higher load on the system (modern hardware should be capable of handling that).
In the latter case, lookups could fail and likely result in re-scans from the application.

[ Other Info ]

Debian bug and patch(not merged): https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=1101371

** Affects: libnss-db (Ubuntu)
     Importance: Wishlist
     Assignee: Ponnuvel Palaniyappan (pponnuvel)
         Status: In Progress

** Affects: libnss-db (Ubuntu Noble)
     Importance: Undecided
         Status: New

** Affects: libnss-db (Ubuntu Plucky)
     Importance: Undecided
         Status: New

** Affects: libnss-db (Ubuntu Questing)
     Importance: Wishlist
     Assignee: Ponnuvel Palaniyappan (pponnuvel)
         Status: In Progress


** Tags: dcr-freezer sts
-- 
[SRU] Poor performance of libnss-db on large db files
https://bugs.launchpad.net/bugs/2121543
You received this bug notification because you are a member of Ubuntu Sponsors, which is subscribed to the bug report.