[Bug 1013446] [NEW] Uncached grp and pwd calls make duplicity slow with large group and passwd maps

Launchpad Bug Tracker 1013446 at bugs.launchpad.net
Wed Nov 14 18:09:17 UTC 2012


You have been subscribed to a public bug by Chris J Arges (christopherarges):

With large /etc/group and /etc/passwd files, duplicity spends the vast
majority of its time mapping group and user names to gids and uids.
This is with duplicity 0.6.18-0ubuntu3 on Ubuntu 12.04 and Python 2.7.3.

I have a largish group map (7MB, with many group entries having lines in
the 100KB - 200KB range.)  A zero change incremental backup of ~200k
files takes about 23 minutes.

Profiling the duplicity run shows that 92% of the time is spent calling
grp.getgrnam!  Another ~2% is spent calling grp.getgrgid, pwd.getpwnam,
and pwd.getpwuid.

Some simple caching of caching of this information would make a big
improvement.

--

SRU Justification:
[Impact] 
 * This fix should be backported into Precise LTS because users of duplicity with large group/passwd files will encounter performance issues.

[Test Case]
 * Using a large group/passwd file configuration, perform a backup and record the time it takes to backup. Profile this as well and determine how much time is spent in grp.getgrnam. Compare this with the patched version and ensure improvement in performance.

[Regression Potential] 
 * This is a backported patch, and is currently in the latest duplicity. However; because it is a new feature we will need to ensure backups work as expected with this patch.

** Affects: duplicity
     Importance: Undecided
         Status: Fix Committed

** Affects: duplicity (Ubuntu)
     Importance: Medium
         Status: New

-- 
Uncached grp and pwd calls make duplicity slow with large group and passwd maps
https://bugs.launchpad.net/bugs/1013446
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.



More information about the Ubuntu-sponsors mailing list