Initial benchmarks for checkout performance improvements

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Aaron Bentley wrote:
> One of our goals is to improve the performance of tree-building
> operations for large trees.  To that end, I've started benchmarking
> various tree-building strategies.

Yesterday's benchmarking suggested that renames were more expensive than
I'd realized for large trees.

So I went and implemented a few new strategies for tree building.

top-rename
==========
The first was "top-rename".  This is a variation of the "rename"
strategy where only the top directories are renamed.  I.e., it is a
simulation of what a modified TreeTransform might do.

The top-level files are put in the top level of the temp directory.  But
lower-level files are put in their parent directories, not the top of
the temp directory.

This means that in order to activate the transform, only the top-level
files need to be renamed.  So you have far fewer rename syscalls.  It
also means your temp directory isn't unreasonably large.  (Directories
which directly contain large numbers of files are slow on many filesystems.)

Performance of the top-rename strategy was similar to copytree:

top-rename 9
real 6.14
user 0.89
sys 2.22

copytree 4
real 4.70
user 1.51
sys 2.44

In this run of benchmarks, cp was actually slower than copytree or
top-rename.  But cp has previously hit 2.183s.

The high performance of top-rename is great news, because I think this
strategy strikes a good balance between safety and performance, and I
think it would not be too hard to implement in the near future.

rename-multi
============
I also implemented a second strategy: "rename-multi".  It is the same as
top-rename, except that it reads the contents of all files in a
directory at the same time, then writes them all out from memory at the
same time.  This does not seem to be faster than "top-rename".

rename-multi 3
real 6.47
user 0.98
sys 2.39

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGXzT+0F+nu1YWqI0RAosQAJ9MltA0b+5zqZtNgJxjS5lElkousACfV5am
NawKDuudoSHY2Xxk5+pcb58=
=93JP
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: bench4
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20070531/ca24302b/attachment-0001.diff