[MERGE] Btree index updates

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The attached code adds another knob (_max_zsync) which uses the same
"estimated packing" to get decent compression, but stops after a certain
number of "compressor.flush(Z_SYNC)" calls. Basically, we want to stop *just
before* we overflow. I played around with some benchmarking, and the numbers
are a bit unstable. So basically, I picked the largest _max_zsync that didn't
seem overflow much (8).

Attached is an XY plot of the compressed size versus the time it takes to get
there. You can see that the first Z_SYNC calls have a huge impact on size, and
then you get more and more marginal returns. The numbers aren't perfectly
stable, there seems to be about an 0.4s margin of error. Which means that lots
of them fall under the same range.

Anyway, this is quite a bit faster than multiple repacks (~20-40%), at a loss
of probably 10-20% space. My idea, though, is to make day-to-day operations
fast, and then allow "bzr pack" to optimize for space instead of speed.

The test suite would claim things are much worse than they really are. The
reason is because the "make_nodes()" revision ids are very low entropy, and
compress far better than real values. So they benefit much more from extra
packing.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIs1dOJdeBCYSNAAMRAii7AKCvKGsDj6OOVglNte3YmBW4EeBWzQCggdms
1LX9NBUxcgiZF4Fj4QE2fvs=
=oih7
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: btree3.patch
Type: text/x-diff
Size: 43771 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080825/4f022d27/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Performance.png
Type: image/png
Size: 18994 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080825/4f022d27/attachment-0001.png