brisbane:CHKMap.iteritems() tweaks

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> 1) Change 'bzr pack' so that it creates a separate set of groups for
> items that are available in the newest XX (say 100) revisions. Or
> possibly group everything into 100 rev chunks.

This was easy to implement for CHK streams. And it changes "time bzr ls
- -r-1" from 4.4s down to 1.6s. (I implemented as splitting at 10
revisions). And the time without the patch is 2.2s up from 1.6s. So the
patch makes a bigger difference when we aren't swamped with extract time.

The total size on disk after packing is barely noticeable:
 125666
 125981

I guess that is 300KiB. But out of 125MiB, that is only 0.2%.

Also, it seems that the breakdown now is 0.93s spent in
'get_build_details'. I suppose in one sense it is nice to be down to the
point where the index performance is the bottle neck, as it means we've
gotten the other things faster. It just happens that chk indexes are
just super wide and have no sense of locality, so the page cache just
gets thrashed. I see 960 btree pages getting read, and there are only
3100 total. So to extract just one full inventory is causing us to read
1/3rd of the .cix... :(

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAknIQEoACgkQJdeBCYSNAANWUgCgs05njsUoTafceLdvcIabCEog
pE4An1JA1sY4w0DhLsPfnKxdB1HZVGGQ
=FH/n
-----END PGP SIGNATURE-----