[MERGE] Add a new repositoy method _generate_text_key_index for use by reconcile/check.

[Robert Collins]

This builds on the find_text_key_references API to generate an ideal
index for the texts of a repository. This can then be used by reconcile
to regenerate a knit or pack repository efficiently.

This implementation has no cache hooked in yet - I was doing this in
parallel. Without the cache it runs acceptably fast for me - 12K texts
in 6 minutes, or about 2K per minute. This seemed the same on both a
launchpad tree and a bzr.dev tree. So, for bzr.dev this will take about
18 minutes (without the cache) to seed the entire index. It seems to use
about 80-90M of memory - a significant improvement on the 6.8G used by
bzr.dev.

Next up I'll add the index LRU cache and hook this into the current
versioned file check logic. After that I'll hook it into the pack
reconcile logic to allow pack reconciliation.

-Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bzr-mail-VrecNX.patch
Type: text/x-patch
Size: 28303 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20071115/50410f84/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20071115/50410f84/attachment-0001.pgp