[RFC] allow fulltext to be more dynamic

[John Arbash Meinel]

bzr currently always caches a fulltext after 26 deltas. It turns out
that this is actually pretty good for bzr.dev. The size of the inventory
versus the average size of the change is pretty close to that sweet spot.

However, as we plan to scale to larger project sizes, it becomes more of
an overhead. The attached patch cleans up the algorithm that determines
whether to use a delta, and makes it look back as far as 200 deltas. And
if it finds a fulltext it compares the total fulltext size, to the
cumulative size of all the deltas. If the fulltext is smaller, we get
another fulltext. This is a little skewed since fulltexts are usually
growing in size, which means we should be comparing against the current
fulltext, not the old one. But I'm comparing the sizes from the knit
index, which means it is compressed, etc. And so doesn't directly compare.

In testing with my local bzr repository, I save about 1MB (15MB versus
16MB) for inventory.knit. I haven't really analyzed the benchmarks for
it, but I have the feeling the difference is small enough that the
performance would also be small.

I also tested it against a subtree of the mozilla tree. And there I
found a change of 23MB versus 26MB for all of the repository.
(inventory.knit was 2.9M versus 3.5M).

Anyway, we really want to address our large-tree performance in other
ways (breaking up the inventory by dir, using recursive 'last-changed'
fields to avoid inspection of subdirs, etc). But this is a fairly simple
fix, that is completely compatible with current versions of bzr.

John
=:->
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dynamic-fulltext-cache.patch
Type: text/x-patch
Size: 4890 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20061121/a0ca889c/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20061121/a0ca889c/attachment.pgp