[MERGE] Faster code for pushing from packs back into knits
John Arbash Meinel
john at arbash-meinel.com
Sat Nov 17 00:27:51 GMT 2007
As I mentioned to Robert earlier the code we have for packs => knits
rebuids the complete history of every text that was modified. Very
painful. The attached code just changes it to add them text by text.
It isn't 100% optimal, because it extracts full texts one by one. However
I'm trying to be more sensitive towards memory consumption. (I think a lot
of our fetch() code buffers far too much in memory.)
It should actually be a bit better about memory consumption since it uses
the LRUSizeCache to cap at 10MB of cached parent texts. (The existing
InterVersionedFile.join() code builds up a new knit into memory, and
caches all parent texts in a dictionary.)
I have to wait for reconcile to finish before I can test this on bzr.dev
branches (my knit branch hasn't been reconciled.) But I did see the time
for a simple change in bzrtools/__init__.py to drop from 7s down to 1.5s
(and 1.3s is just the overhead of a no-op push on this machine).
Also, this adds a bunch more InterRepository implementation tests. I was
surprised to see that knit => knit wasn't being tested, and the knit =>
pack uses a different code path than pack => knit.
It seems that we could probably parameterize the tests a bit better, since
we end up with a whole bunch of ...test_foo(InterKnitRepository) without
showing what the source and target versions are. So it looks like they are
duplicated (at least by test ids).
John
=:->
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/x-patch
Size: 11050 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20071116/96c05327/attachment-0001.bin
More information about the bazaar
mailing list