Compressing weaved revisions?

[John A Meinel]

James Blackwell wrote:
> On Fri, Sep 30, 2005 at 10:04:41AM -0400, Aaron Bentley wrote:
> 
>>For trees with, e.g. 500 revisions, the revision storage may actually be
>>larger than tree storage.
> 
> 
> I converted the Bazaar-NG tree to newformat. I got a ratio of about 7:1,
> which is much better than the old 29:1, but not as good as Mercurial's
> 2.5:1.
> 

Are you using --apparent, and are you considering just revision-store or 
all of .bzr?

These are the numbers I get:

$ du -ksh .bzr
15M     .bzr
$ du -ksh --apparent .bzr
7.2M    .bzr
$ du -ksh .bzr/revision-store/
7.5M    .bzr/revision-store/
$ du -ksh --apparent .bzr/revision-store/
973K    .bzr/revision-store/

$ du -kshc *
...
2.9M    total
$ du -kshc --apparent *
2.2M    total

So .bzr vs working is 5:1 (3.2:1 apparent)
.bzr/everything-else vs .bzr/revision-store is 1:1 (7:1 apparent)

With my revstore2sql plugin, if I trim out all the inventory stuff, I 
can get a revisions.sqlite down to 561K, it gets compression mostly by 
not duplicating the revision_id everywhere (switching to just a number).
It loses some by having indexes, but in theory that would make access 
very fast.

I don't know if we would want to use an sqlite store for everything, 
since there isn't a way to remotely access it, or download only part of 
it. But it is small, and you could download it locally and then upload it.

If I pack all of the inventory into the sqlite db, then the size goes up 
to 9.4M, but 8.1M of that is because I don't do any delta compression of 
inventories (so each revision has a complete list of inventory entries). 
(though this is better than the old 24M version).
But, I can get any inventory out of it in about 0.05s, whereas on this 
machine branch.get_inventory() takes 1.2s. (That is with bzrlib already 
loaded)

John
=:->

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20051006/faf8762f/attachment.pgp