please check out weave-format branch

John A Meinel john at arbash-meinel.com
Fri Sep 23 18:16:48 BST 2005


Martin Pool wrote:
> On 23/09/05, John Arbash Meinel <john at arbash-meinel.com> wrote:
>
>
>>5) The new weave format means that all uploads to a remote store are
>>going to have to be done atomically rather than using just append. Have
>>you looked into a different weave format, which might allow append-only?
>
>
> Yes, Aaron has (also) designed one, of which he wrote in August:
>
> ----------
> It's also possible to do append-only weaves.  Trivially, you can just
> store the weave itself in a revfile.  I have also done some work on a
> natively append-only weave format; you can see my work-so-far here:

Well, I would guess that putting a weave in a revfile is a lot of extra
overhead, since now you are diffing a weave to create the delta compression.

>
> http://www.aaronbentley.com/weavediff

I read over this, and I'm a little confused by the syntax he used. But I
think I generally understand it. I wasn't trying to create a minimal
read form, since the current weave form is read entirely. It would be
nice to have a seekable form, but that requires a separate index to be
generated. And if you want something like that, my form would let you
put all the annotation data into the second file, and the first just
becomes a pool of lines.

>
> So I'm comfortable going with weaves for now since
> 1. latency trumps throughput most of the time, and weaves are only
> somewhat larger than the original files
> 2. we can do append-only weaves later, if we think it's valuable.
> ----------
>
> I agree that the robustness of an append-only format is nice, but I
> want to get some experience with this basic format while we work on a
> more advanced one.

Well, I feel like the current weave format is more complicated than it
has to be. the {} and [] syntax make it tricky to figure out what you
are looking at. Especially since there is no nesting requirements so you
can get:

{ 3
. bla
. bla
[ 4
. de bla
} 3
. something
] 4

And I also saw quite a few cases in the weaves where the closing
bracket/brace didn't have a number.

The problem I have with switching to the current weave format, is that
it feels like I should be making snapshots, in case something gets
messed up. (Think about svn with the berkley db backend, where you copy
it periodically).
But why am I using a SCM if I have to backup its meta-data periodically.
 Coming from Arch, the SCM *was* the backup method. Naturally you should
keep a mirror for redundancy, but mirroring a corrupted branch will give
you a corrupted branch.

Now maybe you feel like the current weavefile format is obvious enough
that it is difficult to corrupt. From my experience of trying to look
through the .weave file, I don't quite feel the same.

John
=:->

>
> --
> Martin
>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20050923/1d76719f/attachment.pgp 


More information about the bazaar mailing list