Revfile vs Atomicity & Dumbfs

[John A Meinel]

Martin Pool wrote:
> On  9 May 2005, John A Meinel <john at arbash-meinel.com> wrote:
>
>
>>Although, I think the clone-and-replace method gets around this, since
>>it already has to break hardlinks, but would preserve ones that haven't
>>changed.
>
>
> We can certainly do copy-append if we don't trust that just appending
> to a revfile will be safe, but I don't see how that can ever occur.

In your design docs you mention that you want hardlink trees to "just
work", which means you can never just append. Which is in the previous
quote. copy-append works fine, but it has O(num changes) behavior,
rather than O(1) for append only.

>
> I don't see why you would ever want to hardlink-and-replace the whole
> .bzr directory; it's enough to write in new objects bottom-up because
> they only become visible when they're only referenced by something
> else.  Making the new revision the head makes the whole transaction
> visible.
>
> If the process is interrupted while writing to a revfile the following
> conditions can occur:
>
> - text file contains data not referenced by the index - harmless, can
>   be vacuumed up later if you really want
>
> - index line is written
>
> - index line is not written

It seems that with a write-ahead-log you could get away without the
later vacuum, because you would effectively vacuum as you go. Yes you
could vacuum later, but that would require lots of locking to make sure
the index gets rewritten inbetween the time you are rewriting the revlog
file. If you always truncate bogus data, then you don't ever have the
locking issue.

>
> If the whole machine crashes or suffer power failure then there are
> more possibilities:
>
> - existing data in the files is corrupted; get a better filesystem; or
>   use copy-then-append mode

well, with plain append, I *think* you are still safe with a power
failure. Unless a filesystem does something weird as you hit block
boundaries, but I would guess it would just append if you are appending.

>
> - the index line is corrupted; this can be detected and this
>   particular revision of the file is lost but nothing else; that last
>   revision may need to be removed.

How is it detected? Do you have a checksum? Or is it just if it points
to something bogus?

>
> - the text file is corrupted; likewise
Sure.
>
>

I believe I understand your method of atomicity to be that appends don't
matter until you write the final file, because they will just be
ignored. I think you could be more proactive with a WAL and allow
automatic corrections.
I'm still not sure about over-the-network atomicity, but I do think it
works for local atomicity.

John
=:->
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 251 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20050509/f5c3a102/attachment.pgp