Revfile vs Atomicity & Dumbfs

[Matt Mackall]

On Mon, May 09, 2005 at 06:05:08PM -0500, John A Meinel wrote:
> Matt Mackall wrote:
> >On Mon, May 09, 2005 at 05:01:53PM -0500, John A Meinel wrote:
> >
> >>I've been reading through the bzr docs, (I've gotten through the mailing
> >>lists, and I'm working on the inside documentation).
> >>
> >>Generally, I like what I'm seeing, but I'm concerned about 2 things,
> >>both stemming from the same source.
> >>
> >>Right now, I think you are just keeping a complete copy of each revision
> >>of a file, which you obviously don't want to do over time. The current
> >>suggestion is to use the "revfile" method, which has an append-only
> >>index and an append-only text store.
> >>
> >>The thing is, append-only isn't very transaction safe, it's certainly
> >>better than write anywhere, but new-file only works better with backups,
> >>and atomicity. And unless I'm mistaken, it is easier to add a new file
> >>to a remote connection, than it is to append to an existing one (at
> >>least with sftp/ftp, webdav may be different).
> >
> >
> >Mercurial, which is where the revfile idea comes from, has a very
> >simple transaction and rollback scheme. Not sure how it interacts with
> >network protocols though. Mercurial treats network sources as
> >read-only.
> >
> 
> Well, I got this from here:
> http://www.selenic.com/mercurial/notes.txt
> 
> >Rollback:
> >
> >Rollback is not yet implemented, but will be easy to add. When
> >performing a commit or a merge, we order things so that the changeset
> >entry gets added last. We keep a transaction log of the name of each
> >file and its length prior to the transaction. On abort, we simply
> >truncate each file to its prior length. This is one of the nice
> >properties of the append-only structure of the revlogs.
> 
> Which basically means they use WAL.

I'm not sure what WAL is, but 'they' is 'me'. Oh, write-ahead logging.
Yes, that's a fair description.

Another approach could be called "log ahead logging", where we simply
record all the files we intend to touch ahead of time with a single
write/sync, and to recover, we revisit those files and truncate them
to the last known changeset. Because Mercurial's revlogs now have
uplinks to the changesets, this is easy.

> I don't know that all filesystems support truncate. But it's probably a
> decent enough rollback mechanism.

Yes. Seems to work and is quite simple.

> I'm still not sure how dumbfs support would work, it seems like you
> would have to transfer the entire revfile, rather than being able to append.

I'm afraid I don't even know what dumbfs is. The plan for "push" in
Mercurial (very nearly complete) is to create a blob called a
changegroup containing all of the outstanding changeset deltas and
hand them over to the other side to process. So there's never a need
to do fs-like remote writes.

-- 
Mathematics is the supreme nostalgia of our time.