Solving the commit-editor-locks-stuff-up problem.

[Martin Pool]

2009/3/24 Robert Collins <robert.collins at canonical.com>:
>> > So the main problem isn't write integrity... thats easy: write the new
>> > dirstate to a tempfile and move it over before we release the tree lock.
>>
>> If we release the lock (i.e. both os lock and directory lock) before we
>> update the parent trees, that's potentially problematic, because another
>> client could *write* and we would then need to abort or do something clever.
>
> Right now, running 'bzr st' will write to the dirstate, even though it
> doesn't take a lockdir lock on the tree; its the stat cache update.
>
>> > The problem is, and always has been reader integrity *and* reader
>> > lock-outs.
>>
>> Right.  I'm kinda surprised we're not doing atomic updates of the
>> dirstate, but it surely wouldn't be hard to add.
>
> I think its straight forward - write a new dirstate file,and use a
> pointer to the file rather than assuming the name is 'dirstate'.

This does totally rule out doing incremental writes: all updates to
the dirstate will do O(n_files*n_trees) work.  If you do 'bzr st
README' and that file has changed and it causes it to rewrite the
whole file it's potentially a lot slower.

However, it looks like there's no getting away from this if you want
incremental reading and if you don't want to split it up into multiple
files or some kind of journal structure.  So it's probably worth at
least trying it, and maybe turning off dirstate updates from inside
logical read operations, at least small ones.

>> > A reader on windows will prevent the commit completing until
>> > it has closed the file. Partial parsing is written and implemented, even
>> > though its not switched on; I don't think we need to discard that when
>> > its very easy to write new files, and nothing needs to be keeping open a
>> > pointer file.
>>
>> I don't understand.
>
> partial parsing - readers read the file incrementally as they need to,
> and on windows keeping a file handle open locks the file implicitly.

The problem arises on Windows because you cannot (normally/reliably)
rename over the top of an open file allowing old clients to keep
reading from the old one.  (This also can happen on network
filesystems.)  Either the rename will fail or worse the client will
see some parts of the new file.

-- 
Martin <http://launchpad.net/~mbp/>