branch locking mk2.

[Robey Pointer]

On 15 Feb 2006, at 20:09, John A Meinel wrote:

> Robey Pointer wrote:
>>
>> The name "fancy_rename" implies (to me) something like rename, but  
>> with
>> added magic.  It sounds like it's really just an attempt to implement
>> "atomic_rename" of files for Transports that don't have it.   
>> Having it
>> in osutils further implies that it's related to the local OS.
>>
>> What if we moved it into Transport.__init__.py and called it
>> "atomic_rename", to emphasize that it's really a base  
>> implementation for
>> Transports that don't have an atomic rename?  Then  
>> Transport.rename (or
>> .replace) could promise to always be atomic.

By the way, knits aside, did anyone have any comments on this?


>> Another issue is that if file-renaming isn't atomic on some  
>> platforms,
>> and we're relying on atomic behavior, this seems... bad.  I don't  
>> have
>> an answer.  But it troubles me.  Over SFTP, you can't even guarantee
>> that the race is short -- we may get disconnected from the server  
>> in the
>> middle of the atomic_rename, and now there's no weave file.  Or  
>> the user
>> may just have hit ^C.  Scary.  Will the append-only knit stuff  
>> make this
>> a non-issue?
>
> Append only weaves would make it much less of an issue, since we would
> append to the files that exist, rather than replacing them with new  
> ones.
>
> There will probably be some control files that are still atomically
> replaced, like revision-history (as long as that exists).
> Also, I don't know how we are trying to keep knit index files intact.
> I'm guessing we just expect that adding 1 line will be an atomic  
> action,
> as long as we add less than X number of bytes.
> I can think of some things we could do (like adding a checksum +
> line-length at the beginning of each line, along with a line delimiter
> before the line (a simple newline could do)).
> Then we can throw out index lines that don't match their checksum. And
> the delimiter makes sure that we know where the entries start & stop.

Seems like we rely on the atomic_replace, so trying to think of some  
way to make it work correctly...

With all transports using the same locking, we'll at least know that  
only one person can be updating metadata at a time.  So maybe  
atomic_rename could always use the same temporary, like  
"<filename>.old".  The worst interruption that could happen then is:

<updater #1>
open('important.weave.temp-foobar-90210', 'w').write(stuff)
rename('important.weave', 'important.weave.old')
---ACK! disconnected

<reader>
open('important.weave', 'r')  =>  error
open('important.weave.old', 'r')  =>  ok, just go with that

<updater #2>
open('important.weave', 'r')  =>  error
open('important.weave.old', 'r')  =>  ok, so fix:
rename('important.weave.old', 'important.weave')


This would require a bit more work when reading files that may be  
atomically replaced.  We'd have to check for failed updates and move  
the old copy back.

robey