[RFC] making TreeTransform more resistant to unexpected failures / writing problems

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Arbash Meinel wrote:
> Aaron Bentley wrote:
> I also didn't realize that you actually nuke the target file. I thought
> you renamed it out of the way, renamed a new one in place, and then
> deleted the out of the way one.

Not at present, but I would do that as part of implementing rollback.

>>>> This probably has some purity issues for Aaron, as he has always wanted
>>>> to detect all conflicts in the beginning, so that by the time he gets
>>>> around to actually doing the updates, he knows what will happen.
>> I certainly do.  For one thing, it's nice to know what applying a
>> transform will actually do.  We don't say "change this tree in this
>> way...maybe", and I don't know why you'd want to.
> 
> There are times when you cannot do what was
> requested, so rather than failing with an abort, you should instead
> record what you were actually able to accomplish

You assert that, but I don't believe it's empirically true.  If we fail
safe, we don't have to leave messy conflicts around for the user to deal
with.  Then the user can retry the operation when it will succeed.

>>>> However, it works for case conflicts
>> I think that's a really horrible way to handle case conflicts.  Our
>> standard code for detecting duplicate file names should be handling that
>> case.

> I could see doing an os.lstat() for the case when the target already
> exists on disk (since the os will let you know if that collides with an
> existing filename). It would handle both case and unicode normalization.

Not only that, but it improves conflict detection speed tremendously.

> However, when you add 2 files that differ only by case/normalization on
> Linux (which is allowed if a bit broken).

Perhaps we can detect that when placing the files in limbo.

>> Depends on the failure.  If the file disappears before I can remove it,
>> should that be a failure?
> 
> It depends what you mean by failure.

No you first :-P

> Right now on Windows, if you have a
> symlink introduced, or a case-conflict, as it is updating your tree, it
> goes *puke* and leaves you in a very unknown state.

Right.  Rollback will prevent the unknown state for symlinks, and lstat
conflict detection should handle the case conflict.

>> I think if we did that, we'd need to do another round of conflict
>> resolution.  And to do that, we'd need to edit the transform to reflect
>> the operations that did succeed.  Which gets really gross and slow.
> 
> Well, you only really need to do it for the ones that fail.

No, conflict resolution can cause cascading conflicts.

>> I'm a bit confused about whether renames are verboten when a file is
>> held open.
> 
> When you open a file it locks the whole path to that file. So if I open:
> bzr/foo/bar/baz.py
> 
> then the directories 'bzr', 'bzr/foo', ... and the file 'baz.py' cannot
> be renamed or deleted.
> 
> Windows doesn't have the concept of an inode. It identifies things by
> their path. So when you open something, that path cannot change, or it
> cannot find the thing you were talking about.

Thanks.

>> I still like rollback the best.  I could be convinced about
>> Conditional-overwrite-on-TT.apply_insertions, but it's not a complete
>> solution, because it doesn't address cases where we're trying to delete
>> the locked file.  But it could be combined with rollback.
> 
> The only reason I don't like 'rollback' is that you cannot ever get past
> it. So if you only work on Windows, and someone else changes your
> codebase on Linux, you can't merge them until *they* do something about it.

I'm not sure what case you're talking about.  Symlinks would still be an
issue.  Case conflicts/normalization conflicts can be fixed in better
ways than rename-aside.

> If instead it generated a conflict, you could resolve that locally,
> commit, and be on your way.

I'm proposing that case/normalization issues should generate the right
kind of conflicts, and come from the same codepath as other duplicate
name conflicts.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGu1Ty0F+nu1YWqI0RAg3NAJ4jeSbk73FFmzy7hxztXaD9p993jwCdEuor
zDodK8Sc4YePiBns2BwmTSA=
=x6//
-----END PGP SIGNATURE-----