[RFC] making TreeTransform more resistant to unexpected failures / writing problems

[Martin Pool]

On 8/9/07, John Arbash Meinel <john at arbash-meinel.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> John Whitley wrote:
> > Aaron Bentley wrote:
> >>> on files that
> >>> don't deserve version control
> >>
> >> So you are saying that Visual Studio's Intellisense doesn't read actual
> >> source files ever?
> >
> > FWIW, I don't think I've ever had a random conflict with Visual Studio
> > and source control, at least with recent versions (VS7/VS8).  By "I", I
> > really mean an entire full-time dev team of (average) 10 developers in
> > the past two years, over two SCM systems.  Though I will note that at
> > least CVSNT has built-in retry behavior.  IIRC, I've only ever seen that
> > triggered for versioned executables that were running during an update,
> > or a smattering of similar cases, none of which were attributable to
> > Visual Studio.  This does mean only versioning actual "source" files,
> > not working files (e.g. Intellisense's .ncb files for Visual Studio).
>
> Note also that CVS versions a single file at a time. So if it got
> kicked, you can just update that file again later.
> Bazaar versions the whole tree at a time. We haven't put in a way to
> track that some files are at a different revision than the rest of the
> tree. (And we *really* don't want to put that in, because it leaves you
> open to a host of bad things that SVN/CVS can get you into. Like "it
> works perfectly here after I committed", but you can't check out that
> same tree because everything is at a different version). If you had a
> file out of date, you could 'bzr revert foo', but that would lose any
> local changes (versus bzr 'update foo')
>
> Anyway, what I would actually like to see is for TreeTransform to try to
> rename-into-place and if that fails, it can rename it 'to the side' and
> mark the file as conflicted.
>
> This probably has some purity issues for Aaron, as he has always wanted
> to detect all conflicts in the beginning, so that by the time he gets
> around to actually doing the updates, he knows what will happen.
>
> However, it works for case conflicts, as well as locked files, or files
> showing up while you are renaming into place. (I guess that means it
> also needs to conflict if the '_apply_deletions' step fails).
>
> Ultimately, I would rather not rollback the entire transaction, instead,
> just mark a few more files as conflicted, and put at least .THIS files
> around.

I tend to agree that we should just emit a warning about the files
that could not be updated and continue with the rest of the transform.
 We could retry the replacement but doing it immediately after will
probably find that it's still in use.  It might be better to do all
the files that we can, then come back and retry the ones that were in
use.  I'm not sure if that would be easy or feasible to fit into the
tree transform code.

> We could try a couple times, but I think you would want to pause
> inbetween (what is the sense of retrying 10 times in 10ms). And you
> still wouldn't be able to get everything (for example if you are failing
> because of a case conflict).

The other thing to consider is that on Windows, it could be better to
write over the files in place rather than renaming the new file into
place.  I think many programs that hold the file open are not denying
writers, but just refusing to let it be renamed or deleted.  That has
some risk that we'll overwrite the only copy of the file, so we might
like to make a backup first.

-- 
Martin