[RFC] TreeTransform fooled by os.rename on case insensitive filesystems

[Vincent Ladeuil]

>>>>> "aaron" == Aaron Bentley <aaron at aaronbentley.com> writes:

    aaron> John Arbash Meinel wrote:
    >> Vincent Ladeuil wrote:
    >>> I think this indicates a deeper problem than just a slight
    >>> mismatch between Windows and CaseInsensitiveFilesystemFeature.

    >>> Any idea about where and how that should be fixed ?

    aaron> It would help if you specified what the deeper problem is.

Sorry, I couldn't find a good description :-/ Let's try again.

Roughly speaking, in that test, we rely on _FileMover via
TreeTransform to detect a name clash when the file system is case
insensitive.

The problem is that we trust a case insensitive fs to raise
FileExists and that doesn't work on HFS+.

I think that this problem should be detected *before* we use the
_FileMover, may be in tt.new_file() ?

What I pointed out is that TreeTransform relies on _FileMover and
its rollback facility to ensure no data loss occurs, this is not
the case here.

    aaron> No, I don't think we should be worried TreeTransform
    aaron> renaming a file over top of another file.  We already
    aaron> consider that a conflict.

You mean that, in the following:

                tt.new_file('foo', tt.root, 'bar')
                tt.new_file('Foo', tt.root, 'spam')
                # Lie to tt that we've already resolved all conflicts.
                tt.apply(no_conflicts=True)

we will never lie like that ?

    aaron> There is a small window between conflict detection and
    aaron> transform application, where a file can be put into
    aaron> the way.  There is no way to avoid this, you can just
    aaron> change the size of the window.

    aaron> This test is testing extreme conditions.  I don't
    aaron> consider it a problem if TreeTransform fails under
    aaron> those conditions.

So you agree with John about making that test a XFAIL on OSX ?

The problem I was referring to is that
CaseInsensitiveFilesystemFeature does not deserve its name then
and neither is the '_case_sensitive_target' attribute of
TreeTransformBase. That just add to confusion.

    >> One problem becomes that in order to handle this sort of
    >> thing, we then need to "Look Before You Leap" for all the
    >> files you are about to add

    aaron> We already look before we leap; it's called conflict
    aaron> detection.  We don't need to replicate it.  Conflict
    aaron> detection and resolution is always done immediately
    aaron> before renaming files.

Can you kindly show me where it's done in that test or is that
conflict detection tricked by the no_conflicts=True ?

If that's the case, should we really keep that test ?

    >> What I would also *like* for Mac, is a way to get back the
    >> final filename, rather than just a name we can access it
    >> with. 

readdir(2) or ls(1), they don't lie if you don't force them to.

    >> And I don't know how to do that. (Very weird result in
    >> testing... It seems that NTFS under Linux *is*
    >> case-sensitive.)

Welcome to the wonders of mounted file systems :-D

    aaron> Interesting.  Of course, NTFS can also be
    aaron> case-sensitive under Windows, if you know the right
    aaron> APIs.

    >> So... in the interest of moving things forward, I'm fine
    >> with turning this into a 'knownFailure()' on Mac, until we
    >> can figure out a solution.

    aaron> Just change it into a file type that we can guarantee
    aaron> won't allow rename-on-top.  i.e. a directory.

EPARSE_ERROR

You mean forcing TT to create a directory for every file ?

        Vincent