Rev 3560: (jam, bialix) Cleanups to enable compiling with VC7, and Win98 support in file:///home/pqm/archives/thelove/bzr/%2Btrunk/
Canonical.com Patch Queue Manager
pqm at pqm.ubuntu.com
Thu Jul 17 23:01:21 BST 2008
At file:///home/pqm/archives/thelove/bzr/%2Btrunk/
------------------------------------------------------------
revno: 3560
revision-id:pqm at pqm.ubuntu.com-20080717220108-mjcsi7zryl615o91
parent: pqm at pqm.ubuntu.com-20080717194034-jrnz8chqzli2a3df
parent: john at arbash-meinel.com-20080717212849-t9cbb79nbi8c9jiy
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Thu 2008-07-17 23:01:08 +0100
message:
(jam, bialix) Cleanups to enable compiling with VC7, and Win98 support
modified:
bzrlib/_walkdirs_win32.pyx _walkdirs_win32.pyx-20080716220454-kweh3tgxez5dvw2l-2
bzrlib/osutils.py osutils.py-20050309040759-eeaff12fbf77ac86
bzrlib/tests/test_osutils.py test_osutils.py-20051201224856-e48ee24c12182989
setup.py setup.py-20050314065409-02f8a0a6e3f9bc70
------------------------------------------------------------
revno: 3557.2.9
revision-id:john at arbash-meinel.com-20080717212849-t9cbb79nbi8c9jiy
parent: john at arbash-meinel.com-20080717202345-rq3rl7wrfndbxh36
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 16:28:49 -0500
message:
Alexander noticed I typed macro instead of macros.
modified:
setup.py setup.py-20050314065409-02f8a0a6e3f9bc70
------------------------------------------------------------
revno: 3557.2.8
revision-id:john at arbash-meinel.com-20080717202345-rq3rl7wrfndbxh36
parent: john at arbash-meinel.com-20080717202107-f1jtymylhqnsr910
author: Alexander Belchenko <bialix at ukr.net>
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 15:23:45 -0500
message:
We don't need all the extra cdef statments in a cdef extern block
modified:
bzrlib/_walkdirs_win32.pyx _walkdirs_win32.pyx-20080716220454-kweh3tgxez5dvw2l-2
------------------------------------------------------------
revno: 3557.2.7
revision-id:john at arbash-meinel.com-20080717202107-f1jtymylhqnsr910
parent: john at arbash-meinel.com-20080717201643-qrbe6thvki1izwig
author: Alexander Belchenko <bialix at ukr.net>
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 15:21:07 -0500
message:
Pass extra arguments for extensions, and supply WIN32 macro for pyrex.
modified:
setup.py setup.py-20050314065409-02f8a0a6e3f9bc70
------------------------------------------------------------
revno: 3557.2.6
revision-id:john at arbash-meinel.com-20080717201643-qrbe6thvki1izwig
parent: john at arbash-meinel.com-20080717152900-0qm7pf511hjgutzy
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 15:16:43 -0500
message:
Switch from os.name to bzrlib.win32utils.winver.
os.name is always 'nt' for any Windows platform... :(
modified:
bzrlib/osutils.py osutils.py-20050309040759-eeaff12fbf77ac86
bzrlib/tests/test_osutils.py test_osutils.py-20051201224856-e48ee24c12182989
------------------------------------------------------------
revno: 3557.2.5
revision-id:john at arbash-meinel.com-20080717152900-0qm7pf511hjgutzy
parent: john at arbash-meinel.com-20080717152134-8xz1wvbu47mhc1oe
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 10:29:00 -0500
message:
Test that the empty-directory logic for all _walkdirs implementations is correct.
Also, I was forgetting to actually build the extension between tests.
Fix a couple small typos that crept in.
modified:
bzrlib/_walkdirs_win32.pyx _walkdirs_win32.pyx-20080716220454-kweh3tgxez5dvw2l-2
bzrlib/tests/test_osutils.py test_osutils.py-20051201224856-e48ee24c12182989
------------------------------------------------------------
revno: 3557.2.4
revision-id:john at arbash-meinel.com-20080717152134-8xz1wvbu47mhc1oe
parent: john at arbash-meinel.com-20080717151458-1qou6e4hkyr309f7
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 10:21:34 -0500
message:
Cleanup the tests a bit, and add a test that we downgrade if os.name isn't 'nt'
modified:
bzrlib/osutils.py osutils.py-20050309040759-eeaff12fbf77ac86
bzrlib/tests/test_osutils.py test_osutils.py-20051201224856-e48ee24c12182989
------------------------------------------------------------
revno: 3557.2.3
revision-id:john at arbash-meinel.com-20080717151458-1qou6e4hkyr309f7
parent: john at arbash-meinel.com-20080717151332-h4hwscv2jom8898r
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 10:14:58 -0500
message:
Change the logic for selecting a real _walkdirs_utf8 implementation,
and put it under test, so that we know it is actively being used.
modified:
bzrlib/osutils.py osutils.py-20050309040759-eeaff12fbf77ac86
bzrlib/tests/test_osutils.py test_osutils.py-20051201224856-e48ee24c12182989
------------------------------------------------------------
revno: 3557.2.2
revision-id:john at arbash-meinel.com-20080717151332-h4hwscv2jom8898r
parent: john at arbash-meinel.com-20080717144123-0ne8ig2vm9oehnn7
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 10:13:32 -0500
message:
Include better documentation about what is going on. (suggested by Bialix)
modified:
bzrlib/_walkdirs_win32.pyx _walkdirs_win32.pyx-20080716220454-kweh3tgxez5dvw2l-2
------------------------------------------------------------
revno: 3557.2.1
revision-id:john at arbash-meinel.com-20080717144123-0ne8ig2vm9oehnn7
parent: pqm at pqm.ubuntu.com-20080717132712-1zbt1asfsuslh1v9
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: win32_find_files
timestamp: Thu 2008-07-17 09:41:23 -0500
message:
Don't use ^=, as it isn't supported with older versions of pyrex.
modified:
bzrlib/_walkdirs_win32.pyx _walkdirs_win32.pyx-20080716220454-kweh3tgxez5dvw2l-2
=== modified file 'bzrlib/_walkdirs_win32.pyx'
--- a/bzrlib/_walkdirs_win32.pyx 2008-07-17 12:21:19 +0000
+++ b/bzrlib/_walkdirs_win32.pyx 2008-07-17 20:23:45 +0000
@@ -18,18 +18,18 @@
cdef extern from "_walkdirs_win32.h":
- cdef struct _HANDLE:
+ struct _HANDLE:
pass
ctypedef _HANDLE *HANDLE
ctypedef unsigned long DWORD
ctypedef long long __int64
ctypedef unsigned short WCHAR
- cdef struct _FILETIME:
+ struct _FILETIME:
DWORD dwHighDateTime
DWORD dwLowDateTime
ctypedef _FILETIME FILETIME
- cdef struct _WIN32_FIND_DATAW:
+ struct _WIN32_FIND_DATAW:
DWORD dwFileAttributes
FILETIME ftCreationTime
FILETIME ftLastAccessTime
@@ -45,16 +45,16 @@
# which fails due to 'incomplete type'
ctypedef _WIN32_FIND_DATAW WIN32_FIND_DATAW
- cdef HANDLE INVALID_HANDLE_VALUE
- cdef HANDLE FindFirstFileW(WCHAR *path, WIN32_FIND_DATAW *data)
- cdef int FindNextFileW(HANDLE search, WIN32_FIND_DATAW *data)
- cdef int FindClose(HANDLE search)
-
- cdef DWORD FILE_ATTRIBUTE_READONLY
- cdef DWORD FILE_ATTRIBUTE_DIRECTORY
- cdef int ERROR_NO_MORE_FILES
-
- cdef int GetLastError()
+ HANDLE INVALID_HANDLE_VALUE
+ HANDLE FindFirstFileW(WCHAR *path, WIN32_FIND_DATAW *data)
+ int FindNextFileW(HANDLE search, WIN32_FIND_DATAW *data)
+ int FindClose(HANDLE search)
+
+ DWORD FILE_ATTRIBUTE_READONLY
+ DWORD FILE_ATTRIBUTE_DIRECTORY
+ int ERROR_NO_MORE_FILES
+
+ int GetLastError()
# Wide character functions
DWORD wcslen(WCHAR *)
@@ -107,10 +107,10 @@
mode_bits = 0100666 # writeable file, the most common
if data.dwFileAttributes & FILE_ATTRIBUTE_READONLY == FILE_ATTRIBUTE_READONLY:
- mode_bits ^= 0222 # remove the write bits
+ mode_bits = mode_bits ^ 0222 # remove the write bits
if data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY == FILE_ATTRIBUTE_DIRECTORY:
# Remove the FILE bit, set the DIR bit, and set the EXEC bits
- mode_bits ^= 0140111
+ mode_bits = mode_bits ^ 0140111
return mode_bits
@@ -196,6 +196,16 @@
return statvalue
def _get_files_in(self, directory, relprefix):
+ """Return the dirblock for all files in the given directory.
+
+ :param directory: A path that can directly access the files on disk.
+ Should be a Unicode object.
+ :param relprefix: A psuedo path for these files (as inherited from the
+ original 'prefix=XXX' when instantiating this class.)
+ It should be a UTF-8 string.
+ :return: A dirblock for all the files of the form
+ [(utf8_relpath, utf8_fname, kind, _Win32Stat, unicode_abspath)]
+ """
cdef WIN32_FIND_DATAW search_data
cdef HANDLE hFindFile
cdef int last_err
@@ -221,13 +231,11 @@
continue
name_unicode = _get_name(&search_data)
name_utf8 = PyUnicode_AsUTF8String(name_unicode)
- relpath = relprefix + name_utf8
- abspath = directory + name_unicode
PyList_Append(dirblock,
- (relpath, name_utf8,
+ (relprefix + name_utf8, name_utf8,
self._get_kind(&search_data),
self._get_stat_value(&search_data),
- abspath))
+ directory + name_unicode))
result = FindNextFileW(hFindFile, &search_data)
# FindNextFileW sets GetLastError() == ERROR_NO_MORE_FILES when it
@@ -240,7 +248,9 @@
result = FindClose(hFindFile)
if result == 0:
last_err = GetLastError()
- pass
+ # TODO: We should probably raise an exception if FindClose
+ # returns an error, however, I don't want to supress an
+ # earlier Exception, so for now, I'm ignoring this
return dirblock
cdef _update_pending(self):
=== modified file 'bzrlib/osutils.py'
--- a/bzrlib/osutils.py 2008-07-17 12:55:11 +0000
+++ b/bzrlib/osutils.py 2008-07-17 20:16:43 +0000
@@ -1179,6 +1179,8 @@
pending.extend(d for d in reversed(dirblock) if d[2] == _directory)
+_real_walkdirs_utf8 = None
+
def _walkdirs_utf8(top, prefix=""):
"""Yield data about all the directories in a tree.
@@ -1193,18 +1195,27 @@
path-from-top might be unicode or utf8, but it is the correct path to
pass to os functions to affect the file in question. (such as os.lstat)
"""
- fs_encoding = _fs_enc.upper()
- if sys.platform == 'win32':
- try:
- from bzrlib._walkdirs_win32 import _walkdirs_utf8_win32_find_file
- except ImportError:
- return _walkdirs_unicode_to_utf8(top, prefix=prefix)
+ global _real_walkdirs_utf8
+ if _real_walkdirs_utf8 is None:
+ fs_encoding = _fs_enc.upper()
+ if win32utils.winver == 'Windows NT':
+ # Win98 doesn't have unicode apis like FindFirstFileW
+ # TODO: We possibly could support Win98 by falling back to the
+ # original FindFirstFile, and using TCHAR instead of WCHAR,
+ # but that gets a bit tricky, and requires custom compiling
+ # for win98 anyway.
+ try:
+ from bzrlib._walkdirs_win32 import _walkdirs_utf8_win32_find_file
+ except ImportError:
+ _real_walkdirs_utf8 = _walkdirs_unicode_to_utf8
+ else:
+ _real_walkdirs_utf8 = _walkdirs_utf8_win32_find_file
+ elif fs_encoding not in ('UTF-8', 'US-ASCII', 'ANSI_X3.4-1968'):
+ # ANSI_X3.4-1968 is a form of ASCII
+ _real_walkdirs_utf8 = _walkdirs_unicode_to_utf8
else:
- return _walkdirs_utf8_win32_find_file(top, prefix=prefix)
- if fs_encoding not in ('UTF-8', 'US-ASCII', 'ANSI_X3.4-1968'): # ascii
- return _walkdirs_unicode_to_utf8(top, prefix=prefix)
- else:
- return _walkdirs_fs_utf8(top, prefix=prefix)
+ _real_walkdirs_utf8 = _walkdirs_fs_utf8
+ return _real_walkdirs_utf8(top, prefix=prefix)
def _walkdirs_fs_utf8(top, prefix=""):
=== modified file 'bzrlib/tests/test_osutils.py'
--- a/bzrlib/tests/test_osutils.py 2008-07-17 12:21:19 +0000
+++ b/bzrlib/tests/test_osutils.py 2008-07-17 20:16:43 +0000
@@ -822,6 +822,80 @@
new_dirblock.append((info[0], info[1], info[2], info[4]))
dirblock[:] = new_dirblock
+ def test__walkdirs_utf8_selection(self):
+ # Just trigger the function once, to make sure it has selected a real
+ # implementation.
+ list(osutils._walkdirs_utf8('.'))
+ if WalkdirsWin32Feature.available():
+ # If the compiled form is available, make sure it is used
+ from bzrlib._walkdirs_win32 import _walkdirs_utf8_win32_find_file
+ self.assertIs(_walkdirs_utf8_win32_find_file,
+ osutils._real_walkdirs_utf8)
+ elif sys.platform == 'win32':
+ self.assertIs(osutils._walkdirs_unicode_to_utf8,
+ osutils._real_walkdirs_utf8)
+ elif osutils._fs_enc.upper() in ('UTF-8', 'ASCII', 'ANSI_X3.4-1968'): # ascii
+ self.assertIs(osutils._walkdirs_fs_utf8,
+ osutils._real_walkdirs_utf8)
+ else:
+ self.assertIs(osutils._walkdirs_unicode_to_utf8,
+ osutils._real_walkdirs_utf8)
+
+ def _save_platform_info(self):
+ cur_winver = win32utils.winver
+ cur_fs_enc = osutils._fs_enc
+ cur_real_walkdirs_utf8 = osutils._real_walkdirs_utf8
+ def restore():
+ win32utils.winver = cur_winver
+ osutils._fs_enc = cur_fs_enc
+ osutils._real_walkdirs_utf8 = cur_real_walkdirs_utf8
+ self.addCleanup(restore)
+
+ def assertWalkdirsUtf8Is(self, expected):
+ """Assert the right implementation for _walkdirs_utf8 is chosen."""
+ # Force it to redetect
+ osutils._real_walkdirs_utf8 = None
+ # Nothing to list, but should still trigger the selection logic
+ self.assertEqual([(('', '.'), [])], list(osutils._walkdirs_utf8('.')))
+ self.assertIs(expected, osutils._real_walkdirs_utf8)
+
+ def test_force_walkdirs_utf8_fs_utf8(self):
+ self._save_platform_info()
+ win32utils.winver = None # Avoid the win32 detection code
+ osutils._fs_enc = 'UTF-8'
+ self.assertWalkdirsUtf8Is(osutils._walkdirs_fs_utf8)
+
+ def test_force_walkdirs_utf8_fs_ascii(self):
+ self._save_platform_info()
+ win32utils.winver = None # Avoid the win32 detection code
+ osutils._fs_enc = 'US-ASCII'
+ self.assertWalkdirsUtf8Is(osutils._walkdirs_fs_utf8)
+
+ def test_force_walkdirs_utf8_fs_ANSI(self):
+ self._save_platform_info()
+ win32utils.winver = None # Avoid the win32 detection code
+ osutils._fs_enc = 'ANSI_X3.4-1968'
+ self.assertWalkdirsUtf8Is(osutils._walkdirs_fs_utf8)
+
+ def test_force_walkdirs_utf8_fs_latin1(self):
+ self._save_platform_info()
+ win32utils.winver = None # Avoid the win32 detection code
+ osutils._fs_enc = 'latin1'
+ self.assertWalkdirsUtf8Is(osutils._walkdirs_unicode_to_utf8)
+
+ def test_force_walkdirs_utf8_nt(self):
+ self.requireFeature(WalkdirsWin32Feature)
+ self._save_platform_info()
+ win32utils.winver = 'Windows NT'
+ from bzrlib._walkdirs_win32 import _walkdirs_utf8_win32_find_file
+ self.assertWalkdirsUtf8Is(_walkdirs_utf8_win32_find_file)
+
+ def test_force_walkdirs_utf8_nt(self):
+ self.requireFeature(WalkdirsWin32Feature)
+ self._save_platform_info()
+ win32utils.winver = 'Windows 98'
+ self.assertWalkdirsUtf8Is(osutils._walkdirs_unicode_to_utf8)
+
def test_unicode_walkdirs(self):
"""Walkdirs should always return unicode paths."""
name0 = u'0file-\xb6'
=== modified file 'setup.py'
--- a/setup.py 2008-07-17 12:55:11 +0000
+++ b/setup.py 2008-07-17 21:28:49 +0000
@@ -216,18 +216,22 @@
pyrex_name = path + '.pyx'
c_name = path + '.c'
if have_pyrex:
- ext_modules.append(Extension(module_name, [pyrex_name]))
+ ext_modules.append(Extension(module_name, [pyrex_name], **kwargs))
else:
if not os.path.isfile(c_name):
unavailable_files.append(c_name)
else:
- ext_modules.append(Extension(module_name, [c_name]))
+ ext_modules.append(Extension(module_name, [c_name], **kwargs))
add_pyrex_extension('bzrlib._dirstate_helpers_c')
add_pyrex_extension('bzrlib._knit_load_data_c')
if sys.platform == 'win32':
- add_pyrex_extension('bzrlib._walkdirs_win32')
+ # pyrex uses the macro WIN32 to detect the platform, even though it should
+ # be using something like _WIN32 or MS_WINDOWS, oh well, we can give it the
+ # right value.
+ add_pyrex_extension('bzrlib._walkdirs_win32',
+ define_macros=[('WIN32', None)])
ext_modules.append(Extension('bzrlib._patiencediff_c', ['bzrlib/_patiencediff_c.c']))
More information about the bazaar-commits
mailing list