Rev 5419: (spiv) Merge lp:bzr/2.2, including fixes for #619872, #631350, in file:///home/pqm/archives/thelove/bzr/%2Btrunk/

Canonical.com Patch Queue Manager pqm at pqm.ubuntu.com
Mon Sep 13 08:59:42 BST 2010


At file:///home/pqm/archives/thelove/bzr/%2Btrunk/

------------------------------------------------------------
revno: 5419 [merge]
revision-id: pqm at pqm.ubuntu.com-20100913075939-nh4c1k7rkuxle7zk
parent: pqm at pqm.ubuntu.com-20100910111302-xbneyohu4we926bl
parent: andrew.bennetts at canonical.com-20100913063659-gs1d1xnsdbj59sx6
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Mon 2010-09-13 08:59:39 +0100
message:
  (spiv) Merge lp:bzr/2.2, including fixes for #619872, #631350,
  	#633745.
modified:
  NEWS                           NEWS-20050323055033-4e00b5db738777ff
  bzr                            bzr.py-20050313053754-5485f144c7006fa6
  bzrlib/dirstate.py             dirstate.py-20060728012006-d6mvoihjb3je9peu-1
  bzrlib/errors.py               errors.py-20050309040759-20512168c4e14fbd
  bzrlib/lockdir.py              lockdir.py-20060220222025-98258adf27fbdda3
  bzrlib/tests/test_errors.py    test_errors.py-20060210110251-41aba2deddf936a8
  bzrlib/tests/test_lockdir.py   test_lockdir.py-20060220222025-33d4221569a3d600
  bzrlib/tests/test_rio.py       test_rio.py-20051128032247-dcd1082dfc86d3d3
  bzrlib/transport/ssh.py        ssh.py-20060824042150-0s9787kng6zv1nwq-1
  doc/en/admin-guide/simple-setups.txt simplesetups.txt-20091205144603-lgpl0e0z6lzk2rdw-11
  doc/en/whats-new/whats-new-in-2.2.txt whatsnewin2.2.txt-20100304041442-cj7jdn23zakcw08l-1
=== modified file 'NEWS'
--- a/NEWS	2010-09-10 09:46:15 +0000
+++ b/NEWS	2010-09-13 06:36:59 +0000
@@ -130,6 +130,10 @@
 * Don't print internal object name when print an invalid revision spec
   error.  (Neil Martinsen-Burrell, #598701)
 
+* Don't traceback when a lockdir's ``held/info`` file is corrupt (e.g.
+  contains only NUL bytes).  Instead warn the user, and allow ``bzr
+  break-lock`` to remove it.  (Andrew Bennetts, #619872)
+  
 * ``EPIPE`` can be raised during test server shutdown. This happened on
   gentoo only so far. (Vincent Ladeuil, #627277)
 
@@ -140,6 +144,11 @@
   directory that was a symlink in the previous commit.
   (Martin Pool, #192859)
 
+* Fix ``AttributeError: 'NoneType' object has no attribute 'close'`` in
+  ``_close_ssh_proc`` when using ``bzr+ssh://``.  This was causing
+  connections to pre-1.6 bzr+ssh servers to fail, and causing warnings on
+  stderr in some other circumstances.  (Andrew Bennetts, #633745)
+
 * Fix spurious paramiko warning on hardy by ensuring that ``selftest``
   properly remove its warning filter. (Vincent Ladeuil, #625686)
 
@@ -150,6 +159,11 @@
   full test suite to pass on gentoo.
   (Vincent Ladeuil, #392127)
 
+* Only call ``setlocale`` in the bzr startup script on posix systems. This
+  avoids an issue with the newer windows C runtimes used by Python 2.6 and
+  later which can mangle bytestrings printed to the console.
+  (Martin [gz], #631350)
+
 * `PathNotChild` should not give a traceback.
   (Martin Pool, #98735)
 
@@ -299,7 +313,7 @@
 
 * ``bzr add SYMLINK/FILE`` now works properly when the symlink points to a
   previously-unversioned directory within the tree: the directory is
-  marked versioned too.  
+  marked versioned too.
   (Martin Pool, #192859)
 
 * CommitBuilder now uses the committer instead of _config.username to generate
@@ -308,10 +322,27 @@
 * Cope with Microsoft FTP server that returns reply '250 Directory
   created' when mkdir succeeds.  (Martin Pool, #224373)
 
-* Fix ``AttributeError on parent.children`` when adding a file under a 
+* Don't traceback when a lockdir's ``held/info`` file is corrupt (e.g.
+  contains only NUL bytes).  Instead warn the user, and allow ``bzr
+  break-lock`` to remove it.  (Andrew Bennetts, #619872)
+  
+* Fix ``AttributeError on parent.children`` when adding a file under a
   directory that was a symlink in the previous commit.
   (Martin Pool, #192859)
 
+* Fix ``AttributeError: 'NoneType' object has no attribute 'close'`` in
+  ``_close_ssh_proc`` when using ``bzr+ssh://``.  This was causing
+  connections to pre-1.6 bzr+ssh servers to fail, and causing warnings on
+  stderr in some other circumstances.  (Andrew Bennetts, #633745)
+
+* Only call ``setlocale`` in the bzr startup script on posix systems. This
+  avoids an issue with the newer windows C runtimes used by Python 2.6 and
+  later which can mangle bytestrings printed to the console.
+  (Martin [gz], #631350)
+
+Improvements
+************
+
 Documentation
 *************
 
@@ -1380,6 +1411,10 @@
 * Don't traceback trying to unversion children files of an already
   unversioned directory.  (Vincent Ladeuil, #494221)
 
+* Don't traceback when a lockdir's ``held/info`` file is corrupt (e.g.
+  contains only NUL bytes).  Instead warn the user, and allow ``bzr
+  break-lock`` to remove it.  (Andrew Bennetts, #619872)
+  
 * Fix ``AttributeError on parent.children`` when adding a file under a 
   directory that was a symlink in the previous commit.
   (Martin Pool, #192859)
@@ -1934,6 +1969,10 @@
 * Don't traceback trying to unversion children files of an already
   unversioned directory.  (Vincent Ladeuil, #494221)
 
+* Don't traceback when a lockdir's ``held/info`` file is corrupt (e.g.
+  contains only NUL bytes).  Instead warn the user, and allow ``bzr
+  break-lock`` to remove it.  (Andrew Bennetts, #619872)
+  
 * Fix ``AttributeError on parent.children`` when adding a file under a 
   directory that was a symlink in the previous commit.
   (Martin Pool, #192859)

=== modified file 'bzr'
--- a/bzr	2010-07-13 07:44:02 +0000
+++ b/bzr	2010-09-13 06:36:59 +0000
@@ -75,6 +75,16 @@
 else:
     import locale
 
+if os.name == "posix":
+    try:
+        locale.setlocale(locale.LC_ALL, '')
+    except locale.Error, e:
+        sys.stderr.write('bzr: warning: %s\n'
+            '  bzr could not set the application locale.\n'
+            '  Although this should be no problem for bzr itself, it might\n'
+            '  cause problems with some plugins. To investigate the issue,\n'
+            '  look at the output of the locale(1p) tool.\n' % e)
+
 
 # The python2.6 release includes some libraries that have deprecation warnings
 # against the interpreter - see https://bugs.launchpad.net/bzr/+bug/387139
@@ -86,17 +96,6 @@
     )
 
 
-try:
-    locale.setlocale(locale.LC_ALL, '')
-except locale.Error, e:
-    sys.stderr.write('bzr: warning: %s\n'
-                     '  bzr could not set the application locale.\n'
-                     '  Although this should be no problem for bzr itself,\n'
-                     '  it might cause problems with some plugins.\n'
-                     '  To investigate the issue, look at the output\n'
-                     '  of the locale(1p) tool available on POSIX systems.\n'
-                     % e)
-
 # instruct bzrlib/__init__.py to install lazy_regex
 sys._bzr_lazy_regex = True
 try:

=== modified file 'bzrlib/dirstate.py'
--- a/bzrlib/dirstate.py	2010-08-02 22:07:54 +0000
+++ b/bzrlib/dirstate.py	2010-09-13 06:36:59 +0000
@@ -2876,12 +2876,12 @@
             # converted to relocated.
             if path_utf8 is None:
                 raise AssertionError('no path')
-            existing_keys = id_index[key[2]]
+            existing_keys = id_index.get(key[2], ())
             if key not in existing_keys:
                 raise AssertionError('We found the entry in the blocks, but'
                     ' the key is not in the id_index.'
                     ' key: %s, existing_keys: %s' % (key, existing_keys))
-            for entry_key in id_index[key[2]]:
+            for entry_key in existing_keys:
                 # TODO:PROFILING: It might be faster to just update
                 # rather than checking if we need to, and then overwrite
                 # the one we are located at.

=== modified file 'bzrlib/errors.py'
--- a/bzrlib/errors.py	2010-08-20 09:52:25 +0000
+++ b/bzrlib/errors.py	2010-09-13 06:36:59 +0000
@@ -1072,6 +1072,18 @@
         self.target = target
 
 
+class LockCorrupt(LockError):
+
+    _fmt = ("Lock is apparently held, but corrupted: %(corruption_info)s\n"
+            "Use 'bzr break-lock' to clear it")
+
+    internal_error = False
+
+    def __init__(self, corruption_info, file_data=None):
+        self.corruption_info = corruption_info
+        self.file_data = file_data
+
+
 class LockNotHeld(LockError):
 
     _fmt = "Lock not held: %(lock)s"

=== modified file 'bzrlib/lockdir.py'
--- a/bzrlib/lockdir.py	2010-06-15 09:53:42 +0000
+++ b/bzrlib/lockdir.py	2010-09-13 02:32:44 +0000
@@ -120,6 +120,7 @@
         LockBreakMismatch,
         LockBroken,
         LockContention,
+        LockCorrupt,
         LockFailed,
         LockNotHeld,
         NoSuchFile,
@@ -348,7 +349,13 @@
         it possibly being still active.
         """
         self._check_not_locked()
-        holder_info = self.peek()
+        try:
+            holder_info = self.peek()
+        except LockCorrupt, e:
+            # The lock info is corrupt.
+            if bzrlib.ui.ui_factory.get_boolean("Break (corrupt %r)" % (self,)):
+                self.force_break_corrupt(e.file_data)
+            return
         if holder_info is not None:
             lock_info = '\n'.join(self._format_lock_info(holder_info))
             if bzrlib.ui.ui_factory.get_boolean("Break %s" % lock_info):
@@ -395,6 +402,35 @@
         for hook in self.hooks['lock_broken']:
             hook(result)
 
+    def force_break_corrupt(self, corrupt_info_lines):
+        """Release a lock that has been corrupted.
+        
+        This is very similar to force_break, it except it doesn't assume that
+        self.peek() can work.
+        
+        :param corrupt_info_lines: the lines of the corrupted info file, used
+            to check that the lock hasn't changed between reading the (corrupt)
+            info file and calling force_break_corrupt.
+        """
+        # XXX: this copes with unparseable info files, but what about missing
+        # info files?  Or missing lock dirs?
+        self._check_not_locked()
+        tmpname = '%s/broken.%s.tmp' % (self.path, rand_chars(20))
+        self.transport.rename(self._held_dir, tmpname)
+        # check that we actually broke the right lock, not someone else;
+        # there's a small race window between checking it and doing the
+        # rename.
+        broken_info_path = tmpname + self.__INFO_NAME
+        f = self.transport.get(broken_info_path)
+        broken_lines = f.readlines()
+        if broken_lines != corrupt_info_lines:
+            raise LockBreakMismatch(self, broken_lines, corrupt_info_lines)
+        self.transport.delete(broken_info_path)
+        self.transport.rmdir(tmpname)
+        result = lock.LockResult(self.transport.abspath(self.path))
+        for hook in self.hooks['lock_broken']:
+            hook(result)
+
     def _check_not_locked(self):
         """If the lock is held by this instance, raise an error."""
         if self._lock_held:
@@ -459,7 +495,13 @@
         return s.to_string()
 
     def _parse_info(self, info_bytes):
-        stanza = rio.read_stanza(osutils.split_lines(info_bytes))
+        lines = osutils.split_lines(info_bytes)
+        try:
+            stanza = rio.read_stanza(lines)
+        except ValueError, e:
+            mutter('Corrupt lock info file: %r', lines)
+            raise LockCorrupt("could not parse lock info file: " + str(e),
+                              lines)
         if stanza is None:
             # see bug 185013; we fairly often end up with the info file being
             # empty after an interruption; we could log a message here but

=== modified file 'bzrlib/tests/test_errors.py'
--- a/bzrlib/tests/test_errors.py	2010-07-14 14:22:40 +0000
+++ b/bzrlib/tests/test_errors.py	2010-09-13 02:32:44 +0000
@@ -159,6 +159,13 @@
             "cannot be broken.",
             str(error))
 
+    def test_lock_corrupt(self):
+        error = errors.LockCorrupt("corruption info")
+        self.assertEqualDiff("Lock is apparently held, but corrupted: "
+            "corruption info\n"
+            "Use 'bzr break-lock' to clear it",
+            str(error))
+
     def test_knit_data_stream_incompatible(self):
         error = errors.KnitDataStreamIncompatible(
             'stream format', 'target format')

=== modified file 'bzrlib/tests/test_lockdir.py'
--- a/bzrlib/tests/test_lockdir.py	2010-06-15 09:53:42 +0000
+++ b/bzrlib/tests/test_lockdir.py	2010-09-13 02:32:44 +0000
@@ -564,6 +564,62 @@
         finally:
             bzrlib.ui.ui_factory = orig_factory
 
+    def test_break_lock_corrupt_info(self):
+        """break_lock works even if the info file is corrupt (and tells the UI
+        that it is corrupt).
+        """
+        ld = self.get_lock()
+        ld2 = self.get_lock()
+        ld.create()
+        ld.lock_write()
+        ld.transport.put_bytes_non_atomic('test_lock/held/info', '\0')
+        class LoggingUIFactory(bzrlib.ui.SilentUIFactory):
+            def __init__(self):
+                self.prompts = []
+            def get_boolean(self, prompt):
+                self.prompts.append(('boolean', prompt))
+                return True
+        ui = LoggingUIFactory()
+        orig_factory = bzrlib.ui.ui_factory
+        bzrlib.ui.ui_factory = ui
+        try:
+            ld2.break_lock()
+            self.assertLength(1, ui.prompts)
+            self.assertEqual('boolean', ui.prompts[0][0])
+            self.assertStartsWith(ui.prompts[0][1], 'Break (corrupt LockDir')
+            self.assertRaises(LockBroken, ld.unlock)
+        finally:
+            bzrlib.ui.ui_factory = orig_factory
+
+    def test_break_lock_missing_info(self):
+        """break_lock works even if the info file is missing (and tells the UI
+        that it is corrupt).
+        """
+        ld = self.get_lock()
+        ld2 = self.get_lock()
+        ld.create()
+        ld.lock_write()
+        ld.transport.delete('test_lock/held/info')
+        class LoggingUIFactory(bzrlib.ui.SilentUIFactory):
+            def __init__(self):
+                self.prompts = []
+            def get_boolean(self, prompt):
+                self.prompts.append(('boolean', prompt))
+                return True
+        ui = LoggingUIFactory()
+        orig_factory = bzrlib.ui.ui_factory
+        bzrlib.ui.ui_factory = ui
+        try:
+            ld2.break_lock()
+            self.assertRaises(LockBroken, ld.unlock)
+            self.assertLength(0, ui.prompts)
+        finally:
+            bzrlib.ui.ui_factory = orig_factory
+        # Suppress warnings due to ld not being unlocked
+        # XXX: if lock_broken hook was invoked in this case, this hack would
+        # not be necessary.  - Andrew Bennetts, 2010-09-06.
+        del self._lock_actions[:]
+
     def test_create_missing_base_directory(self):
         """If LockDir.path doesn't exist, it can be created
 
@@ -681,6 +737,51 @@
             ['<unknown>', '<unknown>', '<unknown>', '(unknown)'],
             formatted_info)
 
+    def test_corrupt_lockdir_info(self):
+        """We can cope with corrupt (and thus unparseable) info files."""
+        # This seems like a fairly common failure case too - see
+        # <https://bugs.edge.launchpad.net/bzr/+bug/619872> for instance.
+        # In particular some systems tend to fill recently created files with
+        # nul bytes after recovering from a system crash.
+        t = self.get_transport()
+        t.mkdir('test_lock')
+        t.mkdir('test_lock/held')
+        t.put_bytes('test_lock/held/info', '\0')
+        lf = LockDir(t, 'test_lock')
+        self.assertRaises(errors.LockCorrupt, lf.peek)
+        # Currently attempt_lock gives LockContention, but LockCorrupt would be
+        # a reasonable result too.
+        self.assertRaises(
+            (errors.LockCorrupt, errors.LockContention), lf.attempt_lock)
+        self.assertRaises(errors.LockCorrupt, lf.validate_token, 'fake token')
+
+    def test_missing_lockdir_info(self):
+        """We can cope with absent info files."""
+        t = self.get_transport()
+        t.mkdir('test_lock')
+        t.mkdir('test_lock/held')
+        lf = LockDir(t, 'test_lock')
+        # In this case we expect the 'not held' result from peek, because peek
+        # cannot be expected to notice that there is a 'held' directory with no
+        # 'info' file.
+        self.assertEqual(None, lf.peek())
+        # And lock/unlock may work or give LockContention (but not any other
+        # error).
+        try:
+            lf.attempt_lock()
+        except LockContention:
+            # LockContention is ok, and expected on Windows
+            pass
+        else:
+            # no error is ok, and expected on POSIX (because POSIX allows
+            # os.rename over an empty directory).
+            lf.unlock()
+        # Currently raises TokenMismatch, but LockCorrupt would be reasonable
+        # too.
+        self.assertRaises(
+            (errors.TokenMismatch, errors.LockCorrupt),
+            lf.validate_token, 'fake token')
+
 
 class TestLockDirHooks(TestCaseWithTransport):
 

=== modified file 'bzrlib/tests/test_rio.py'
--- a/bzrlib/tests/test_rio.py	2009-03-23 14:59:43 +0000
+++ b/bzrlib/tests/test_rio.py	2010-09-06 06:13:52 +0000
@@ -188,6 +188,14 @@
         self.assertEqual(s, None)
         self.assertTrue(s is None)
 
+    def test_read_nul_byte(self):
+        """File consisting of a nul byte causes an error."""
+        self.assertRaises(ValueError, read_stanza, ['\0'])
+
+    def test_read_nul_bytes(self):
+        """File consisting of many nul bytes causes an error."""
+        self.assertRaises(ValueError, read_stanza, ['\0' * 100])
+
     def test_read_iter(self):
         """Read several stanzas from file"""
         tmpf = TemporaryFile()

=== modified file 'bzrlib/transport/ssh.py'
--- a/bzrlib/transport/ssh.py	2010-06-17 13:16:51 +0000
+++ b/bzrlib/transport/ssh.py	2010-09-09 07:31:02 +0000
@@ -645,11 +645,28 @@
 _subproc_weakrefs = set()
 
 def _close_ssh_proc(proc):
-    for func in [proc.stdin.close, proc.stdout.close, proc.wait]:
-        try:
-            func()
+    """Carefully close stdin/stdout and reap the SSH process.
+
+    If the pipes are already closed and/or the process has already been
+    wait()ed on, that's ok, and no error is raised.  The goal is to do our best
+    to clean up (whether or not a clean up was already tried).
+    """
+    dotted_names = ['stdin.close', 'stdout.close', 'wait']
+    for dotted_name in dotted_names:
+        attrs = dotted_name.split('.')
+        try:
+            obj = proc
+            for attr in attrs:
+                obj = getattr(obj, attr)
+        except AttributeError:
+            # It's ok for proc.stdin or proc.stdout to be None.
+            continue
+        try:
+            obj()
         except OSError:
-            pass
+            # It's ok for the pipe to already be closed, or the process to
+            # already be finished.
+            continue
 
 
 class SSHConnection(object):

=== modified file 'doc/en/admin-guide/simple-setups.txt'
--- a/doc/en/admin-guide/simple-setups.txt	2009-12-07 21:51:01 +0000
+++ b/doc/en/admin-guide/simple-setups.txt	2010-09-06 20:47:57 +0000
@@ -3,7 +3,7 @@
 
 Consider the following simple scenario where we will be serving Bazaar branches
 that live on a single server.  Those branches are in the subdirectories of
-``/srv/bzr`` (or ``C:\\bzr``) and they will all be related to a single project
+``/srv/bzr`` (or ``C:\bzr``) and they will all be related to a single project
 called "ProjectX".  ProjectX will have a trunk branch and at least one feature
 branch.  As we get further, we will consider other scenarios, but this will be
 a sufficiently motivating example.

=== modified file 'doc/en/whats-new/whats-new-in-2.2.txt'
--- a/doc/en/whats-new/whats-new-in-2.2.txt	2010-08-11 09:27:45 +0000
+++ b/doc/en/whats-new/whats-new-in-2.2.txt	2010-08-20 03:53:57 +0000
@@ -8,6 +8,9 @@
 development series. The 2.0 and 2.1 series will also continue to get
 bugfixes. (Currently 2.0 is planned to be supported for another 6 months.)
 
+The main changes in 2.2 are: **better local and network performance**,
+**reduced memory usage**, and several user-interface improvements.
+
 Users are encouraged to upgrade from the other stable series.  This
 document outlines the improvements in Bazaar 2.2 vs Bazaar 2.1. As well as
 summarizing improvements made to the core product, it highlights
@@ -78,8 +81,8 @@
   (John Arbash Meinel, #562429)
 
 
-Command impprovements
-*********************
+Command improvements
+********************
 
 * Added ``bzr remove-branch`` command that can remove a local or remote 
   branch. (Jelmer Vernooij, #276295)
@@ -201,32 +204,25 @@
 * Improved performance of qlog, and treewidget-based dialogs (qcommit,
   qadd, qrevert etc.)
 
-* qpush, qmerge, etc.: When there are uncommitted changes in the working
+* qpush, qmerge, etc.: when there are uncommitted changes in the working
   tree, user has the option to commit, or revert.
 
-* qcommit: user can update bound branch/chgeckout if it is not up to date.
-
-* Fixed bug with Mac OS X support when windows start in background.
-  (Thanks to Luis Arias)
-
-* qlog: Context menu actions for tag, and revert will now show a branch
+* qcommit: user can update bound branch/checkout if it is not up to date.
+
+* Better support of Mac OS X: dialog windows no more start in background.
+
+* qlog: Context menu actions for tag and revert will now show a branch
   menu if more than one branch is open.
 
 * qlog: more context menu actions for update, cherry-pick, and reverse
   cherry-pick.
 
-* Language of GUI can be set in DEFAULT section of bazaar.conf as
-``language = code``. Language codes are the same as for LANG environment
-  variable. Environment variable LANGUAGE still preferred over settings
+* Language of GUI can be set in DEFAULT section of bazaar.conf 
+  as ``language = code``. Language codes are the same 
+  as for ``LANG`` environment variable. 
+  Environment variable ``LANGUAGE`` still preferred over settings 
   in bazaar.conf.
 
-* Language of GUI can be set in DEFAULT section of bazaar.conf 
-  as ``language = code``. Language codes are the same as for LANG environment
-  variable. Environment variable LANGUAGE still preferred over settings
-  in bazaar.conf. (Alexander Belchenko)
-
-* New Thai and Hungarian translations.
-
 
 Platform-specific changes
 *************************




More information about the bazaar-commits mailing list