Rev 3721: Faster push when there are no new revisions, in file:///home/pqm/archives/thelove/bzr/%2Btrunk/

Canonical.com Patch Queue Manager pqm at pqm.ubuntu.com
Mon Sep 22 07:46:00 BST 2008


At file:///home/pqm/archives/thelove/bzr/%2Btrunk/

------------------------------------------------------------
revno: 3721
revision-id: pqm at pqm.ubuntu.com-20080922064555-jayr5evddyn8w7g6
parent: pqm at pqm.ubuntu.com-20080922042327-druvxn7q10gs3fw4
parent: andrew.bennetts at canonical.com-20080922061640-rv0rkxvy81o5sfui
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Mon 2008-09-22 07:45:55 +0100
message:
  Faster push when there are no new revisions,
  	and when there are no local tags. (Andrew Bennetts)
modified:
  NEWS                           NEWS-20050323055033-4e00b5db738777ff
  bzrlib/branch.py               branch.py-20050309040759-e4baf4e0d046576e
  bzrlib/smart/client.py         client.py-20061116014825-2k6ada6xgulslami-1
  bzrlib/tests/__init__.py       selftest.py-20050531073622-8d0e3c8845c97a64
  bzrlib/tests/branch_implementations/test_push.py test_push.py-20070130153159-fhfap8uoifevg30j-1
    ------------------------------------------------------------
    revno: 3703.3.9
    revision-id: andrew.bennetts at canonical.com-20080922061640-rv0rkxvy81o5sfui
    parent: andrew.bennetts at canonical.com-20080922054013-goyt6y12ctmlk6am
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Mon 2008-09-22 16:16:40 +1000
    message:
      Remove reference to removed test_effort module.
    modified:
      bzrlib/tests/__init__.py       selftest.py-20050531073622-8d0e3c8845c97a64
    ------------------------------------------------------------
    revno: 3703.3.8
    revision-id: andrew.bennetts at canonical.com-20080922054013-goyt6y12ctmlk6am
    parent: andrew.bennetts at canonical.com-20080922053345-t1l5lpbc90f1dxvi
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Mon 2008-09-22 15:40:13 +1000
    message:
      Add NEWS entry.
    modified:
      NEWS                           NEWS-20050323055033-4e00b5db738777ff
    ------------------------------------------------------------
    revno: 3703.3.7
    revision-id: andrew.bennetts at canonical.com-20080922053345-t1l5lpbc90f1dxvi
    parent: andrew.bennetts at canonical.com-20080922044847-ad1y8ibge3s2ak4i
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Mon 2008-09-22 15:33:45 +1000
    message:
      Move empty push effort tests to branch_implementations.
    removed:
      bzrlib/tests/test_effort.py    test_effort.py-20080915085512-lslq0r1lpbbkdj3q-1
    modified:
      bzrlib/tests/branch_implementations/test_push.py test_push.py-20070130153159-fhfap8uoifevg30j-1
    ------------------------------------------------------------
    revno: 3703.3.6
    revision-id: andrew.bennetts at canonical.com-20080922044847-ad1y8ibge3s2ak4i
    parent: andrew.bennetts at canonical.com-20080916073924-b3w9mk29c0c87jmm
    parent: pqm at pqm.ubuntu.com-20080921012105-ote1u11mokjim9ir
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Mon 2008-09-22 14:48:47 +1000
    message:
      Merge from bzr.dev.
    modified:
      NEWS                           NEWS-20050323055033-4e00b5db738777ff
      bzrlib/branch.py               branch.py-20050309040759-e4baf4e0d046576e
      bzrlib/builtins.py             builtins.py-20050830033751-fc01482b9ca23183
      bzrlib/bundle/__init__.py      changeset.py-20050513021216-b02ab57fb9738913
      bzrlib/bzrdir.py               bzrdir.py-20060131065624-156dfea39c4387cb
      bzrlib/directory_service.py    directory_service.py-20080305221044-vr2mkvlsk8jypa2y-1
      bzrlib/help_topics/__init__.py help_topics.py-20060920210027-rnim90q9e0bwxvy4-1
      bzrlib/log.py                  log.py-20050505065812-c40ce11702fe5fb1
      bzrlib/missing.py              missing.py-20050812153334-097f7097e2a8bcd1
      bzrlib/osutils.py              osutils.py-20050309040759-eeaff12fbf77ac86
      bzrlib/remote.py               remote.py-20060720103555-yeeg2x51vn0rbtdp-1
      bzrlib/revisionspec.py         revisionspec.py-20050907152633-17567659fd5c0ddb
      bzrlib/smart/branch.py         branch.py-20061124031907-mzh3pla28r83r97f-1
      bzrlib/smart/medium.py         medium.py-20061103051856-rgu2huy59fkz902q-1
      bzrlib/smart/message.py        message.py-20080222013625-ncqmh3nrxjkxab87-1
      bzrlib/smart/protocol.py       protocol.py-20061108035435-ot0lstk2590yqhzr-1
      bzrlib/smart/request.py        request.py-20061108095550-gunadhxmzkdjfeek-1
      bzrlib/smart/server.py         server.py-20061110062051-chzu10y32vx8gvur-1
      bzrlib/tests/blackbox/test_branch.py test_branch.py-20060524161337-noms9gmcwqqrfi8y-1
      bzrlib/tests/blackbox/test_export.py test_export.py-20051229024010-e6c26658e460fb1c
      bzrlib/tests/blackbox/test_init.py test_init.py-20060309032856-a292116204d86eb7
      bzrlib/tests/blackbox/test_remove_tree.py test_remove_tree.py-20061110192919-5j3xjciiaqbs2dvo-1
      bzrlib/tests/blackbox/test_switch.py test_switch.py-20071122111948-0c5en6uz92bwl76h-1
      bzrlib/tests/per_repository/__init__.py __init__.py-20060131092037-9564957a7d4a841b
      bzrlib/tests/per_repository/test_repository.py test_repository.py-20060131092128-ad07f494f5c9d26c
      bzrlib/tests/test_bundle.py    test.py-20050630184834-092aa401ab9f039c
      bzrlib/tests/test_directory_service.py test_directory_servi-20080305221044-vr2mkvlsk8jypa2y-2
      bzrlib/tests/test_missing.py   test_missing.py-20051212000028-694fa4f658a81f48
      bzrlib/tests/test_remote.py    test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
      bzrlib/tests/test_smart.py     test_smart.py-20061122024551-ol0l0o0oofsu9b3t-2
      bzrlib/tests/test_transform.py test_transaction.py-20060105172520-b3ffb3946550e6c4
      bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
      bzrlib/tests/tree_implementations/__init__.py __init__.py-20060717075546-420s7b0bj9hzeowi-2
      bzrlib/tests/tree_implementations/test_inv.py test_inv.py-20070312023226-0cdvk5uwhutis9vg-1
      bzrlib/tests/tree_implementations/test_test_trees.py test_tree_trees.py-20060720091921-3nwi5h21lf06vf5p-1
      bzrlib/tests/tree_implementations/test_tree.py test_tree.py-20061215160206-usu7lwcj8aq2n3br-1
      bzrlib/tests/workingtree_implementations/test_rename_one.py test_rename_one.py-20070226161242-2d8ibdedl700jgio-1
      bzrlib/tests/workingtree_implementations/test_workingtree.py test_workingtree.py-20060203003124-817757d3e31444fb
      bzrlib/transform.py            transform.py-20060105172343-dd99e54394d91687
      bzrlib/transport/local.py      local_transport.py-20050711165921-9b1f142bfe480c24
      bzrlib/tree.py                 tree.py-20050309040759-9d5f2496be663e77
      doc/en/user-guide/http_smart_server.txt fastcgi.txt-20061005091552-rz8pva0olkxv0sd8-3
      tools/doc_generate/autodoc_man.py bzrman.py-20050601153041-0ff7f74de456d15e
    ------------------------------------------------------------
    revno: 3703.3.5
    revision-id: andrew.bennetts at canonical.com-20080916073924-b3w9mk29c0c87jmm
    parent: andrew.bennetts at canonical.com-20080916054124-w60risuh774ldcw5
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Tue 2008-09-16 17:39:24 +1000
    message:
      Allow subclasses to control if _basic_push can skip tag merging.
    modified:
      bzrlib/branch.py               branch.py-20050309040759-e4baf4e0d046576e
    ------------------------------------------------------------
    revno: 3703.3.4
    revision-id: andrew.bennetts at canonical.com-20080916054124-w60risuh774ldcw5
    parent: andrew.bennetts at canonical.com-20080915100200-0q8xn3n6wmzl2u32
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Tue 2008-09-16 15:41:24 +1000
    message:
      Move check to skip _SmartClient hook invocation.
    modified:
      bzrlib/smart/client.py         client.py-20061116014825-2k6ada6xgulslami-1
    ------------------------------------------------------------
    revno: 3703.3.3
    revision-id: andrew.bennetts at canonical.com-20080915100200-0q8xn3n6wmzl2u32
    parent: andrew.bennetts at canonical.com-20080915093221-6czr839im3qiclq3
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Mon 2008-09-15 20:02:00 +1000
    message:
      Neater effort tests for push by using a _SmartClient.hooks['call'] hook.
    modified:
      bzrlib/smart/client.py         client.py-20061116014825-2k6ada6xgulslami-1
      bzrlib/tests/__init__.py       selftest.py-20050531073622-8d0e3c8845c97a64
      bzrlib/tests/test_effort.py    test_effort.py-20080915085512-lslq0r1lpbbkdj3q-1
    ------------------------------------------------------------
    revno: 3703.3.2
    revision-id: andrew.bennetts at canonical.com-20080915093221-6czr839im3qiclq3
    parent: andrew.bennetts at canonical.com-20080912073747-30erzge03sk27is0
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Mon 2008-09-15 19:32:21 +1000
    message:
      Simple effort tests for pushing an empty branch.
    added:
      bzrlib/tests/test_effort.py    test_effort.py-20080915085512-lslq0r1lpbbkdj3q-1
    modified:
      bzrlib/tests/__init__.py       selftest.py-20050531073622-8d0e3c8845c97a64
    ------------------------------------------------------------
    revno: 3703.3.1
    revision-id: andrew.bennetts at canonical.com-20080912073747-30erzge03sk27is0
    parent: pqm at pqm.ubuntu.com-20080911061059-svzqfejar17ui4zw
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: faster-empty-push
    timestamp: Fri 2008-09-12 17:37:47 +1000
    message:
      Skip unnecessary work in BzrBranch._basic_push.
      
      A push of 0 new revisions where the source branch no tags now only needs a
      constant 8 HPSS round trips.  That's still higher than necessary, but much
      better than the >20 it previously took.  And the round trips no longer increase
      depending on the the number and size of the remote indices!).
    modified:
      bzrlib/branch.py               branch.py-20050309040759-e4baf4e0d046576e
=== modified file 'NEWS'
--- a/NEWS	2008-09-21 01:21:05 +0000
+++ b/NEWS	2008-09-22 05:40:13 +0000
@@ -17,6 +17,10 @@
       standalone branch regardless of the presence of shared repositories.
       (Daniel Watkins)
 
+    * ``bzr push`` is faster in the case there are no new revisions to
+      push.  It is also faster if there are no tags in the local branch.
+      (Andrew Bennetts)
+
     * Location aliases can now accept a trailing path.  (Micheal Hudson)
 
     * Switching in heavyweight checkouts uses the master branch's context, not

=== modified file 'bzrlib/branch.py'
--- a/bzrlib/branch.py	2008-09-15 14:27:19 +0000
+++ b/bzrlib/branch.py	2008-09-22 04:48:47 +0000
@@ -1802,16 +1802,25 @@
         result.source_branch = self
         result.target_branch = target
         result.old_revno, result.old_revid = target.last_revision_info()
-
-        # We assume that during 'push' this repository is closer than
-        # the target.
-        graph = self.repository.get_graph(target.repository)
-        target.update_revisions(self, stop_revision, overwrite=overwrite,
-                                graph=graph)
-        result.tag_conflicts = self.tags.merge_to(target.tags, overwrite)
+        if result.old_revid != self.last_revision():
+            # We assume that during 'push' this repository is closer than
+            # the target.
+            graph = self.repository.get_graph(target.repository)
+            target.update_revisions(self, stop_revision, overwrite=overwrite,
+                                    graph=graph)
+        if self._push_should_merge_tags():
+            result.tag_conflicts = self.tags.merge_to(target.tags, overwrite)
         result.new_revno, result.new_revid = target.last_revision_info()
         return result
 
+    def _push_should_merge_tags(self):
+        """Should _basic_push merge this branch's tags into the target?
+        
+        The default implementation returns False if this branch has no tags,
+        and True the rest of the time.  Subclasses may override this.
+        """
+        return self.tags.supports_tags() and self.tags.get_tag_dict()
+
     def get_parent(self):
         """See Branch.get_parent."""
         parent = self._get_parent_location()

=== modified file 'bzrlib/smart/client.py'
--- a/bzrlib/smart/client.py	2008-06-17 04:22:26 +0000
+++ b/bzrlib/smart/client.py	2008-09-16 05:41:24 +0000
@@ -17,7 +17,10 @@
 import bzrlib
 from bzrlib.smart import message, protocol
 from bzrlib.trace import warning
-from bzrlib import errors
+from bzrlib import (
+    errors,
+    hooks,
+    )
 
 
 class _SmartClient(object):
@@ -50,8 +53,16 @@
             encoder.call(method, *args)
         return response_handler
 
+    def _run_call_hooks(self, method, args, body, readv_body):
+        if not _SmartClient.hooks['call']:
+            return
+        params = CallHookParams(method, args, body, readv_body)
+        for hook in _SmartClient.hooks['call']:
+            hook(params)
+            
     def _call_and_read_response(self, method, args, body=None, readv_body=None,
             expect_response_body=True):
+        self._run_call_hooks(method, args, body, readv_body)
         if self._medium._protocol_version is not None:
             response_handler = self._send_request(
                 self._medium._protocol_version, method, args, body=body,
@@ -163,3 +174,34 @@
         """
         return self._medium.remote_path_from_transport(transport)
 
+
+class SmartClientHooks(hooks.Hooks):
+
+    def __init__(self):
+        hooks.Hooks.__init__(self)
+        self['call'] = []
+
+        
+_SmartClient.hooks = SmartClientHooks()
+
+
+class CallHookParams(object):
+    
+    def __init__(self, method, args, body, readv_body):
+        self.method = method
+        self.args = args
+        self.body = body
+        self.readv_body = readv_body
+
+    def __repr__(self):
+        attrs = dict((k, v) for (k, v) in self.__dict__.iteritems()
+                     if v is not None)
+        return '<%s %r>' % (self.__class__.__name__, attrs)
+
+    def __eq__(self, other):
+        if type(other) is not type(self):
+            return NotImplemented
+        return self.__dict__ == other.__dict__
+
+    def __ne__(self, other):
+        return not self == other

=== modified file 'bzrlib/tests/__init__.py'
--- a/bzrlib/tests/__init__.py	2008-09-04 20:32:04 +0000
+++ b/bzrlib/tests/__init__.py	2008-09-22 06:16:40 +0000
@@ -801,15 +801,18 @@
     def _clear_hooks(self):
         # prevent hooks affecting tests
         import bzrlib.branch
+        import bzrlib.smart.client
         import bzrlib.smart.server
         self._preserved_hooks = {
             bzrlib.branch.Branch: bzrlib.branch.Branch.hooks,
             bzrlib.mutabletree.MutableTree: bzrlib.mutabletree.MutableTree.hooks,
+            bzrlib.smart.client._SmartClient: bzrlib.smart.client._SmartClient.hooks,
             bzrlib.smart.server.SmartTCPServer: bzrlib.smart.server.SmartTCPServer.hooks,
             }
         self.addCleanup(self._restoreHooks)
         # reset all hooks to an empty instance of the appropriate type
         bzrlib.branch.Branch.hooks = bzrlib.branch.BranchHooks()
+        bzrlib.smart.client._SmartClient.hooks = bzrlib.smart.client.SmartClientHooks()
         bzrlib.smart.server.SmartTCPServer.hooks = bzrlib.smart.server.SmartServerHooks()
 
     def _silenceUI(self):

=== modified file 'bzrlib/tests/branch_implementations/test_push.py'
--- a/bzrlib/tests/branch_implementations/test_push.py	2008-05-23 07:33:05 +0000
+++ b/bzrlib/tests/branch_implementations/test_push.py	2008-09-22 05:33:45 +0000
@@ -18,13 +18,21 @@
 
 import os
  
-from bzrlib import bzrdir, errors
+from bzrlib import (
+    branch,
+    builtins,
+    bzrdir,
+    debug,
+    errors,
+    tests,
+    )
 from bzrlib.branch import Branch
 from bzrlib.bzrdir import BzrDir
 from bzrlib.memorytree import MemoryTree
-from bzrlib.remote import RemoteBranch
 from bzrlib.revision import NULL_REVISION
+from bzrlib.smart import client, server
 from bzrlib.tests.branch_implementations.test_branch import TestCaseWithBranch
+from bzrlib.transport import get_transport
 from bzrlib.transport.local import LocalURLServer
 
 
@@ -257,3 +265,68 @@
              2, rev2, True, None, True)
             ],
             self.hook_calls)
+
+
+class EmptyPushSmartEffortTests(TestCaseWithBranch):
+    """Tests that a push of 0 revisions should make a limited number of smart
+    protocol RPCs.
+    """
+
+    def setUp(self):
+        # Skip some scenarios that don't apply to these tests.
+        if (self.transport_server is not None and
+            issubclass(self.transport_server, server.SmartTCPServer)):
+            raise tests.TestNotApplicable(
+                'Does not apply when remote backing branch is also '
+                'a smart branch')
+        if isinstance(self.branch_format, branch.BzrBranchFormat4):
+            raise tests.TestNotApplicable(
+                'Branch format 4 is not usable via HPSS.')
+        super(EmptyPushSmartEffortTests, self).setUp()
+        # Create a smart server that publishes whatever the backing VFS server
+        # does.
+        self.smart_server = server.SmartTCPServer_for_testing()
+        self.smart_server.setUp(self.get_server())
+        self.addCleanup(self.smart_server.tearDown)
+        # Make two empty branches, 'empty' and 'target'.
+        self.empty_branch = self.make_branch('empty')
+        self.make_branch('target')
+        # Log all HPSS calls into self.hpss_calls.
+        client._SmartClient.hooks.install_named_hook(
+            'call', self.capture_hpss_call, None)
+        self.hpss_calls = []
+
+    def capture_hpss_call(self, params):
+        self.hpss_calls.append(params.method)
+
+    def test_empty_branch_api(self):
+        """The branch_obj.push API should make a limited number of HPSS calls.
+        """
+        transport = get_transport(self.smart_server.get_url()).clone('target')
+        target = Branch.open_from_transport(transport)
+        self.empty_branch.push(target)
+        self.assertEqual(
+            ['BzrDir.open',
+             'BzrDir.open_branch',
+             'BzrDir.find_repositoryV2',
+             'Branch.get_stacked_on_url',
+             'Branch.lock_write',
+             'Branch.last_revision_info',
+             'Branch.unlock'],
+            self.hpss_calls)
+
+    def test_empty_branch_command(self):
+        """The 'bzr push' command should make a limited number of HPSS calls.
+        """
+        cmd = builtins.cmd_push()
+        cmd.outf = tests.StringIOWrapper()
+        cmd.run(
+            directory=self.get_url() + 'empty',
+            location=self.smart_server.get_url() + 'target')
+        # HPSS calls as of 2008/09/22:
+        # [BzrDir.open, BzrDir.open_branch, BzrDir.find_repositoryV2,
+        # Branch.get_stacked_on_url, get, get, Branch.lock_write,
+        # Branch.last_revision_info, Branch.unlock]
+        self.assertTrue(len(self.hpss_calls) <= 9, self.hpss_calls)
+
+




More information about the bazaar-commits mailing list