Rev 3868: Make InterPackRepo set the new pack's cache size even when there is in file:///home/pqm/archives/thelove/bzr/%2Btrunk/

Canonical.com Patch Queue Manager pqm at pqm.ubuntu.com
Fri Nov 28 01:11:21 GMT 2008


At file:///home/pqm/archives/thelove/bzr/%2Btrunk/

------------------------------------------------------------
revno: 3868
revision-id: pqm at pqm.ubuntu.com-20081128011117-4l4jwujo8pm2mitn
parent: pqm at pqm.ubuntu.com-20081127232013-edew5idnputt4fxn
parent: andrew.bennetts at canonical.com-20081127230808-t0n6k0tvxrt32bkt
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Fri 2008-11-28 01:11:17 +0000
message:
  Make InterPackRepo set the new pack's cache size even when there is
  	stacking.
modified:
  NEWS                           NEWS-20050323055033-4e00b5db738777ff
  bzrlib/fetch.py                fetch.py-20050818234941-26fea6105696365d
  bzrlib/repository.py           rev_storage.py-20051111201905-119e9401e46257e3
    ------------------------------------------------------------
    revno: 3834.4.5
    revision-id: andrew.bennetts at canonical.com-20081127230808-t0n6k0tvxrt32bkt
    parent: andrew.bennetts at canonical.com-20081127224753-xwuxxznkt8wzh76f
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: newpack-cache-limit
    timestamp: Fri 2008-11-28 10:08:08 +1100
    message:
      Add some comments to fetch.py
    modified:
      bzrlib/fetch.py                fetch.py-20050818234941-26fea6105696365d
    ------------------------------------------------------------
    revno: 3834.4.4
    revision-id: andrew.bennetts at canonical.com-20081127224753-xwuxxznkt8wzh76f
    parent: andrew.bennetts at canonical.com-20081127223412-9zavibobz97s0gdv
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: newpack-cache-limit
    timestamp: Fri 2008-11-28 09:47:53 +1100
    message:
      Add NEWS entry, add comment to the hack in InterPackRepo.fetch.
    modified:
      NEWS                           NEWS-20050323055033-4e00b5db738777ff
      bzrlib/repository.py           rev_storage.py-20051111201905-119e9401e46257e3
    ------------------------------------------------------------
    revno: 3834.4.3
    revision-id: andrew.bennetts at canonical.com-20081127223412-9zavibobz97s0gdv
    parent: andrew.bennetts at canonical.com-20081127223107-m4bvuih7dan46vjs
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: newpack-cache-limit
    timestamp: Fri 2008-11-28 09:34:12 +1100
    message:
      Remove XXX comment, use _get_target_pack_collection helper so that InterPackToRemotePack works.
    modified:
      bzrlib/repository.py           rev_storage.py-20051111201905-119e9401e46257e3
    ------------------------------------------------------------
    revno: 3834.4.2
    revision-id: andrew.bennetts at canonical.com-20081127223107-m4bvuih7dan46vjs
    parent: andrew.bennetts at canonical.com-20081112231847-fnbqfdu5dtvwthog
    parent: pqm at pqm.ubuntu.com-20081127093507-qn31zeicepc4g6li
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: newpack-cache-limit
    timestamp: Fri 2008-11-28 09:31:07 +1100
    message:
      Merge from bzr.dev.
    added:
      bzrlib/foreign.py              foreign.py-20081112170002-olsxmandkk8qyfuq-1
      bzrlib/tests/test_foreign.py   test_foreign.py-20081125004048-ywb901edgp9lluxo-1
      contrib/convert_to_1.9.py      convert_to_dev2.py-20081014130524-z1ydl3mq9b4ehlvv-1
      tools/win32/build_release.py   build_release.py-20081105204355-2ghh5cv01v1x4rzz-1
    modified:
      NEWS                           NEWS-20050323055033-4e00b5db738777ff
      bzrlib/__init__.py             __init__.py-20050309040759-33e65acf91bbcd5d
      bzrlib/_readdir_pyx.pyx        readdir.pyx-20060609152855-rm6v321vuaqyh9tu-1
      bzrlib/branch.py               branch.py-20050309040759-e4baf4e0d046576e
      bzrlib/btree_index.py          index.py-20080624222253-p0x5f92uyh5hw734-7
      bzrlib/builtins.py             builtins.py-20050830033751-fc01482b9ca23183
      bzrlib/commit.py               commit.py-20050511101309-79ec1a0168e0e825
      bzrlib/config.py               config.py-20051011043216-070c74f4e9e338e8
      bzrlib/errors.py               errors.py-20050309040759-20512168c4e14fbd
      bzrlib/fetch.py                fetch.py-20050818234941-26fea6105696365d
      bzrlib/help_topics/__init__.py help_topics.py-20060920210027-rnim90q9e0bwxvy4-1
      bzrlib/index.py                index.py-20070712131115-lolkarso50vjr64s-1
      bzrlib/knit.py                 knit.py-20051212171256-f056ac8f0fbe1bd9
      bzrlib/lockable_files.py       control_files.py-20051111201905-bb88546e799d669f
      bzrlib/log.py                  log.py-20050505065812-c40ce11702fe5fb1
      bzrlib/mutabletree.py          mutabletree.py-20060906023413-4wlkalbdpsxi2r4y-2
      bzrlib/option.py               option.py-20051014052914-661fb36e76e7362f
      bzrlib/plugin.py               plugin.py-20050622060424-829b654519533d69
      bzrlib/plugins/launchpad/account.py account.py-20071011033320-50y6vfftywf4yllw-1
      bzrlib/plugins/launchpad/lp_directory.py lp_indirect.py-20070126012204-de5rugwlt22c7u7e-1
      bzrlib/plugins/launchpad/test_account.py test_account.py-20071011033320-50y6vfftywf4yllw-2
      bzrlib/python-compat.h         pythoncompat.h-20080924041409-9kvi0fgtuuqp743j-1
      bzrlib/remote.py               remote.py-20060720103555-yeeg2x51vn0rbtdp-1
      bzrlib/repofmt/pack_repo.py    pack_repo.py-20070813041115-gjv5ma7ktfqwsjgn-1
      bzrlib/repofmt/weaverepo.py    presplitout.py-20070125045333-wfav3tsh73oxu3zk-1
      bzrlib/repository.py           rev_storage.py-20051111201905-119e9401e46257e3
      bzrlib/shelf_ui.py             shelver.py-20081005210102-33worgzwrtdw0yrm-1
      bzrlib/tests/__init__.py       selftest.py-20050531073622-8d0e3c8845c97a64
      bzrlib/tests/blackbox/test_log.py test_log.py-20060112090212-78f6ea560c868e24
      bzrlib/tests/branch_implementations/test_sprout.py test_sprout.py-20070521151739-b8t8p7axw1h966ws-1
      bzrlib/tests/branch_implementations/test_stacking.py test_stacking.py-20080214020755-msjlkb7urobwly0f-1
      bzrlib/tests/commands/test_commit.py test_commit.py-20070913161801-ydrx2k5gmv7k7eiu-1
      bzrlib/tests/interrepository_implementations/test_fetch.py test_fetch.py-20080425213627-j60cjh782ufm83ry-1
      bzrlib/tests/per_repository/test_commit_builder.py test_commit_builder.py-20060606110838-76e3ra5slucqus81-1
      bzrlib/tests/per_repository/test_repository.py test_repository.py-20060131092128-ad07f494f5c9d26c
      bzrlib/tests/per_repository/test_revision.py testrevprops.py-20051013073044-92bc3c68302ce1bf
      bzrlib/tests/test_fetch.py     testfetch.py-20050825090644-f73e07e7dfb1765a
      bzrlib/tests/test_knit.py      test_knit.py-20051212171302-95d4c00dd5f11f2b
      bzrlib/tests/test_log.py       testlog.py-20050728115707-1a514809d7d49309
      bzrlib/tests/test_pack_repository.py test_pack_repository-20080801043947-eaw0e6h2gu75kwmy-1
      bzrlib/tests/test_permissions.py test_permissions.py-20051215004520-ccf475789c80e80c
      bzrlib/tests/test_plugins.py   plugins.py-20050622075746-32002b55e5e943e9
      bzrlib/tests/test_remote.py    test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
      bzrlib/tests/test_repository.py test_repository.py-20060131075918-65c555b881612f4d
      bzrlib/tests/test_revision.py  testrevision.py-20050804210559-46f5e1eb67b01289
      bzrlib/tests/test_sftp_transport.py testsftp.py-20051027032739-247570325fec7e7e
      bzrlib/tests/test_shelf_ui.py  test_shelf_ui.py-20081027155203-wtcuazg85wp9u4fv-1
      bzrlib/tests/workingtree_implementations/test_parents.py test_set_parents.py-20060807231740-yicmnlci1mj8smu1-1
      bzrlib/transport/remote.py     ssh.py-20060608202016-c25gvf1ob7ypbus6-1
      bzrlib/versionedfile.py        versionedfile.py-20060222045106-5039c71ee3b65490
      bzrlib/workingtree.py          workingtree.py-20050511021032-29b6ec0a681e02e3
      bzrlib/workingtree_4.py        workingtree_4.py-20070208044105-5fgpc5j3ljlh5q6c-1
    ------------------------------------------------------------
    revno: 3834.4.1
    revision-id: andrew.bennetts at canonical.com-20081112231847-fnbqfdu5dtvwthog
    parent: pqm at pqm.ubuntu.com-20081112073322-dpai0jsluo3cvpf2
    committer: Andrew Bennetts <andrew.bennetts at canonical.com>
    branch nick: newpack-cache-limit
    timestamp: Thu 2008-11-13 09:18:47 +1000
    message:
      Make InterPackRepo set the packer cache size even when there is stacking.
    modified:
      bzrlib/fetch.py                fetch.py-20050818234941-26fea6105696365d
      bzrlib/repository.py           rev_storage.py-20051111201905-119e9401e46257e3
=== modified file 'NEWS'
--- a/NEWS	2008-11-27 09:00:45 +0000
+++ b/NEWS	2008-11-27 22:47:53 +0000
@@ -70,6 +70,11 @@
       "PermissionDenied: "None"" on the client.
       (Andrew Bennetts, #299254)
       
+    * Pushing to a stacked pack repository now batches writes, the same
+      way writes are batched to ordinary pack repository.  This makes
+      pushing to a stacked branch over the network much faster.
+      (Andrew Bennetts, #294479)
+
     * TooManyConcurrentRequests no longer occur when a fetch fails and
       tries to abort a write group.  This allows the root cause (e.g. a
       network interruption) to be reported.  (Andrew Bennetts, #297014)

=== modified file 'bzrlib/fetch.py'
--- a/bzrlib/fetch.py	2008-11-25 20:39:57 +0000
+++ b/bzrlib/fetch.py	2008-11-27 23:08:08 +0000
@@ -77,10 +77,13 @@
     """
 
     def __init__(self, to_repository, from_repository, last_revision=None, pb=None,
-        find_ghosts=True):
+        find_ghosts=True, _write_group_acquired_callable=None):
         """Create a repo fetcher.
 
         :param find_ghosts: If True search the entire history for ghosts.
+        :param _write_group_acquired_callable: Don't use; this parameter only
+            exists to facilitate a hack done in InterPackRepo.fetch.  We would
+            like to remove this parameter.
         """
         # result variables.
         self.failed_revisions = []
@@ -95,6 +98,7 @@
         # must not mutate self._last_revision as its potentially a shared instance
         self._last_revision = last_revision
         self.find_ghosts = find_ghosts
+        self._write_group_acquired_callable = _write_group_acquired_callable
         if pb is None:
             self.pb = bzrlib.ui.ui_factory.nested_progress_bar()
             self.nested_pb = self.pb
@@ -107,6 +111,10 @@
             try:
                 self.to_repository.start_write_group()
                 try:
+                    if self._write_group_acquired_callable is not None:
+                        # Used by InterPackRepo.fetch to set_write_cache_size
+                        # on the new pack.
+                        self._write_group_acquired_callable()
                     self.__fetch()
                 except:
                     self.to_repository.abort_write_group(suppress_errors=True)

=== modified file 'bzrlib/repository.py'
--- a/bzrlib/repository.py	2008-11-25 18:51:48 +0000
+++ b/bzrlib/repository.py	2008-11-27 22:47:53 +0000
@@ -2836,12 +2836,16 @@
             # fetching from a stacked repository or into a stacked repository
             # we use the generic fetch logic which uses the VersionedFiles
             # attributes on repository.
-            #
-            # XXX: Andrew suggests removing the check on the target
-            # repository.
             from bzrlib.fetch import RepoFetcher
+            # Make sure the generic fetcher sets the write cache size on the
+            # new pack (just like Packer.pack does) to avoid doing many tiny
+            # writes (which can be slow over a network connection).
+            # XXX: ideally the transport layer would do this automatically.
+            pack_coll = self._get_target_pack_collection()
+            set_cache_size = (
+                lambda: pack_coll._new_pack.set_write_cache_size(1024*1024))
             fetcher = RepoFetcher(self.target, self.source, revision_id,
-                                  pb, find_ghosts)
+                                  pb, find_ghosts, set_cache_size)
             return fetcher.count_copied, fetcher.failed_revisions
         mutter("Using fetch logic to copy between %s(%s) and %s(%s)",
                self.source, self.source._format, self.target, self.target._format)




More information about the bazaar-commits mailing list