Rev 2703: Merge transport improvements to support pack based repositories. in http://people.ubuntu.com/~robertc/baz2.0/repository

Robert Collins robertc at robertcollins.net
Sun Aug 5 09:20:33 BST 2007


At http://people.ubuntu.com/~robertc/baz2.0/repository

------------------------------------------------------------
revno: 2703
revision-id: robertc at robertcollins.net-20070805082029-4q3fj1fc7yn08yfh
parent: robertc at robertcollins.net-20070803034933-wtol46z6u16twzzl
parent: robertc at robertcollins.net-20070805081501-ipg5fapwuigozr50
committer: Robert Collins <robertc at robertcollins.net>
branch nick: repository
timestamp: Sun 2007-08-05 18:20:29 +1000
message:
  Merge transport improvements to support pack based repositories.
modified:
  NEWS                           NEWS-20050323055033-4e00b5db738777ff
  bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
  bzrlib/transport/__init__.py   transport.py-20050711165921-4978aa7ce1285ad5
  bzrlib/transport/chroot.py     chroot.py-20061011104729-0us9mgm97z378vnt-1
  bzrlib/transport/decorator.py  decorator.py-20060402223305-e913a0f25319ab42
  bzrlib/transport/ftp.py        ftp.py-20051116161804-58dc9506548c2a53
  bzrlib/transport/http/__init__.py http_transport.py-20050711212304-506c5fd1059ace96
  bzrlib/transport/local.py      local_transport.py-20050711165921-9b1f142bfe480c24
  bzrlib/transport/memory.py     memory.py-20051016101338-cd008dbdf69f04fc
  bzrlib/transport/remote.py     ssh.py-20060608202016-c25gvf1ob7ypbus6-1
  bzrlib/transport/sftp.py       sftp.py-20051019050329-ab48ce71b7e32dfe
    ------------------------------------------------------------
    revno: 2592.1.25.2.7.1.28.1.6.1.3.1.9.2.6
    revision-id: robertc at robertcollins.net-20070805081501-ipg5fapwuigozr50
    parent: robertc at robertcollins.net-20070805055353-k382i5ur5no56nnx
    committer: Robert Collins <robertc at robertcollins.net>
    branch nick: transport-get-file
    timestamp: Sun 2007-08-05 18:15:01 +1000
    message:
      * New methods on ``bzrlib.transport.Transport`` ``open_file_stream`` and
        ``close_file_stream`` allow incremental addition of data to a file
        without requiring that all the data be buffered in memory.
        (Robert Collins)
    modified:
      NEWS                           NEWS-20050323055033-4e00b5db738777ff
      bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
    ------------------------------------------------------------
    revno: 2592.1.25.2.7.1.28.1.6.1.3.1.9.2.5
    revision-id: robertc at robertcollins.net-20070805055353-k382i5ur5no56nnx
    parent: robertc at robertcollins.net-20070805053815-jeb19qdogkh5zrq5
    committer: Robert Collins <robertc at robertcollins.net>
    branch nick: transport-get-file
    timestamp: Sun 2007-08-05 15:53:53 +1000
    message:
      Sync up with open file streams on get/get_bytes.
    modified:
      bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
      bzrlib/transport/local.py      local_transport.py-20050711165921-9b1f142bfe480c24
    ------------------------------------------------------------
    revno: 2592.1.25.2.7.1.28.1.6.1.3.1.9.2.4
    revision-id: robertc at robertcollins.net-20070805053815-jeb19qdogkh5zrq5
    parent: robertc at robertcollins.net-20070805025745-eg2qmr8jzsky39y2
    committer: Robert Collins <robertc at robertcollins.net>
    branch nick: transport-get-file
    timestamp: Sun 2007-08-05 15:38:15 +1000
    message:
      Add mode parameter to Transport.open_file_stream.
    modified:
      bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
      bzrlib/transport/__init__.py   transport.py-20050711165921-4978aa7ce1285ad5
      bzrlib/transport/chroot.py     chroot.py-20061011104729-0us9mgm97z378vnt-1
      bzrlib/transport/decorator.py  decorator.py-20060402223305-e913a0f25319ab42
      bzrlib/transport/local.py      local_transport.py-20050711165921-9b1f142bfe480c24
      bzrlib/transport/sftp.py       sftp.py-20051019050329-ab48ce71b7e32dfe
    ------------------------------------------------------------
    revno: 2592.1.25.2.7.1.28.1.6.1.3.1.9.2.3
    revision-id: robertc at robertcollins.net-20070805025745-eg2qmr8jzsky39y2
    parent: robertc at robertcollins.net-20070805014730-qjx8zkquv3pagglo
    committer: Robert Collins <robertc at robertcollins.net>
    branch nick: transport-get-file
    timestamp: Sun 2007-08-05 12:57:45 +1000
    message:
      Start open_file_stream logic.
    modified:
      bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
      bzrlib/transport/__init__.py   transport.py-20050711165921-4978aa7ce1285ad5
      bzrlib/transport/chroot.py     chroot.py-20061011104729-0us9mgm97z378vnt-1
      bzrlib/transport/decorator.py  decorator.py-20060402223305-e913a0f25319ab42
      bzrlib/transport/ftp.py        ftp.py-20051116161804-58dc9506548c2a53
      bzrlib/transport/local.py      local_transport.py-20050711165921-9b1f142bfe480c24
      bzrlib/transport/memory.py     memory.py-20051016101338-cd008dbdf69f04fc
      bzrlib/transport/remote.py     ssh.py-20060608202016-c25gvf1ob7ypbus6-1
      bzrlib/transport/sftp.py       sftp.py-20051019050329-ab48ce71b7e32dfe
    ------------------------------------------------------------
    revno: 2592.1.25.2.7.1.28.1.6.1.3.1.9.2.2
    revision-id: robertc at robertcollins.net-20070805014730-qjx8zkquv3pagglo
    parent: pqm at pqm.ubuntu.com-20070803043116-l7u1uypblmx1uxnr
    committer: Robert Collins <robertc at robertcollins.net>
    branch nick: transport-get-file
    timestamp: Sun 2007-08-05 11:47:30 +1000
    message:
      * New method ``bzrlib.transport.Transport.get_recommended_page_size``.
        This provides a hint to users of transports as to the reasonable
        minimum data to read. In principle this can take latency and
        bandwidth into account on a per-connection basis, but for now it
        just has hard coded values based on the url. (e.g. http:// has a large
        page size, file:// has a small one.) (Robert Collins)
    modified:
      NEWS                           NEWS-20050323055033-4e00b5db738777ff
      bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
      bzrlib/transport/__init__.py   transport.py-20050711165921-4978aa7ce1285ad5
      bzrlib/transport/decorator.py  decorator.py-20060402223305-e913a0f25319ab42
      bzrlib/transport/ftp.py        ftp.py-20051116161804-58dc9506548c2a53
      bzrlib/transport/http/__init__.py http_transport.py-20050711212304-506c5fd1059ace96
      bzrlib/transport/sftp.py       sftp.py-20051019050329-ab48ce71b7e32dfe
    ------------------------------------------------------------
    revno: 2592.1.25.2.7.1.28.1.6.1.3.1.9.2.1
    revision-id: pqm at pqm.ubuntu.com-20070803043116-l7u1uypblmx1uxnr
    parent: pqm at pqm.ubuntu.com-20070802221338-9333q05a8caaciwo
    parent: robertc at robertcollins.net-20070803010718-4grgme3fuhpaoatl
    committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
    branch nick: +trunk
    timestamp: Fri 2007-08-03 05:31:16 +0100
    message:
      (robertc) Do not test for " " in the file_id_involved repository tests. (Robert Collins).
    modified:
      bzrlib/tests/repository_implementations/test_fileid_involved.py test_file_involved.py-20051215205901-728a172d1014daaa
=== modified file 'NEWS'
--- a/NEWS	2007-08-03 00:29:10 +0000
+++ b/NEWS	2007-08-05 08:20:29 +0000
@@ -212,6 +212,18 @@
       available in one step. This allows the write lock to remain while making
       a series of data insertions.  (e.g. data conversion). (Robert Collins)
 
+    * New method ``bzrlib.transport.Transport.get_recommended_page_size``.
+      This provides a hint to users of transports as to the reasonable
+      minimum data to read. In principle this can take latency and
+      bandwidth into account on a per-connection basis, but for now it
+      just has hard coded values based on the url. (e.g. http:// has a large
+      page size, file:// has a small one.) (Robert Collins)
+
+    * New methods on ``bzrlib.transport.Transport`` ``open_file_stream`` and
+      ``close_file_stream`` allow incremental addition of data to a file
+      without requiring that all the data be buffered in memory.
+      (Robert Collins)
+
   TESTING:
 
     * Remove selftest ``--clean-output``, ``--numbered-dirs`` and

=== modified file 'bzrlib/tests/test_transport_implementations.py'
--- a/bzrlib/tests/test_transport_implementations.py	2007-07-20 18:59:29 +0000
+++ b/bzrlib/tests/test_transport_implementations.py	2007-08-05 08:15:01 +0000
@@ -230,6 +230,29 @@
 
         self.assertRaises(NoSuchFile, t.get_bytes, 'c')
 
+    def test_get_with_open_file_stream_sees_all_content(self):
+        t = self.get_transport()
+        if t.is_readonly():
+            return
+        handle = t.open_file_stream('foo')
+        try:
+            handle('b')
+            self.assertEqual('b', t.get('foo').read())
+        finally:
+            t.close_file_stream('foo')
+
+    def test_get_bytes_with_open_file_stream_sees_all_content(self):
+        t = self.get_transport()
+        if t.is_readonly():
+            return
+        handle = t.open_file_stream('foo')
+        try:
+            handle('b')
+            self.assertEqual('b', t.get_bytes('foo'))
+            self.assertEqual('b', t.get('foo').read())
+        finally:
+            t.close_file_stream('foo')
+
     def test_put(self):
         t = self.get_transport()
 
@@ -633,6 +656,33 @@
         t.mkdir('dnomode', mode=None)
         self.assertTransportMode(t, 'dnomode', 0777 & ~umask)
 
+    def test_opening_a_file_stream_creates_file(self):
+        t = self.get_transport()
+        if t.is_readonly():
+            return
+        handle = t.open_file_stream('foo')
+        try:
+            self.assertEqual('', t.get_bytes('foo'))
+        finally:
+            t.close_file_stream('foo')
+
+    def test_opening_a_file_stream_can_set_mode(self):
+        t = self.get_transport()
+        if t.is_readonly():
+            return
+        if not t._can_roundtrip_unix_modebits():
+            # Can't roundtrip, so no need to run this test
+            return
+        def check_mode(name, mode, expected):
+            handle = t.open_file_stream(name, mode=mode)
+            t.close_file_stream(name)
+            self.assertTransportMode(t, name, expected)
+        check_mode('mode644', 0644, 0644)
+        check_mode('mode666', 0666, 0666)
+        check_mode('mode600', 0600, 0600)
+        # The default permissions should be based on the current umask
+        check_mode('nomode', None, 0666 & ~osutils.get_umask())
+
     def test_copy_to(self):
         # FIXME: test:   same server to same server (partly done)
         # same protocol two servers
@@ -866,6 +916,11 @@
         # plain "listdir".
         # self.assertEqual([], os.listdir('.'))
 
+    def test_recommended_page_size(self):
+        """Transports recommend a page size for partial access to files."""
+        t = self.get_transport()
+        self.assertIsInstance(t.recommended_page_size(), int)
+
     def test_rmdir(self):
         t = self.get_transport()
         # Not much to do with a readonly transport
@@ -1527,6 +1582,17 @@
         self.assertEqual(d[2], (0, '0'))
         self.assertEqual(d[3], (3, '34'))
 
+    def test_get_with_open_file_stream_sees_all_content(self):
+        t = self.get_transport()
+        if t.is_readonly():
+            return
+        handle = t.open_file_stream('foo')
+        try:
+            handle('bcd')
+            self.assertEqual([(0, 'b'), (2, 'd')], list(t.readv('foo', ((0,1), (2,1)))))
+        finally:
+            t.close_file_stream('foo')
+
     def test_get_smart_medium(self):
         """All transports must either give a smart medium, or know they can't.
         """

=== modified file 'bzrlib/transport/__init__.py'
--- a/bzrlib/transport/__init__.py	2007-07-22 17:18:05 +0000
+++ b/bzrlib/transport/__init__.py	2007-08-05 05:38:15 +0000
@@ -66,6 +66,11 @@
 from bzrlib import registry
 
 
+# a dictionary of open file streams. Keys are absolute paths, values are
+# transport defined.
+_file_streams = {}
+
+
 def _get_protocol_handlers():
     """Return a dictionary of {urlprefix: [factory]}"""
     return transport_list_registry
@@ -313,6 +318,15 @@
         """
         raise NotImplementedError(self.clone)
 
+    def close_file_stream(self, relpath):
+        """Close a file stream at relpath.
+
+        :raises: NoSuchFile if there is no open file stream for relpath.
+        :seealso: open_file_stream.
+        :return: None
+        """
+        raise NotImplementedError(self.close_file_stream)
+
     def ensure_base(self):
         """Ensure that the directory this transport references exists.
 
@@ -460,6 +474,18 @@
             path = '/' + path
         return path
 
+    def recommended_page_size(self):
+        """Return the recommended page size for this transport.
+
+        This is potentially different for every path in a given namespace.
+        For example, local transports might use an operating system call to 
+        get the block size for a given path, which can vary due to mount
+        points.
+
+        :return: The page size in bytes.
+        """
+        return 4 * 1024
+
     def relpath(self, abspath):
         """Return the local path portion from a given absolute path.
 
@@ -820,6 +846,22 @@
             self.mkdir(path, mode=mode)
         return len(self._iterate_over(relpaths, mkdir, pb, 'mkdir', expand=False))
 
+    def open_file_stream(self, relpath, mode=None):
+        """Open a file stream at relpath.
+
+        A file stream is a callback which adds data to the file. Buffering
+        may occur internally until the stream is closed with close_file_stream.
+        Calls to readv or the get_* methods will be synchronised with any
+        internal buffering that may be present.
+
+        :seealso: close_file_stream.
+        :param relpath: The relative path to the file.
+        :param mode: The mode for the newly created file, 
+                     None means just use the default
+        :return: A write callback to add data to the file.
+        """
+        raise NotImplementedError(self.open_file_stream)
+
     @deprecated_method(zero_eleven)
     def append(self, relpath, f, mode=None):
         """Append the text in the file-like object to the supplied location.

=== modified file 'bzrlib/transport/chroot.py'
--- a/bzrlib/transport/chroot.py	2007-07-20 03:20:20 +0000
+++ b/bzrlib/transport/chroot.py	2007-08-05 05:38:15 +0000
@@ -89,9 +89,15 @@
     def append_file(self, relpath, f, mode=None):
         return self._call('append_file', relpath, f, mode)
 
+    def _can_roundtrip_unix_modebits(self):
+        return self.server.backing_transport._can_roundtrip_unix_modebits()
+
     def clone(self, relpath):
         return ChrootTransport(self.server, self.abspath(relpath))
 
+    def close_file_stream(self, relpath):
+        return self._call('close_file_stream', relpath)
+
     def delete(self, relpath):
         return self._call('delete', relpath)
 
@@ -133,6 +139,9 @@
     def mkdir(self, relpath, mode=None):
         return self._call('mkdir', relpath, mode)
 
+    def open_file_stream(self, relpath, mode=None):
+        return self._call('open_file_stream', relpath, mode)
+
     def put_file(self, relpath, f, mode=None):
         return self._call('put_file', relpath, f, mode)
 

=== modified file 'bzrlib/transport/decorator.py'
--- a/bzrlib/transport/decorator.py	2007-07-20 03:20:20 +0000
+++ b/bzrlib/transport/decorator.py	2007-08-05 05:38:15 +0000
@@ -65,12 +65,20 @@
         """See Transport.append_bytes()."""
         return self._decorated.append_bytes(relpath, bytes, mode=mode)
 
+    def _can_roundtrip_unix_modebits(self):
+        """See Transport._can_roundtrip_unix_modebits()."""
+        return self._decorated._can_roundtrip_unix_modebits()
+
     def clone(self, offset=None):
         """See Transport.clone()."""
         decorated_clone = self._decorated.clone(offset)
         return self.__class__(
             self._get_url_prefix() + decorated_clone.base, decorated_clone)
 
+    def close_file_stream(self, relpath):
+        """See Transport.close_file_stream."""
+        return self._decorated.close_file_stream(relpath)
+
     def delete(self, relpath):
         """See Transport.delete()."""
         return self._decorated.delete(relpath)
@@ -110,6 +118,10 @@
         """See Transport.mkdir()."""
         return self._decorated.mkdir(relpath, mode)
 
+    def open_file_stream(self, relpath, mode=None):
+        """See Transport.open_file_stream."""
+        return self._decorated.open_file_stream(relpath, mode=mode)
+
     def put_file(self, relpath, f, mode=None):
         """See Transport.put_file()."""
         return self._decorated.put_file(relpath, f, mode)
@@ -130,6 +142,10 @@
         """See Transport.list_dir()."""
         return self._decorated.list_dir(relpath)
 
+    def recommended_page_size(self):
+        """See Transport.recommended_page_size()."""
+        return self._decorated.recommended_page_size()
+
     def rename(self, rel_from, rel_to):
         return self._decorated.rename(rel_from, rel_to)
     

=== modified file 'bzrlib/transport/ftp.py'
--- a/bzrlib/transport/ftp.py	2007-07-20 18:59:29 +0000
+++ b/bzrlib/transport/ftp.py	2007-08-05 02:57:45 +0000
@@ -46,6 +46,7 @@
     )
 from bzrlib.trace import mutter, warning
 from bzrlib.transport import (
+    _file_streams,
     Server,
     ConnectedTransport,
     )
@@ -105,6 +106,10 @@
             self._set_connection(connection, credentials)
         return connection
 
+    def close_file_stream(self, relpath):
+        """See Transport.close_file_stream."""
+        del _file_streams[self.abspath(relpath)]
+
     def _create_connection(self, credentials=None):
         """Create a new connection with the provided credentials.
 
@@ -323,6 +328,22 @@
             self._translate_perm_error(e, abspath,
                 unknown_exc=errors.FileExists)
 
+    def open_file_stream(self, relpath):
+        """See Transport.open_file_stream."""
+        def append_data(bytes):
+            self.append_bytes(relpath, bytes)
+        self.put_bytes(relpath, "")
+        _file_streams[self.abspath(relpath)] = append_data
+        return append_data
+
+    def recommended_page_size(self):
+        """See Transport.recommended_page_size().
+
+        For FTP we suggest a large page size to reduce the overhead
+        introduced by latency.
+        """
+        return 64 * 1024
+
     def rmdir(self, rel_path):
         """Delete the directory at rel_path"""
         abspath = self._remote_path(rel_path)

=== modified file 'bzrlib/transport/http/__init__.py'
--- a/bzrlib/transport/http/__init__.py	2007-07-20 18:59:29 +0000
+++ b/bzrlib/transport/http/__init__.py	2007-08-05 01:47:30 +0000
@@ -294,6 +294,14 @@
             # After one or more tries, we get the data.
             yield start, data
 
+    def recommended_page_size(self):
+        """See Transport.recommended_page_size().
+
+        For HTTP we suggest a large page size to reduce the overhead
+        introduced by latency.
+        """
+        return 64 * 1024
+
     @staticmethod
     @deprecated_method(zero_seventeen)
     def offsets_to_ranges(offsets):

=== modified file 'bzrlib/transport/local.py'
--- a/bzrlib/transport/local.py	2007-08-02 03:17:46 +0000
+++ b/bzrlib/transport/local.py	2007-08-05 05:53:53 +0000
@@ -33,6 +33,7 @@
     osutils,
     urlutils,
     symbol_versioning,
+    transport,
     )
 from bzrlib.trace import mutter
 from bzrlib.transport import LateReadError
@@ -84,6 +85,11 @@
                 abspath = self.base
             return LocalTransport(abspath)
 
+    def close_file_stream(self, relpath):
+        """See Transport.close_file_stream."""
+        handle = transport._file_streams.pop(self.abspath(relpath))
+        handle.close()
+
     def _abspath(self, relative_reference):
         """Return a path for use in os calls.
 
@@ -138,6 +144,9 @@
 
         :param relpath: The relative path to the file
         """
+        canonical_url = self.abspath(relpath)
+        if canonical_url in transport._file_streams:
+            transport._file_streams[canonical_url].flush()
         try:
             path = self._abspath(relpath)
             return open(path, 'rb')
@@ -301,6 +310,14 @@
         """Create a directory at the given path."""
         self._mkdir(self._abspath(relpath), mode=mode)
 
+    def open_file_stream(self, relpath, mode=None):
+        """See Transport.open_file_stream."""
+        # initialise the file
+        self.put_bytes_non_atomic(relpath, "", mode=mode)
+        handle = open(self._abspath(relpath), 'wb')
+        transport._file_streams[self.abspath(relpath)] = handle
+        return handle.write
+
     def _get_append_file(self, relpath, mode=None):
         """Call os.open() for the given relpath"""
         file_abspath = self._abspath(relpath)

=== modified file 'bzrlib/transport/memory.py'
--- a/bzrlib/transport/memory.py	2007-07-20 03:20:20 +0000
+++ b/bzrlib/transport/memory.py	2007-08-05 02:57:45 +0000
@@ -36,6 +36,7 @@
     )
 from bzrlib.trace import mutter
 from bzrlib.transport import (
+    _file_streams,
     LateReadError,
     register_transport,
     Server,
@@ -89,6 +90,10 @@
         result._locks = self._locks
         return result
 
+    def close_file_stream(self, relpath):
+        """See Transport.close_file_stream."""
+        del _file_streams[self.abspath(relpath)]
+
     def abspath(self, relpath):
         """See Transport.abspath()."""
         # while a little slow, this is sufficiently fast to not matter in our
@@ -165,6 +170,14 @@
             raise FileExists(relpath)
         self._dirs[_abspath]=mode
 
+    def open_file_stream(self, relpath):
+        """See Transport.open_file_stream."""
+        def append_data(bytes):
+            self.append_bytes(relpath, bytes)
+        self.put_bytes(relpath, "")
+        _file_streams[self.abspath(relpath)] = append_data
+        return append_data
+
     def listable(self):
         """See Transport.listable."""
         return True

=== modified file 'bzrlib/transport/remote.py'
--- a/bzrlib/transport/remote.py	2007-07-30 14:36:04 +0000
+++ b/bzrlib/transport/remote.py	2007-08-05 02:57:45 +0000
@@ -127,6 +127,10 @@
         # No credentials
         return None, None
 
+    def close_file_stream(self, relpath):
+        """See Transport.close_file_stream."""
+        del transport._file_streams[self.abspath(relpath)]
+
     def is_readonly(self):
         """Smart server transport can do read/write file operations."""
         resp = self._call2('Transport.is_readonly')
@@ -213,6 +217,14 @@
             self._serialise_optional_mode(mode))
         self._translate_error(resp)
 
+    def open_file_stream(self, relpath):
+        """See Transport.open_file_stream."""
+        def append_data(bytes):
+            self.append_bytes(relpath, bytes)
+        self.put_bytes(relpath, "")
+        transport._file_streams[self.abspath(relpath)] = append_data
+        return append_data
+
     def put_bytes(self, relpath, upload_contents, mode=None):
         # FIXME: upload_file is probably not safe for non-ascii characters -
         # should probably just pass all parameters as length-delimited

=== modified file 'bzrlib/transport/sftp.py'
--- a/bzrlib/transport/sftp.py	2007-07-20 18:59:29 +0000
+++ b/bzrlib/transport/sftp.py	2007-08-05 05:38:15 +0000
@@ -53,6 +53,7 @@
         )
 from bzrlib.trace import mutter, warning
 from bzrlib.transport import (
+    _file_streams,
     local,
     register_urlparse_netloc_protocol,
     Server,
@@ -156,6 +157,11 @@
         super(SFTPTransport, self).__init__(base,
                                             _from_transport=_from_transport)
 
+    def close_file_stream(self, relpath):
+        """See Transport.close_file_stream."""
+        handle = _file_streams.pop(self.abspath(relpath))
+        handle.close()
+
     def _remote_path(self, relpath):
         """Return the path to be passed along the sftp protocol for relpath.
         
@@ -257,6 +263,14 @@
         except (IOError, paramiko.SSHException), e:
             self._translate_io_exception(e, path, ': error retrieving')
 
+    def recommended_page_size(self):
+        """See Transport.recommended_page_size().
+
+        For SFTP we suggest a large page size to reduce the overhead
+        introduced by latency.
+        """
+        return 64 * 1024
+
     def _sftp_readv(self, fp, offsets, relpath='<unknown>'):
         """Use the readv() member of fp to do async readv.
 
@@ -525,6 +539,28 @@
         """Create a directory at the given path."""
         self._mkdir(self._remote_path(relpath), mode=mode)
 
+    def open_file_stream(self, relpath, mode=None):
+        """See Transport.open_file_stream."""
+        # initialise the file to zero-length
+        # this is three round trips, but we don't use this 
+        # api more than once per write_group at the moment so 
+        # it is a tolerable overhead. Better would be to truncate
+        # the file after opening. RBC 20070805
+        self.put_bytes_non_atomic(relpath, "", mode)
+        abspath = self._remote_path(relpath)
+        # TODO: jam 20060816 paramiko doesn't publicly expose a way to
+        #       set the file mode at create time. If it does, use it.
+        #       But for now, we just chmod later anyway.
+        handle = None
+        try:
+            handle = self._get_sftp().file(abspath, mode='wb')
+            handle.set_pipelined(True)
+        except (paramiko.SSHException, IOError), e:
+            self._translate_io_exception(e, abspath,
+                                         ': unable to open')
+        _file_streams[self.abspath(relpath)] = handle
+        return handle.write
+
     def _translate_io_exception(self, e, path, more_info='',
                                 failure_exc=PathError):
         """Translate a paramiko or IOError into a friendlier exception.



More information about the bazaar-commits mailing list