Rev 4890: Change the buffering algorithm for ProtocolThree. in http://bazaar.launchpad.net/~jameinel/bzr/2.1.0b4-proto3-buffering

[John Arbash Meinel]

At http://bazaar.launchpad.net/~jameinel/bzr/2.1.0b4-proto3-buffering

------------------------------------------------------------
revno: 4890
revision-id: john at arbash-meinel.com-20091211222858-7k63etcp980w0y0b
parent: pqm at pqm.ubuntu.com-20091210235739-zwgaqsbb2mk57gyv
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: 2.1.0b4-proto3-buffering
timestamp: Fri 2009-12-11 16:28:58 -0600
message:
  Change the buffering algorithm for ProtocolThree.
  
  Instead of flushing when we have had 100 write calls, flush when we have 1MB.
  There are a lot of potential times when we could flush.
  osutils.send_all() splits up the request into 64kB chunks. So we could
  buffer to 64kB and then call flush. My concern there is for 'rounding'
  errors. Where we end up with 65kB and then we have a nice big 64kB
  send, and then a tiny 1kB send. Hopefully this will be a little bit better.
  
  Another option which I might try, is to flush at >64kB, but round to a
  64kB boundary, and keep any bytes that are past for the future.
-------------- next part --------------
=== modified file 'bzrlib/smart/protocol.py'
--- a/bzrlib/smart/protocol.py	2009-10-21 06:05:49 +0000
+++ b/bzrlib/smart/protocol.py	2009-12-11 22:28:58 +0000
@@ -1065,17 +1065,20 @@
 
     def __init__(self, write_func):
         self._buf = []
+        self._buf_len = 0
         self._real_write_func = write_func
 
     def _write_func(self, bytes):
         self._buf.append(bytes)
-        if len(self._buf) > 100:
+        self._buf_len += len(bytes)
+        if self._buf_len > 1*1024*1024:
             self.flush()
 
     def flush(self):
         if self._buf:
             self._real_write_func(''.join(self._buf))
             del self._buf[:]
+            self._buf_len = 0
 
     def _serialise_offsets(self, offsets):
         """Serialise a readv offset list."""