[MERGE][#178353] Smart request to stream revisions without buffering

[John Arbash Meinel]

John Arbash Meinel has voted tweak.
Status is now: Conditionally approved
Comment:
1) This seems like it can only break up data into "chunks" with a 
knit-hunk granularity. So if you add a 100MB file, then it will send a 
100MB chunk. Conversely, if you have 100 2-byte files, it will send 100 
"chunks". I'm okay with it, but it does seem like separating the 
chunking layer from the data your are transmitting would let you do 
something like "always 1MB chunks".

2) I still don't see anything about handling a Knit stream put into a 
Pack repository, etc. (Or maybe more explicitly requesting the kind of 
stream you want.)

3)

=== modified file 'doc/developers/index.txt'
--- doc/developers/index.txt    2007-12-05 18:58:49 +0000
+++ doc/developers/index.txt    2008-01-02 03:08:59 +0000
@@ -34,7 +34,6 @@
  * `Network protocol <network-protocol.html>`_ |--| Custom network 
protocol.

  Data formats
-============

  * `Knit pack repositories <packrepo.html>`_ |--| KnitPack repositories
    (new in Bazaar 0.92).

^- This doesn't seem like a valid change. I realize there is only one 
entry, but it seems like they should be in their own (sub)section.

4)
+                try:
+                    # These records should have only one name, and that 
name
+                    # should be a one-element tuple.
+                    [name_tuple] = record_names
+                except ValueError:
+                    raise errors.SmartProtocolError(
+                        'Repository data stream had invalid record name 
%r'
+                        % (record_names,))

Odd construct, why not just do:

if len(record_names) != 1:
   raise ...

Using [foo] = bar as you have is... odd. At least you could use a tuple 
instead with (foo,) = bar or for extra points "foo, = bar".  (No I'm not 
advocating it, but showing why I think it is a poor way to do it.)



Otherwise I'm happy to see that it wasn't terrible to get a genuinely 
streaming send. The buffering was pretty bad on large repositories. I am 
curious about the overhead of the chunk headers, but they generally seem 
small. (A hex length and a newline is log16(data)+1 extra bytes.)


For details, see: 
http://bundlebuggy.aaronbentley.com/request/%3C20080102225927.GD8365%40steerpike.home.puzzling.org%3E