[MERGE][RFC] Proposed version 3 of smart server protocol

[Andrew Bennetts]

John Arbash Meinel wrote:
> Andrew Bennetts wrote:
>> Hi all,
>> This bundle adds a section to network-protocol.txt describing new version 
>> of the
>> smart protocol.  If you think there's some other change to the protocol 
>> that
>> should be taken into account, now would be a good time to mention it ;)
>> The text should be pretty self-explanatory.  Let me know if it isn't.
>> -Andrew.
>
> If we are going to use bencode, do we want to extend it with a "unicode" 
> type. That way you don't have to worry as much about making sure you drop 
> your Unicode down to utf-8 and then back up again. It would just be a 
> convenience thing but right now:
>
> >>> bzrlib.util.bencode.bencode(u'foo')
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python2.5/site-packages/bzrlib/util/bencode.py", line 142, 
> in bencode
>     encode_func[type(x)](x, r)
> KeyError: <type 'unicode'>

Well, right now we just take plain bytes on the wire, so this isn't a step
backwards.  We can certainly add some convenience functions, but I think we
probably want to keep handling byte-strings in many cases, to avoid unnecessary
encoding and decoding to and from UTF-8.

> I don't know that bencoded is really the best format for it. As bencode 
> doesn't give overall lengths. It tells you how many items are in a list,  
> and how long a string is, but it doesn't tell you how many bytes your list 
> is, nor how many bytes your integer will be, etc.
>
> I realize we may have a fix for pipelining anyway, but bencode only gives 
> us lengths for strings. Which seems a bit... minimal. Especially 
> considering every request is going to have at least one "bencoded_sequence" 
> which I assume will be a list.
>
>
> You are making the LENGTH_PREFIXED_BODY start to take: "32-bit unsigned 
> integer in network byte order", so the protocol is no longer human-readable 
> anyway. It seems reasonable to bring that up into the REQUEST_ARGS.

You're right, one thing I meant to add in but forgot was a LENGTH_PREFIX before
each bencoded part, for the length of the bencoded part.  The intention is that
once the protocol version line is read, all variable-length bits of the protocol
will have length-prefixes.

> A small comment as to what you changed in the default css would also be 
> nice.

In the CSS?  I'll do that.  (To satisfy your curiosity immediately, it was for
the “.. note:” section).

> BB:tweak

Thanks!

-Andrew.