RFC: smart server protocol change for 0.16

[Martin Pool]

On 4/12/07, John Arbash Meinel <john at arbash-meinel.com> wrote:
> Robert Collins wrote:
> > One of the things I noticed while hacking on RemoteGraph was that error
> > handling really needs to be improved a little. After chatting with
> > Martin, we agreed that a nice improvement to the protocol would be that
> > all responses have a success/fail prefix.
> >
> > That is, rather than using 'ok' for boolean results and 'resultX' for
> > non boolean, we'd return responses like:
> > 'ok', 'True'
> > or
> > 'ok', 'resultX'
> > for success, and
> > 'error', 'ErrorClass', ...
> > for errors.
> >
> > This would make it unambigous about whether a given result is an error
> > or not, and allow us to centralise the error handling as it would no
> > longer be per-verb.
> >
> > If we do this consistently, its a API break - but I think its a
> > worthwhile one. We could add synonyms for all the current verbs which
> > return the older style reponses, but personally I dont think thats worth
> > it.
> >
> > -Rob
>
> I think this is much cleaner in the protocol, and worth breaking for.
> Better to break now and get a nice protocol (before it is widespread)
> than being hobbled from here out.
>
> But if we are going to break, I think we should include things like the
> client version that Martin has mentioned. (Of course, I've always been
> in favor of versioned commands, so my view is a bit skewed in this regard).

Here are the things I wanted to do:

 * include a protocol version, right at the start, in requests and
responses, distinct from versioning of any particular request, so that
if we change our mind about any of this stuff we can fix it later.  (A
bit like the http version marker.)

 * Also include the bzrlib version in each request and response

 * The request/response arguments should allow for passing lists and
dicts containing arbitrary strings.  I had thought to change them to
use bencode.

 * We should be able to treat the body as a file-like object so that
we don't need to hold the thing in memory on the sender or receiver.
On the network they can be a series of length-prefixed chunks.

 * Responses should be identified as either ok or an error.

I think in general if an error occurs on the server it should be
re-raised on the client.  Since we can't and don't want to marshall
arbitrary Python objects this may be a bit of a puzzle.  But most
exceptions have only a dict of string parameters, and passing the
classname plus that dict may be enough.  For exceptions that are
unexpected or unrepresentable we could pass a string form.

Looking at the current hpss code I think it may be possible to bring
this in without needing to change too much of the code for specific
commands.  We'll see.

I'd like to bring this in before 0.16 so may do something for it on the weekend.

It's probably possible to keep compatibility for old clients against
new servers by detecting the different start to the request.  A bit
harder to make the clients downgrade, but at this stage of the game I
think it's ok to require that the server be no older than the client.

-- 
Martin