[MERGE][RFC] Proposed version 3 of smart server protocol

[Martin Pool]

(I'm working from the document I last saw, so this may be out of date.)

One thing this leaves open is that there is no way for the receiver
(which could be either client or server) to interrupt a streaming
message body.  It has to always receive the whole thing, or drop the
connection.

The opposite case, of the sender discovering that it cannot continue
streaming, is already handled in Andrew's spec.

It might want to interrupt streaming in these cases
 * it discovers it has enough information and needs no more
 * it wants to send a correction to the previous request, for example
to narrow the range
 * an error occurs in the receiver
 * the user's actions make the message obsolete - either by an
interrupt or perhaps by moving elsewhere in the gui

In a substantial number of these cases, the process is being
terminated and we will drop the connection soon anyhow.  Still, it
might be nice to give a proper message beyond "connection closed", for
UI, for debugging, and in case the sender program wants to act on the
error.

Some of these cases are admittedly not very likely without a
twisted-style or threaded stack.

It is plausible we could handle this by, eg, having the client ack
each packet (with some slack window), though that has a risk of
unnecessary stalls, or having the server look for an incoming
cancellation without blocking to read it.  It would probably have to
handle the case that what is actually received is the sender's next
message; or conversely that the client's cancellation arrives after
the sender has finished.  (We cannot rely on tcp urgent data, which is
meant for such cases, amongst other reasons because it's not passed
over ssh tunnels.)

I'm not sure if this would map well to http.