Paramiko throws EOFError rather than returning a 0 length file or ENOENT

[Vincent Ladeuil]

>>>>> "Robert" == Robert Collins <robertc at robertcollins.net> writes:

    Robert> On Wed, 2006-09-13 at 09:37 +0200, Vincent Ladeuil wrote:
    >> 
    >> Yes it can. But I was a bit vague, I'll try to clarify for the
    >> urllib2/httplib context which happen to be *our* case:
    >> 
    >> - httplib implements a pipelining of request/responses, so in
    >> theory, you could issue a second request while processing the
    >> first response, that means you should guard, not only against
    >> short reads but also against too long reads (i.e. you MUST not
    >> read part of the second response),
    >> 
    >> - urllib2 simplify the problem by requiring the server to close
    >> the connection at each request, hence, no risk of reading a
    >> second response (this is what _urllib does today for bzr). 

    Robert> There are many more things than that that can go
    Robert> wrong.

Indeed, that's why I told John that I want to improve the test
suite. I have just completed an integration of the read-only part
of the webdav plugin (as of John suggestion) into the
HttpTransport_urllib and all the tests pass. But I clearly feel
holes here and there if we encounter a "strange" http server (for
fuzzy values of strange).

    Robert> With HTTP/1.1 you must support chunked encoding,
    Robert> which gives a clear EOF indication for each request.

The protocol gives a clear definition, yes. Implementing that
clear EOF indication detection is a bit more complex than it
appears but I've made good progress on that front* and I'm
confident we can achieve that.

    Robert> You also have to be willing to resubmit the second
    Robert> request if the server signals EOF or ERROR on the
    Robert> first response by closing the connection. pipelining
    Robert> of non idempotent requests is extremly tricky on
    Robert> http. pipelining of range requests is possible but
    Robert> prone to arbitrary delays when the
    Robert> server/intermediaries decide to do a full object
    Robert> response.

We don't do pipelining *yet*, to be precise, urllib2 don't, and
in the implementation I propose neither do I. So we are safe on
that side for now.

If we decide to do pipelining in the future, I think we can do so
with a conservative approach by being overly restrictive on what
we consider idempotent or not. And from there fine tune our
choices for potential optimizations

    Robert> Persistent connections are an optional feature in
    Robert> HTTP/1.0 and only spottily implemented.

Indeed, part of the tests I want to add is conformance to various
levels of http protocol, again we can be conservative and just
avoid persistent connections for 1.0 (as urllib2 already does
even for 1.1...).

Also on that subject, I'd like to use the test suite against
*real* servers and gather as much data as possible. It's a bit
vague, I know, so any comments appreciated. That would help me
un-fuzzy the strangeness :-)

    Robert> I could go on :(

Please do ! Then I would just have to implement the tests instead
of stressing myself about what *else* could go wrong :-)

   Vincent

* Not without having some nightmares :)