[MERGE] Updated sftp_readv

Vincent Ladeuil v.ladeuil+lp at free.fr
Wed Dec 19 07:34:18 GMT 2007


>>>>> "john" == John Arbash Meinel <john at arbash-meinel.com> writes:

<snip/>

    john> I'm also surprised to see that huge of a difference. If I'm counting correctly,
    john> it makes it about 5x the total repository size. And I certainly don't see 5
    john> copies in the code.

    john> 2x seems unavoidable (all = ''.join(data) has to have 2 copies), and I suppose
    john> if there were multiple coalesced ranges and 'all' was still lying around that
    john> would be 3 copies. And then splitting that data into its ranges would be a 4th
    john> copy. I still don't see a 5th copy. I wonder if that is in higher level code.

I would "blame" paramiko instead (awaking, before coffee, guess ;),
I think Vlad is using a sftp server on a local network and since
_sftp_readv send all requests to paramiko at once before
iterating the *results*, all sftp requests mat be fulfilled
faster than _sftp_readv can consume them.

Threading the requests so that they can occur asynchronously is
*good* for high latencies, but may be it should be limited a bit
for high ones.

       Vincent



More information about the bazaar mailing list