prefetch still broken with readv and paramiko 1.6.1

[Robey Pointer]

On 25 Jul 2006, at 9:28, John Arbash Meinel wrote:

> Well, I've found a few more bugs in the readv/prefetching logic.

Would you try the current bzr trunk?  Located here:

	http://www.lag.net/paramiko/bzr/paramiko

I took a 90 degree turn, decided to stop explaining how readv is  
meant to be used, and  tried to adapt it to how you're using it.  So  
I'd appreciate feedback on whether that makes it work right for you.


> Further, I found a bug in 'readv()' if you are requesting large  
> ranges.
> It seems that 'sftp.read()' is only able to return a maximum range of
> 64KiB (65536 bytes)

On some servers it's as low as 32K.  That's a bug that currently  
paramiko doesn't chop up large readv requests into the correct chunk  
sizes.  I think it should be fixed in the trunk.

I still think that in general you won't want to use prefetch when you  
know you're just going to readv() a few sections of the file.  But  
that should at least work now and not be as huge a penalty as it was  
before.

[ What I mean by the above: Prefetch basically pretends like you did  
a read() of the entire file, and downloads the whole thing.  If you  
know you're only going to want to read *part* of the file, some of  
that download is wasted.  You'd be better off skipping the prefetch  
and just doing a readv().  I know I haven't been explaining this well. ]

robey