sftp prefetch disabled

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robey Pointer wrote:
> 
> On 17 Jul 2006, at 10:30, John Arbash Meinel wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Robert disabled sftp prefetch a while ago, and I'm trying to figure
>> out why.
>>
>> Specifically revision id:
>> robertc at robertcollins.net-20060419233208-2ed6906796994316
>> date: Thu 2006-04-20 09:32:08 +1000
>>
>> Has the commit message:
>> ... Disable all SFTP prefetching for known paramikos - direct readv
>> support is now a TODO.
> 
> There were bugs in the early implementations of prefetch.  It looks like
> Robert just pushed the version requirement up; he didn't completely
> disable it.  As far as I can tell from the current code, prefetch is
> enabled for all recent versions of paramiko.
> 
> The version check could be changed to 1.6.  There wasn't a 1.5.5 and
> before 1.6 there were interaction issues if you used both readv() and
> prefetch() (which bzr often tries to do, unfortunately).  But the
> version check should probably stay; I don't think it's safe to assume
> that everyone has upgraded yet.

All readv requests start out by doing a 'get()' and then seeking on the
file. At one point I implemented a 'Transport._get_seekable' which
defaulted to Transport.get, but sftp could override to provide an object
that did not have prefetch enabled.
When I tested, it had no effect, but it turns out that was because it
wasn't actually enabled. :) (My paramiko is 1.5.2)

So we could enable prefetch for 1.5.2 on up, as long as we don't use it
with readv(), right?

> 
> 
>> I will probably do a little bit of testing to see what effect prefetch
>> has. (I thought I was, until I dug deep enough to see this). My gut
>> feeling is that for readv() it would be a net loss, but for a plain
>> 'get()' it would be good.
> 
> I agree with that assessment.  Currently readv() is implemented in
> paramiko as a prefetch() over a limited range.  That is, prefetch() is
> effectively readv(everything)  without waiting for the results.
> 
> robey
> 
> 

Does paramiko have an explicit readv() command? Right now we are using a
plain get followed by [seek + read(size), ...]

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEvke7JdeBCYSNAAMRAt/sAKCZSikyGEGbBSpzEigMKLyYJY1tEwCgh3KA
5TBBPiiOEWIph/Cig//+ktQ=
=vIKm
-----END PGP SIGNATURE-----