prefetch still broken with readv and paramiko 1.6.1

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robey Pointer wrote:
> 
> On 26 Jul 2006, at 5:14, John Arbash Meinel wrote:

...

> I didn't mean that to come out sounding snotty.  What I meant was:
> Instead of trying to tell people how I think they should do things, I
> decided to let people do what they think is best and try to make that
> work the way they think it ought to.
> 
> 
>>
>>> I still think that in general you won't want to use prefetch when you
>>> know you're just going to readv() a few sections of the file.  But that
>>> should at least work now and not be as huge a penalty as it was before.
>>>
>>
>> I'm not using readv() with prefetch(). I do understand they overlap.
>> What I was *testing* is that if you are downloading *most* of the file
>> using seek+read, it is actually faster to do a prefetch(). But if you
>> aren't downloading most of the file, it is much better to just read the
>> sections you want.
>>
>> And the best is to use readv() which does an async request for just the
>> sections you want.
> 
> Sorry, I'd thought one of your bug reports was related to using
> prefetch() immediately before readv().
> 

Well, there is a bug in bzr-0.8.2, which means that it is incompatible
with paramiko 1.6.1. It issues a prefetch(), and then uses seek+read but
not in order. Which *really* messes with paramiko's mind.

As long as things were in order prefetch() + seek+read, or
prefetch+readv worked fine.

...

> I also fixed it (knock on wood) so that you can do a prefetch() and then
> readv() without setting off fireworks.  It should even try to use
> prefetch buffers when a readv overlaps with chunks of the file that have
> been prefetched (or are in the process of being prefetched) but haven't
> been read by the app yet.
> 
> robey

See my recent post on the performance implications of your latest changes.

For now, I'll submit a review of my bzr implementation which breaks up
the chunks itself. This should be compatible with any paramiko
implementation that has readv. (In case someone has paramiko-1.6 or
1.6.1 installed) If they have a version without readv(), it just reverts
back to seek+read. Which is better than what we had, but still better
than the old code.

Interestingly enough bzr.dev is slower than bzr-0.8, but I think that is
because of the Inventory caching stuff, which I just submitted the fix.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEx8DxJdeBCYSNAAMRAowuAJ90QBwxXyCDySneq9XO2vtQ28DDaACfZfHM
I7+T6WF32KCAQv70CYibBE0=
=YWd9
-----END PGP SIGNATURE-----