[MERGE] sftp readv performance tweaks (2x faster)

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Well, I ran scripts all weekend to figure out how fast I could make sftp
with my changes. And the attached image shows how it went.

The nicest number is pulling bzr.dev 1800 up to the lastest revision
over a remote connection. It takes 98s with bzr.dev, and 108s with
rsync. Yes, we beat rsync, though just barely, and everywhere else rsync
still wins.

I have a lot of data that I sorted through, which is available from:

http://bzr.arbash-meinel.com/branches/real-world-performance/jameinel

But what I came up with is that we should be sorting the readv requests
by position, combining them into as many as 200 consecutive hunks, and
combining hunks even if they are as 8192 bytes apart.

I can post lots of graphs as to how I came to this conclusion, but in
the interest of being brief...

In the general case, the updates I propose make operations over sftp at
least 50% faster, and in some cases it is a full 100%. I would like to
see this merged into bzr.dev for 0.9, because it just a tweak of readv().

Also, FTP and LocalTransport should benefit from this, but I haven't
done any tuning or benchmarking to see exactly how much. FTP could
probably be tuned if people really care. LocalTransport is not very
latency bound, so there probably isn't a lot of improvement to get there.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEu9aGJdeBCYSNAAMRAgcZAJ0YXNWNS6i4Vd9lzzXNLY31ZBcQmwCgm5lR
W72Y61XZO/OihpfTxlE8fFk=
=dAiy
-----END PGP SIGNATURE-----
-------------- next part --------------
A non-text attachment was scrubbed...
Name: summary.png
Type: image/png
Size: 33300 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060717/6c93661d/attachment.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: readv-combining.diff
Type: text/x-patch
Size: 12167 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060717/6c93661d/attachment.bin