bzr+ssh versus sftp

John Arbash Meinel john at arbash-meinel.com
Fri Nov 30 01:20:40 GMT 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin Pool wrote:
> As a point of information, I don't see much difference on my gutsy core2duo:
> 
> mbp at grace% python -m timeit -s "from bzrlib.branch import Branch" -s
> "b = Branch.open('sftp://localhost/home/mbp/repo3/trunk/')" -s "t =
> b.repository.bzrdir.transport" "bytes =
> t.get_bytes('repository/packs/73c25a0766469cc1430aa89525bec910.pack')"
> 100 loops, best of 3: 4.66 msec per loop
> 
> mbp at grace% python -m timeit -s "from bzrlib.branch import Branch" -s
> "b = Branch.open('bzr+ssh://localhost/home/mbp/repo3/trunk/')" -s "t =
> b.repository.bzrdir.transport" "bytes =
> t.get_bytes('repository/packs/73c25a0766469cc1430aa89525bec910.pack')"
> 100 loops, best of 3: 4.38 msec per loop
> 

Yeah, my server is 244ms versus 388ms (with the latest paramiko). So it is also
within the noise floor. You might want to check that you are using a large
file. I really do think it is something on my Mac that I'm provoking.

Can Ian or Vincent try it out? I know they are both running IntelMac (and I'm
running PPC), so it may or may not show up. But it would be something to see if
it is worth investigating. I also know that I now have unfixable disk
corruption (as told to me by Disk Utility), so it could just be part of that.

However, doing:

(cat big_file | ssh server cat) > /dev/null

Shows me that I am able to send bytes at about 900KB/s over my wireless
network. My server is able to do that loopback at about 3.7MB/s (using
- -cBlowfish improves that to 4.6MB/s).

My laptop is able to loopback it at 3.96 MB/s.

So it really seems to be something we are doing weird in python (to compare,
the effective 1-way throughput is becoming 333KB/s while the above are testing
round-trip at almost 4MB/s).

Also of note, when using "bzr+ssh" my cpu graph goes mostly into sys overhead,
rather than user time.

This test:
time python -c "from bzrlib import transport;
t =
transport.get_transport('bzr+ssh://localhost/Users/jameinel/dev/bzr/.bzr/repository/packs');

b = t.get_bytes('dc7c3b52d3071cad77c4460671b80fda.pack')"
2.56s user 3.41s system 62% cpu 9.476 total

It also goes into a state where ^C will not kill the process.
I think it is probably that we are provoking the system by sending a 2MB chunk
 of data. At the very least the system time is about 1.5x that of the user time.

So I *don't* think it is specifically a bug in the bzrlib code, other than that
we are provoking something in the OS. (using sftp changes it to: 0.75s user
0.85s system 47% cpu 3.381 total)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD4DBQFHT2VoJdeBCYSNAAMRAh12AKC3jQPymtW1QYbbky8XSQio388TTgCY2p7d
2QfiCtcdwJF/XpldxZqv4Q==
=tGL3
-----END PGP SIGNATURE-----



More information about the bazaar mailing list