bzr+ssh versus sftp
John Arbash Meinel
john at arbash-meinel.com
Fri Nov 30 01:20:40 GMT 2007
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Martin Pool wrote:
> As a point of information, I don't see much difference on my gutsy core2duo:
>
> mbp at grace% python -m timeit -s "from bzrlib.branch import Branch" -s
> "b = Branch.open('sftp://localhost/home/mbp/repo3/trunk/')" -s "t =
> b.repository.bzrdir.transport" "bytes =
> t.get_bytes('repository/packs/73c25a0766469cc1430aa89525bec910.pack')"
> 100 loops, best of 3: 4.66 msec per loop
>
> mbp at grace% python -m timeit -s "from bzrlib.branch import Branch" -s
> "b = Branch.open('bzr+ssh://localhost/home/mbp/repo3/trunk/')" -s "t =
> b.repository.bzrdir.transport" "bytes =
> t.get_bytes('repository/packs/73c25a0766469cc1430aa89525bec910.pack')"
> 100 loops, best of 3: 4.38 msec per loop
>
Yeah, my server is 244ms versus 388ms (with the latest paramiko). So it is also
within the noise floor. You might want to check that you are using a large
file. I really do think it is something on my Mac that I'm provoking.
Can Ian or Vincent try it out? I know they are both running IntelMac (and I'm
running PPC), so it may or may not show up. But it would be something to see if
it is worth investigating. I also know that I now have unfixable disk
corruption (as told to me by Disk Utility), so it could just be part of that.
However, doing:
(cat big_file | ssh server cat) > /dev/null
Shows me that I am able to send bytes at about 900KB/s over my wireless
network. My server is able to do that loopback at about 3.7MB/s (using
- -cBlowfish improves that to 4.6MB/s).
My laptop is able to loopback it at 3.96 MB/s.
So it really seems to be something we are doing weird in python (to compare,
the effective 1-way throughput is becoming 333KB/s while the above are testing
round-trip at almost 4MB/s).
Also of note, when using "bzr+ssh" my cpu graph goes mostly into sys overhead,
rather than user time.
This test:
time python -c "from bzrlib import transport;
t =
transport.get_transport('bzr+ssh://localhost/Users/jameinel/dev/bzr/.bzr/repository/packs');
b = t.get_bytes('dc7c3b52d3071cad77c4460671b80fda.pack')"
2.56s user 3.41s system 62% cpu 9.476 total
It also goes into a state where ^C will not kill the process.
I think it is probably that we are provoking the system by sending a 2MB chunk
of data. At the very least the system time is about 1.5x that of the user time.
So I *don't* think it is specifically a bug in the bzrlib code, other than that
we are provoking something in the OS. (using sftp changes it to: 0.75s user
0.85s system 47% cpu 3.381 total)
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD4DBQFHT2VoJdeBCYSNAAMRAh12AKC3jQPymtW1QYbbky8XSQio388TTgCY2p7d
2QfiCtcdwJF/XpldxZqv4Q==
=tGL3
-----END PGP SIGNATURE-----
More information about the bazaar
mailing list