Too much data being downloaded with sftp
Sowers, Brent - ES
Brent.Sowers at itt.com
Thu Sep 9 16:37:01 BST 2010
I'm using sftp with Bazaar to push, pull, branch, and checkout from branches on our central server. I've noticed that the amount of data transferred when checking out or branching to a new system has been getting bigger and bigger. I just upgraded to 2.2.0 thinking that maybe this would fix the problem, but a ton of data is still being transferred, maybe more than with 2.1.0. My repository isn't too big, maybe 300 commits total, and when I look at the disk usage on the server, the directory containing the entire repository is around 20 megs. But when I do a checkout from one of the branches on to a new system, the total amount of data transferred is 430 megs!
I ran a checkout on a new system with -Dhpss on the command line, and here is what gets put in to my .bzr.log. I've substituted the server name here. The repository name is lp.
Thu 2010-09-09 11:11:15 -0400
0.221 bazaar version: 2.2.0
0.221 bzr arguments: [u'checkout', u'sftp://www.server-name.com/opt/bzr/lp/brent/refactor', u'lp', u'-Dhpss']
0.334 looking for plugins in /home/brent/.bazaar/plugins
0.334 looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.571 encoding stdout as sys.stdout encoding 'UTF-8'
0.721 ssh implementation is OpenSSH
3.758 creating repository in file:///opt/lh/lp/.bzr/.
3.761 creating branch <bzrlib.branch.BzrBranchFormat7 object at 0x858a74c> in file:///opt/lh/lp/
3.875 Using fetch logic to copy between CHKInventoryRepository('sftp://www.server-name.com/opt/bzr/.bzr/repository/')(RepositoryFormat
2a()) and CHKInventoryRepository('file:///opt/lh/lp/.bzr/repository/')(RepositoryFormat2a())
3.875 fetch up to rev {brent.sowers at itt.com-20100908193331-ymgpwwif2a13htxm<mailto:%7Bbrent.sowers at itt.com-20100908193331-ymgpwwif2a13htxm>}
10.733 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at 0x8b6ed2c>, 364238, 3507449) to an LRUSizeCache failed. value 87303229 is too big to fit in a the cache with size 41943040 52428800
... the previous line is repeated a ton of times. The numbers are the exact same on every line ...
44.195 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at 0x8b6ed2c>, 364238, 3507449) to an LRUSizeCache failed. value
87303229 is too big to fit in a the cache with size 41943040 52428800
49.617 trying to create missing lock '/opt/lh/lp/.bzr/checkout/dirstate'
49.618 opening working tree '/opt/lh/lp'
51.234 Transferred: 431148kB (8913.1kB/s r:431148kB w:0kB)
51.234 return code 0
Any help on what's going on here would be appreciated. Going over a gigabit LAN as I was for this example isn't too much of a hassle, but doing this over a T1 line is excruciating. I just don't understand how so much data is being transferred when the entire repository on the server is only 20 megs.
________________________________
This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender.
Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of ITT Corporation. The recipient should check this e-mail and any attachments for the presence of viruses. ITT accepts no liability for any damage caused by any virus transmitted by this e-mail.
More information about the bazaar
mailing list