Too much data being downloaded with sftp

Sowers, Brent - ES Brent.Sowers at itt.com
Thu Sep 9 16:37:01 BST 2010


I'm using sftp with Bazaar to push, pull, branch, and checkout from branches on our central server.  I've noticed that the amount of data transferred when checking out or branching to a new system has been getting bigger and bigger.  I just upgraded to 2.2.0 thinking that maybe this would fix the problem, but a ton of data is still being transferred, maybe more than with 2.1.0.  My repository isn't too big, maybe 300 commits total, and when I look at the disk usage on the server, the directory containing the entire repository is around 20 megs.  But when I do a checkout from one of the branches on to a new system, the total amount of data transferred is 430 megs!

I ran a checkout on a new system with -Dhpss on the command line, and here is what gets put in to my .bzr.log.  I've substituted the server name here.  The repository name is lp.
Thu 2010-09-09 11:11:15 -0400
0.221  bazaar version: 2.2.0
0.221  bzr arguments: [u'checkout', u'sftp://www.server-name.com/opt/bzr/lp/brent/refactor', u'lp', u'-Dhpss']
0.334  looking for plugins in /home/brent/.bazaar/plugins
0.334  looking for plugins in /usr/lib/python2.5/site-packages/bzrlib/plugins
0.571  encoding stdout as sys.stdout encoding 'UTF-8'
0.721  ssh implementation is OpenSSH
3.758  creating repository in file:///opt/lh/lp/.bzr/.
3.761  creating branch <bzrlib.branch.BzrBranchFormat7 object at 0x858a74c> in file:///opt/lh/lp/
3.875  Using fetch logic to copy between CHKInventoryRepository('sftp://www.server-name.com/opt/bzr/.bzr/repository/')(RepositoryFormat
2a()) and CHKInventoryRepository('file:///opt/lh/lp/.bzr/repository/')(RepositoryFormat2a())
3.875  fetch up to rev {brent.sowers at itt.com-20100908193331-ymgpwwif2a13htxm<mailto:%7Bbrent.sowers at itt.com-20100908193331-ymgpwwif2a13htxm>}
10.733  Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at 0x8b6ed2c>, 364238, 3507449) to an LRUSizeCache failed. value 87303229 is too big to fit in a the cache with size 41943040 52428800
... the previous line is repeated a ton of times. The numbers are the exact same on every line ...
44.195  Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at 0x8b6ed2c>, 364238, 3507449) to an LRUSizeCache failed. value
87303229 is too big to fit in a the cache with size 41943040 52428800
49.617  trying to create missing lock '/opt/lh/lp/.bzr/checkout/dirstate'
49.618  opening working tree '/opt/lh/lp'
51.234  Transferred: 431148kB (8913.1kB/s r:431148kB w:0kB)
51.234  return code 0
Any help on what's going on here would be appreciated.  Going over a gigabit LAN as I was for this example isn't too much of a hassle, but doing this over a T1 line is excruciating.  I just don't understand how so much data is being transferred when the entire repository on the server is only 20 megs.

________________________________
This e-mail and any files transmitted with it may be proprietary and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the sender.
Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of ITT Corporation. The recipient should check this e-mail and any attachments for the presence of viruses. ITT accepts no liability for any damage caused by any virus transmitted by this e-mail.



More information about the bazaar mailing list