Bug report

John Arbash Meinel john at arbash-meinel.com
Mon Dec 17 13:52:27 GMT 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alexander Belchenko wrote:
> John Arbash Meinel пишет:
>> Alexander Belchenko wrote:
>>> Vincent,
>>>
>>> What do you think, your patch for http readv to read answer by chunks,
>>> is applicable to sftp transport? Symptoms looks pretty similar at first
>>> glance.
>>>
>>> Alexander
>>
>> sftp already reads in chunks. So it is something different.
>>
>> I wonder if it has to do with Andrew's work to have the protocol write in
>> "chunked encoding".
> 
> Umm? Vlad used plain sftp protocol. It's unrelated to bzr smart server IMO.
> 
>>
>> Having memory problems with 1GB of RAM seems surprising.
>>
>> If he is interested in testing, I suppose he could try doing one of the:
>>
>> bzr branch -r 100, bzr pull -r 200, bzr pull -r 300, sort of things to
>> see if
>> that works.
> 
> This tricks works for my problem with smart server. IMO should works and
> for sftp too.
> 
> But will be nice if we can to force this behavior either automatically
> at transport level or from command line.
> 

If it works for sftp, then I'm curious if something else is going on. Also,
what version of paramiko are you using? (I recall that <1.6? would make
requests larger than the SFTP spec required implementations to support.)

Vincent has a decent "transportstats" plugin which could be interesting here.
I'm really surprised to see plain "sftp" failing. The last time I looked at the
code, no single request would be >32KB, though we might make a lot of async
requests for lots of ranges.

Maybe that is the problem, that we are queuing up too many requests at once.

We could check some of that from the log file. It should have a line like:

SFTP.readv() %s offsets => %s coalesced => %s requests

Which would at least tell us how many concurrent requests we are trying to make.

You could also try something like the attached patch. Which makes sure that we
don't asynchronously request more than 10MB of data at a time. (32KB * 300 =
9.6MB.)

John
=:->


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHZn8aJdeBCYSNAAMRAmBKAJ4kurGqClPLfW55DSipW05sYq7+IgCfdSB/
ZnEMQoEtfGF8auLLg1mv9dI=
=GUFA
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sftp_requests.diff
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20071217/2404e120/attachment-0001.diff 


More information about the bazaar mailing list