bzr log http+urllib does not work, http+pycurl is too slow

Alexander Belchenko bialix at ukr.net
Tue Dec 11 16:22:04 GMT 2007


I have Trac running as separate python application on separate win32 
machine.
I put pack shared repo in the Trac's htdocs directory, so my branches are
available via http protocol (I use custom port 8000).
Well, at least `bzr revno http://host:8000/chrome/site/branches/Logic` works
as expected. But not log command.

On my client machine I use custom bzr.exe 1.0rc3 compiled without pycurl, so
I encounter this bug.

bzr arguments: [u'--no-plugins', u'log', 
u'http://host:8000/chrome/site/branches/Logic']
encoding stdout as sys.stdout encoding 'cp866'
failed to import pycurl: No module named pycurl
failed to instantiate transport <bzrlib.registry._LazyObjectGetter 
object at 10da5a8, module='bzrlib.transport.http._pycurl' 
attribute='PyCurlTransport'> for 
'http://host:8000/chrome/site/branches/Logic': DependencyNotPresent()
http readv of 930af4714be1e33318bb630b8d626540.rix  offsets => 1 collapsed 1
http readv of 930af4714be1e33318bb630b8d626540.rix  offsets => 1 collapsed 1
http readv of 930af4714be1e33318bb630b8d626540.rix  offsets => 1 collapsed 1
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 10 
collapsed 3
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 14 
collapsed 5
Traceback (most recent call last):
   File "bzrlib\commands.pyc", line 802, in run_bzr_catch_errors
   File "bzrlib\commands.pyc", line 758, in run_bzr
   File "bzrlib\commands.pyc", line 492, in run_argv_aliases
   File "bzrlib\commands.pyc", line 768, in ignore_pipe
   File "bzrlib\builtins.pyc", line 1750, in run
   File "bzrlib\log.pyc", line 189, in show_log
   File "bzrlib\log.pyc", line 303, in _show_log
   File "bzrlib\log.pyc", line 290, in iter_revisions
   File "bzrlib\decorators.pyc", line 127, in read_locked
   File "bzrlib\repository.pyc", line 1003, in get_revisions
   File "bzrlib\decorators.pyc", line 127, in read_locked
   File "bzrlib\repository.pyc", line 1012, in _get_revisions
   File "bzrlib\store\revision\knit.pyc", line 88, in get_revisions
   File "bzrlib\store\revision\knit.pyc", line 104, in 
_get_serialized_revisions
   File "bzrlib\knit.pyc", line 1062, in get_texts
   File "bzrlib\knit.pyc", line 1068, in get_line_list
   File "bzrlib\knit.pyc", line 1086, in _get_content_maps
   File "bzrlib\knit.pyc", line 1051, in _get_record_map
   File "bzrlib\knit.pyc", line 2426, in read_records_iter
   File "bzrlib\knit.pyc", line 2033, in get_raw_records
   File "bzrlib\pack.pyc", line 253, in iter_records
   File "bzrlib\pack.pyc", line 294, in _read_format
   File "bzrlib\pack.pyc", line 221, in _read_line
   File "bzrlib\pack.pyc", line 185, in readline
   File "bzrlib\pack.pyc", line 172, in _next
   File "bzrlib\transport\http\__init__.pyc", line 236, in _readv
   File "bzrlib\transport\http\__init__.pyc", line 318, in _coalesce_readv
   File "bzrlib\transport\http\__init__.pyc", line 281, in get_and_yield
   File "bzrlib\transport\http\_urllib.pyc", line 135, in _get
   File "bzrlib\transport\http\_urllib.pyc", line 75, in _perform
   File "bzrlib\transport\http\_urllib2_wrappers.pyc", line 170, in 
cleanup_pipe
   File "bzrlib\transport\http\_urllib2_wrappers.pyc", line 137, in finish
   File "httplib.pyc", line 529, in read
   File "socket.pyc", line 309, in read
error: (10055, 'No buffer space available')

return code 4


With pycurl installed I have another error:

bzr arguments: [u'--no-plugins', u'log', 
u'http://host:8000/chrome/site/branches/Logic']
encoding stdout as sys.stdout encoding 'cp866'
got pycurl error: 6, Could not resolve host: host (Domain name not 
found), (6, 'Could not resolve host: host (Domain name not found)'), 
url: http://host:8000/chrome/site/branches/Logic/.bzr/branch-format
Traceback (most recent call last):
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\commands.py", line 
802, in run_bzr_catch_errors
     return run_bzr(argv)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\commands.py", line 
758, in run_bzr
     ret = run(*run_argv)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\commands.py", line 
492, in run_argv_aliases
     return self.run(**all_cmd_args)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\commands.py", line 
768, in ignore_pipe
     result = func(*args, **kwargs)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\builtins.py", line 
1695, in run
     location)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\bzrdir.py", line 679, 
in open_containing_tree_or_branch
     bzrdir, relpath = klass.open_containing(location)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\bzrdir.py", line 634, 
in open_containing
     return BzrDir.open_containing_from_transport(transport)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\bzrdir.py", line 655, 
in open_containing_from_transport
     result = BzrDir.open_from_transport(a_transport)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\bzrdir.py", line 609, 
in open_from_transport
     redirected)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\lazy_import.py", line 
125, in __call__
     return obj(*args, **kwargs)
   File 
"C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\transport\__init__.py", line 
1661, in do_catching_redirections
     return action(transport)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\bzrdir.py", line 586, 
in find_format
     transport, _server_formats=_server_formats)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\bzrdir.py", line 
1307, in find_format
     return format.probe_transport(transport)
   File "C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\bzrdir.py", line 
1317, in probe_transport
     format_string = transport.get(".bzr/branch-format").read()
   File 
"C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\transport\http\__init__.py", 
line 134, in get
     code, response_file = self._get(relpath, None)
   File 
"C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\transport\http\_pycurl.py", 
line 163, in _get
     return self._get_full(relpath)
   File 
"C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\transport\http\_pycurl.py", 
line 198, in _get_full
     self._curl_perform(curl, header)
   File 
"C:\work\Bazaar\releases\bzr-1.0rc3\bzrlib\transport\http\_pycurl.py", 
line 321, in _curl_perform
     'curl connection error (%s)\non %s' % (e[1], url))
ConnectionError: Connection error: curl connection error (Could not 
resolve host: acad (Domain name not found))
on http://host:8000/chrome/site/branches/Logic/.bzr/branch-format

return code 3

If I use real IP in the URL I finally get log but it's very-very slow 
(comparing to smart server or direct access to branch via windows UNC 
network). But using real IP does not help when I use http+urllib URL.

May be slowness caused by Trac sever, I'm not sure. Remote machine is 
old and slow (CPU 750MHz). But fast enough when I use bzr:// protocol. 
In absolute numbers:

bzr revno http://.... -> 2.7 seconds
bzr revno bzr://....  -> 2.4 seconds
bzr revno file://...  -> 0.75 seconds

bzr log http://... -> 187.766 seconds
bzr log bzr://.... -> 8.2 seconds
bzr log file://... -> 7.6 seconds


Here the relevant part of .bzr.log corresponding to executing bzr log 
http://... command:

bzr arguments: [u'--no-plugins', u'log', 
u'http+pycurl://X.X.X.X:8000/chrome/site/branches/Logic']
encoding stdout as sys.stdout encoding 'cp866'
http readv of 930af4714be1e33318bb630b8d626540.rix  offsets => 1 collapsed 1
http readv of 930af4714be1e33318bb630b8d626540.rix  offsets => 1 collapsed 1
http readv of 930af4714be1e33318bb630b8d626540.rix  offsets => 1 collapsed 1
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 10 
collapsed 3
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 14 
collapsed 5
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 20 
collapsed 2
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 29 
collapsed 3
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 43 
collapsed 2
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 64 
collapsed 3
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 95 
collapsed 2
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 142 
collapsed 2
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 201 
collapsed 3
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 201 
collapsed 5
http readv of 930af4714be1e33318bb630b8d626540.pack  offsets => 134 
collapsed 3
return code 0

The size of 930af4714be1e33318bb630b8d626540.rix is 177KB, 
930af4714be1e33318bb630b8d626540.pack -- 112 MB.
Shared repository was repacked with `bzr pack` commad.

Do you think I need to run some more tests in my configuration?



More information about the bazaar mailing list