Rev 3951: Fix debug handling for pycurl and implement pycurl http activity in file:///net/bigmamac/Volumes/home/vila/src/bzr/experimental/pb-http/

[Vincent Ladeuil]

At file:///net/bigmamac/Volumes/home/vila/src/bzr/experimental/pb-http/

------------------------------------------------------------
revno: 3951
revision-id: v.ladeuil+lp at free.fr-20090129165458-o52vbf2nl4dxadoc
parent: v.ladeuil+lp at free.fr-20090129142728-1iu8017zgso23w0i
committer: Vincent Ladeuil <v.ladeuil+lp at free.fr>
branch nick: pb-http
timestamp: Thu 2009-01-29 17:54:58 +0100
message:
  Fix debug handling for pycurl and implement pycurl http activity
  reporting at socket level.
  
  * bzrlib/transport/http/_pycurl.py:
  (PyCurlTransport._debug_cb): Two birds with one stone: correctly
  handle the debug traces with mutter (no more stderr pollution !)
  and report activity for all the transmitted headers and data in
  both directions.
  (PyCurlTransport._set_curl_options): Setup a pycurl debug callback
  for both activity reporting and http debug flag handling.
-------------- next part --------------
=== modified file 'BRANCH.TODO'
--- a/BRANCH.TODO	2009-01-29 14:27:28 +0000
+++ b/BRANCH.TODO	2009-01-29 16:54:58 +0000
@@ -3,8 +3,8 @@
 # 
 #
 
-* try to address pycurl failing activity tests or raise not
-   applicable otherwise
+* add more tests for has and readv (_get_full wasn't addressed
+   by jam's patch but _get_ranged should be reported twice currently).
 
 * complete _ReportingFileSocket implementation for write
    operations ?

=== modified file 'NEWS'
--- a/NEWS	2009-01-24 04:03:08 +0000
+++ b/NEWS	2009-01-29 16:54:58 +0000
@@ -12,6 +12,10 @@
     * Progress bars now show the rate of activity for some sftp 
       operations, and they are drawn different.  (Martin Pool, #172741)
 
+    * Progress bars now show the rate of activity for some^H *all* hhtp ;)
+      operations for both urllib and pycurl implementations.
+      (Vincent Ladeuil)
+
   BUG FIXES:
 
     * There was a bug in how we handled resolving when a file is deleted

=== modified file 'bzrlib/transport/http/_pycurl.py'
--- a/bzrlib/transport/http/_pycurl.py	2009-01-23 21:22:39 +0000
+++ b/bzrlib/transport/http/_pycurl.py	2009-01-29 16:54:58 +0000
@@ -215,8 +215,8 @@
 
     # The parent class use 0 to minimize the requests, but since we can't
     # exploit the results as soon as they are received (pycurl limitation) we'd
-    # better issue more requests and provide a more responsive UI do the cost
-    # of more latency costs.
+    # better issue more requests and provide a more responsive UI incurring
+    # more latency costs.
     # If you modify this, think about modifying the comment in http/__init__.py
     # too.
     _get_max_size = 4 * 1024 * 1024
@@ -307,15 +307,28 @@
             raise errors.InvalidHttpResponse(
                 url, 'Unable to handle http code %d%s' % (code,msg))
 
+    def _debug_cb(self, kind, text):
+        if kind in (pycurl.INFOTYPE_HEADER_IN, pycurl.INFOTYPE_DATA_IN,
+                    pycurl.INFOTYPE_SSL_DATA_IN):
+            self._report_activity(len(text), 'read')
+            if (kind == pycurl.INFOTYPE_HEADER_IN
+                and 'http' in debug.debug_flags):
+                mutter('< %s' % text)
+        elif kind in (pycurl.INFOTYPE_HEADER_OUT, pycurl.INFOTYPE_DATA_OUT,
+                      pycurl.INFOTYPE_SSL_DATA_OUT):
+            self._report_activity(len(text), 'write')
+            if (kind == pycurl.INFOTYPE_HEADER_OUT
+                and 'http' in debug.debug_flags):
+                mutter('> %s' % text)
+        elif kind == pycurl.INFOTYPE_TEXT and 'http' in debug.debug_flags:
+            mutter('* %s' % text)
+
     def _set_curl_options(self, curl):
         """Set options for all requests"""
-        if 'http' in debug.debug_flags:
-            curl.setopt(pycurl.VERBOSE, 1)
-            # pycurl doesn't implement the CURLOPT_STDERR option, so we can't
-            # do : curl.setopt(pycurl.STDERR, trace._trace_file)
-
         ua_str = 'bzr/%s (pycurl: %s)' % (bzrlib.__version__, pycurl.version)
         curl.setopt(pycurl.USERAGENT, ua_str)
+        curl.setopt(pycurl.VERBOSE, 1)
+        curl.setopt(pycurl.DEBUGFUNCTION, self._debug_cb)
         if self.cabundle:
             curl.setopt(pycurl.CAINFO, self.cabundle)
         # Set accepted auth methods