Rev 1869: (jam, mpee,
jrydberg) update http with keep-alive and multi-range requests.
http operations 2-4x faster in
/home/pqm/archives/thelove/bzr/+trunk/
Canonical.com Patch Queue Manager
pqm at pqm.ubuntu.com
Wed Jul 19 21:32:38 BST 2006
------------------------------------------------------------
revno: 1869
revision-id: pqm at pqm.ubuntu.com-20060719203238-892f369155dca5dc
parent: pqm at pqm.ubuntu.com-20060718202456-fc3ad12fe5b7abd0
parent: john at arbash-meinel.com-20060719180226-ce7d12331fedbb70
committer: Canonical.com Patch Queue Manager<pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Wed 2006-07-19 21:32:38 +0100
message:
(jam, mpee, jrydberg) update http with keep-alive and multi-range requests. http operations 2-4x faster
added:
bzrlib/tests/test_http_response.py test_http_response.py-20060628233143-950b2a482a32505d
bzrlib/transport/http/_pycurl_errors.py _pycurl_errors.py-20060714003436-4zste49ltg697ft9-1
bzrlib/transport/http/response.py _response.py-20060613154423-a2ci7hd4iw5c7fnt-1
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/errors.py errors.py-20050309040759-20512168c4e14fbd
bzrlib/tests/__init__.py selftest.py-20050531073622-8d0e3c8845c97a64
bzrlib/tests/test_http.py testhttp.py-20051018020158-b2eef6e867c514d9
bzrlib/tests/test_transport_implementations.py test_transport_implementations.py-20051227111451-f97c5c7d5c49fce7
bzrlib/transport/__init__.py transport.py-20050711165921-4978aa7ce1285ad5
bzrlib/transport/http/__init__.py http_transport.py-20050711212304-506c5fd1059ace96
bzrlib/transport/http/_pycurl.py pycurlhttp.py-20060110060940-4e2a705911af77a6
bzrlib/transport/http/_urllib.py _urlgrabber.py-20060113083826-0bbf7d992fbf090c
------------------------------------------------------------
merged: john at arbash-meinel.com-20060719180226-ce7d12331fedbb70
parent: john at arbash-meinel.com-20060719141749-5c8a2a433f966f29
parent: pqm at pqm.ubuntu.com-20060718202456-fc3ad12fe5b7abd0
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Wed 2006-07-19 13:02:26 -0500
message:
[merge] bzr.dev 1868, cleanup NEWS
------------------------------------------------------------
merged: john at arbash-meinel.com-20060719141749-5c8a2a433f966f29
parent: john at arbash-meinel.com-20060718205753-900cc77474088d2b
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Wed 2006-07-19 09:17:49 -0500
message:
Update _extract_headers, make it less generic, and non recursive.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060718205753-900cc77474088d2b
parent: john at arbash-meinel.com-20060718202750-0ecae2405c7a7b55
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Tue 2006-07-18 15:57:53 -0500
message:
parse redirect headers properly.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060718202750-0ecae2405c7a7b55
parent: john at arbash-meinel.com-20060714192524-4396095e5dd43417
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Tue 2006-07-18 15:27:50 -0500
message:
code cleanups from Martin Pool.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060714192524-4396095e5dd43417
parent: john at arbash-meinel.com-20060714045829-5aeeed36f35603c1
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Fri 2006-07-14 14:25:24 -0500
message:
Remove the ability to read negative offsets from readv()
------------------------------------------------------------
merged: john at arbash-meinel.com-20060714045829-5aeeed36f35603c1
parent: john at arbash-meinel.com-20060714030220-1c9d1e0d18cdb629
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 23:58:29 -0500
message:
small bug found by mpee
------------------------------------------------------------
merged: john at arbash-meinel.com-20060714030220-1c9d1e0d18cdb629
parent: john at arbash-meinel.com-20060714022338-ff35af6b8d04e9a6
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 22:02:20 -0500
message:
pycurl expects us to just set the range of bytes, not including bytes=
------------------------------------------------------------
merged: john at arbash-meinel.com-20060714022338-ff35af6b8d04e9a6
parent: john at arbash-meinel.com-20060714014636-f60fd83545520ba6
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 21:23:38 -0500
message:
pycurl expects us to just set the range of bytes, not including bytes=
------------------------------------------------------------
merged: john at arbash-meinel.com-20060714014636-f60fd83545520ba6
parent: john at arbash-meinel.com-20060714003508-ddc3b6f0f7ad509c
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 20:46:36 -0500
message:
For pycurl inverse of (NOBODY,1) is (HTTPGET,1) not (NOBODY,0)
added the pycurl error codes from curl/curl.h
updated NEWS
------------------------------------------------------------
merged: john at arbash-meinel.com-20060714003508-ddc3b6f0f7ad509c
parent: john at arbash-meinel.com-20060713204221-1e69c237999296f8
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 19:35:08 -0500
message:
shorten the readv message to cause a smaller debug log.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713204221-1e69c237999296f8
parent: john at arbash-meinel.com-20060713195822-965e2c4422506a75
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 15:42:21 -0500
message:
Cleanup pass #2
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713195822-965e2c4422506a75
parent: john at arbash-meinel.com-20060713184334-a8a8c873459b4200
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 14:58:22 -0500
message:
cleanup pass, allow pycurl connections to be shared between transports.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713184334-a8a8c873459b4200
parent: john at arbash-meinel.com-20060713183858-ebf4aa1f9ef8bb6e
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 13:43:34 -0500
message:
Update http errors to properly use BzrNewError
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713183858-ebf4aa1f9ef8bb6e
parent: john at arbash-meinel.com-20060713183512-c520ddfbbdde3fef
parent: pqm at pqm.ubuntu.com-20060713133754-64c134fffd39fd99
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 13:38:58 -0500
message:
[merge] bzr.dev 1863
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713183512-c520ddfbbdde3fef
parent: john at arbash-meinel.com-20060713183332-8bc576e3eb31efd6
parent: michael at ellerman.id.au-20060704070457-69d44bec57cc8d44
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 13:35:12 -0500
message:
[merge] mpee's http work. Merged the spirit, if not the actual code.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060704070457-69d44bec57cc8d44
parent: michael at ellerman.id.au-20060704070301-eeb87f3378c742be
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-07-04 17:04:57 +1000
message:
Raise HttpError if we get a code we don't understand.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060704070301-eeb87f3378c742be
parent: michael at ellerman.id.au-20060704070158-e6dd9855fcb2bb37
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-07-04 17:03:01 +1000
message:
Be more careful when grabbing Content-Range header.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060704070158-e6dd9855fcb2bb37
parent: michael at ellerman.id.au-20060704070020-6559fbfcc063de43
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-07-04 17:01:58 +1000
message:
Be more careful when grabbing Content-Type header.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060704070020-6559fbfcc063de43
parent: michael at ellerman.id.au-20060627081952-4250a017ffad77f0
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-07-04 17:00:20 +1000
message:
Add HttpError to help diagnosing HTTP problems.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060627081952-4250a017ffad77f0
parent: michael at ellerman.id.au-20060627081935-2aa506a26a646ad4
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-06-27 18:19:52 +1000
message:
Don't import errno twice, thanks to pychecker.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060627081935-2aa506a26a646ad4
parent: michael at ellerman.id.au-20060627081525-b62b0b1b3ea7651c
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-06-27 18:19:35 +1000
message:
Raise TransportError() if we get a response code we don't understand.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060627081525-b62b0b1b3ea7651c
parent: michael at ellerman.id.au-20060627081245-93aa836b60253097
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-06-27 18:15:25 +1000
message:
Tests for HTTP multipart helpers.
Make HttpTransportBase.(offsets_to_ranges|range_header) static, and add
some tests for both.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060627081245-93aa836b60253097
parent: michael at ellerman.id.au-20060627071453-14db5ba9988f6c8c
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-06-27 18:12:45 +1000
message:
Don't send extra spaces in the HTTP Range header, it's just more I/O.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060627071453-14db5ba9988f6c8c
parent: michael at ellerman.id.au-20060621052521-dad2db9cd4956194
parent: john at arbash-meinel.com-20060617162214-9043c7594a3ad90d
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Tue 2006-06-27 17:14:53 +1000
message:
Merge John's HTTP work.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060621052521-dad2db9cd4956194
parent: michael at ellerman.id.au-20060616083821-97e9a43fd7566821
parent: pqm at pqm.ubuntu.com-20060620063849-400861427a3a9ff1
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Wed 2006-06-21 15:25:21 +1000
message:
Merge trunk.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713183332-8bc576e3eb31efd6
parent: john at arbash-meinel.com-20060713182621-10fc8ce282741455
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 13:33:32 -0500
message:
Update and add tests for the HttpTransportBase.range_header
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713182621-10fc8ce282741455
parent: john at arbash-meinel.com-20060713173135-451d2f4f5c6c8909
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 13:26:21 -0500
message:
Fix up the http transports so that tests pass with the new configuration.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713173135-451d2f4f5c6c8909
parent: john at arbash-meinel.com-20060713141256-caf2fbe04f23f3ce
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 12:31:35 -0500
message:
Update and test handle_response.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713141256-caf2fbe04f23f3ce
parent: john at arbash-meinel.com-20060713134347-3e3fe280e0bd3ac9
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 09:12:56 -0500
message:
Test that we can extract headers properly.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713134347-3e3fe280e0bd3ac9
parent: john at arbash-meinel.com-20060713131106-4f059a8003d852bd
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 08:43:47 -0500
message:
Move the functions/regexes to be static members
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713131106-4f059a8003d852bd
parent: john at arbash-meinel.com-20060713011110-9efcb64313b3143b
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-13 08:11:06 -0500
message:
Move offset_to_http_ranges back onto HttpTransportBase, clarify tests.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060713011110-9efcb64313b3143b
parent: john at arbash-meinel.com-20060711144459-40085c06b14494e0
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Wed 2006-07-12 20:11:10 -0500
message:
Add Keep-Alive for 1.0 servers
------------------------------------------------------------
merged: john at arbash-meinel.com-20060711144459-40085c06b14494e0
parent: john at arbash-meinel.com-20060706202201-768652a601fbda8c
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Tue 2006-07-11 09:44:59 -0500
message:
(broken) Work on factoring out handle_response so we can test with fake headers.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706202201-768652a601fbda8c
parent: john at arbash-meinel.com-20060706201251-680da840bdf98200
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 15:22:01 -0500
message:
Added an exception check for HttpMulti
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706201251-680da840bdf98200
parent: john at arbash-meinel.com-20060706195530-6d119290fa466736
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 15:12:51 -0500
message:
A couple of cleanup passes
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706195530-6d119290fa466736
parent: john at arbash-meinel.com-20060706194549-8ab6408a2f4ff920
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 14:55:30 -0500
message:
Add tests for HttpMultiRangeResponse
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706194549-8ab6408a2f4ff920
parent: john at arbash-meinel.com-20060706184711-94b439f5de6b30a2
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 14:45:49 -0500
message:
Adding tests for _parse_boundary.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706184711-94b439f5de6b30a2
parent: john at arbash-meinel.com-20060706184021-90352f45f0a79e49
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 13:47:11 -0500
message:
Refactor tests
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706184021-90352f45f0a79e49
parent: john at arbash-meinel.com-20060706183533-641776c30da93b56
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 13:40:21 -0500
message:
Factor out _parse_range into a separate function for easier testing/better hierarchy.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706183533-641776c30da93b56
parent: john at arbash-meinel.com-20060706182616-82f4555fe7613667
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 13:35:33 -0500
message:
Testing basic functionality of HttpMultipartRangeResponse
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706182616-82f4555fe7613667
parent: john at arbash-meinel.com-20060706174851-2bcc983af9d978fa
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 13:26:16 -0500
message:
Found a few bugs in error handling code, updated tests
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706174851-2bcc983af9d978fa
parent: john at arbash-meinel.com-20060706174621-b3d278556f7c860f
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 12:48:51 -0500
message:
Add tests for ResponseRange and streamline class
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706174621-b3d278556f7c860f
parent: john at arbash-meinel.com-20060706174556-70e9f8ee16fdd01a
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 12:46:21 -0500
message:
_pycurl doesn't use extract_auth, don't import it
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706174556-70e9f8ee16fdd01a
parent: john at arbash-meinel.com-20060706163003-aa969b1e9369b47f
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 12:45:56 -0500
message:
Simplify test_osutils.py to avoid >80 char lines
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706163003-aa969b1e9369b47f
parent: john at arbash-meinel.com-20060706162912-1680b115cdb24071
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 11:30:03 -0500
message:
Make tail_amount optional, so that has() still works.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060706162912-1680b115cdb24071
parent: john at arbash-meinel.com-20060705140746-5f3fd21c12c7d661
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Thu 2006-07-06 11:29:12 -0500
message:
[merge] Johan Rydberg test updates
------------------------------------------------------------
merged: john at arbash-meinel.com-20060705140746-5f3fd21c12c7d661
parent: john at arbash-meinel.com-20060617162214-9043c7594a3ad90d
parent: pqm at pqm.ubuntu.com-20060705001902-fdddc760baafe4f5
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Wed 2006-07-05 09:07:46 -0500
message:
[merge] bzr.dev
------------------------------------------------------------
merged: john at arbash-meinel.com-20060617162214-9043c7594a3ad90d
parent: john at arbash-meinel.com-20060617155251-c445c9574fcdf6aa
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Sat 2006-06-17 11:22:14 -0500
message:
Missed a couple of imports
------------------------------------------------------------
merged: john at arbash-meinel.com-20060617155251-c445c9574fcdf6aa
parent: john at arbash-meinel.com-20060617152019-b5366fa382aca78c
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Sat 2006-06-17 10:52:51 -0500
message:
Move the common Multipart stuff into plain http, and wrap pycurl response so that it matches the urllib response object.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060617152019-b5366fa382aca78c
parent: john at arbash-meinel.com-20060617131706-98a8e24de239a0d4
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Sat 2006-06-17 10:20:19 -0500
message:
Adding HEADERFUNCTION which lets us get any response codes we want.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060617131706-98a8e24de239a0d4
parent: john at arbash-meinel.com-20060617131218-7f812f5d62a47086
parent: michael at ellerman.id.au-20060616083821-97e9a43fd7566821
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Sat 2006-06-17 08:17:06 -0500
message:
[merge] Michael Ellerman: Support HTTP multipart ranges in urllib.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060616083821-97e9a43fd7566821
parent: michael at ellerman.id.au-20060608162722-df63ead19775ce65
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Fri 2006-06-16 18:38:21 +1000
message:
Add support for HTTP multipart ranges and hook it into http+urllib.
------------------------------------------------------------
merged: michael at ellerman.id.au-20060608162722-df63ead19775ce65
parent: pqm at pqm.ubuntu.com-20060607170142-7d3d91a14bbccd3f
committer: Michael Ellerman <michael at ellerman.id.au>
branch nick: http
timestamp: Fri 2006-06-09 02:27:22 +1000
message:
Don't coallesce try blocks, it can lead to confusing exceptions.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060617131218-7f812f5d62a47086
parent: john at arbash-meinel.com-20060617122817-2a0a1c84eb9c459c
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Sat 2006-06-17 08:12:18 -0500
message:
Use 2 pycurl objects because of RANGE peculiarities.
------------------------------------------------------------
merged: john at arbash-meinel.com-20060617122817-2a0a1c84eb9c459c
parent: pqm at pqm.ubuntu.com-20060617021848-38906188fc2a0300
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: http
timestamp: Sat 2006-06-17 07:28:17 -0500
message:
Re-use the curl object for a plain _get request
More information about the bazaar-commits
mailing list