Rev 3220: (andrew, in file:///home/pqm/archives/thelove/bzr/%2Btrunk/
Canonical.com Patch Queue Manager
pqm at pqm.ubuntu.com
Thu Feb 7 06:59:59 GMT 2008
At file:///home/pqm/archives/thelove/bzr/%2Btrunk/
------------------------------------------------------------
revno: 3220
revision-id:pqm at pqm.ubuntu.com-20080207065948-pjxwy4z6ljrpugj8
parent: pqm at pqm.ubuntu.com-20080207052502-jz28cn21r726b737
parent: andrew.bennetts at canonical.com-20080207034724-p3zhr7zp9kow4nax
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Thu 2008-02-07 06:59:48 +0000
message:
(andrew,
#185394) Disconnect and reconnect the smart medium after getting
unknown
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/smart/medium.py medium.py-20061103051856-rgu2huy59fkz902q-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
------------------------------------------------------------
revno: 3213.1.8
revision-id:andrew.bennetts at canonical.com-20080207034724-p3zhr7zp9kow4nax
parent: andrew.bennetts at canonical.com-20080207033103-n0jj2h356g730o6b
parent: pqm at pqm.ubuntu.com-20080206163804-6zyjbbfpsm8txfdm
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Thu 2008-02-07 14:47:24 +1100
message:
Merge from bzr.dev.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/branch.py branch.py-20050309040759-e4baf4e0d046576e
bzrlib/bzrdir.py bzrdir.py-20060131065624-156dfea39c4387cb
bzrlib/errors.py errors.py-20050309040759-20512168c4e14fbd
bzrlib/graph.py graph_walker.py-20070525030359-y852guab65d4wtn0-1
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/smart/protocol.py protocol.py-20061108035435-ot0lstk2590yqhzr-1
bzrlib/smart/repository.py repository.py-20061128022038-vr5wy5bubyb8xttk-1
bzrlib/tests/blackbox/test_version_info.py test_bb_version_info.py-20051228204928-91711c6559d952f7
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
bzrlib/tests/test_revisionnamespaces.py testrevisionnamespaces.py-20050711050225-8b4af89e6b1efe84
bzrlib/tests/test_smart.py test_smart.py-20061122024551-ol0l0o0oofsu9b3t-2
bzrlib/tests/test_version_info.py test_version_info.py-20051228204928-2c364e30b702b41b
bzrlib/version_info_formats/format_custom.py format_custom.py-20071029100350-ajovqhbpb5khf6gu-1
------------------------------------------------------------
revno: 3213.1.7
revision-id:andrew.bennetts at canonical.com-20080207033103-n0jj2h356g730o6b
parent: andrew.bennetts at canonical.com-20080207032824-jfnsjmu1ckk4r37h
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Thu 2008-02-07 14:31:03 +1100
message:
Mention in NOTES WHEN UPGRADING that a password/passphrase may need to be re-entered.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
------------------------------------------------------------
revno: 3213.1.6
revision-id:andrew.bennetts at canonical.com-20080207032824-jfnsjmu1ckk4r37h
parent: andrew.bennetts at canonical.com-20080207032013-v117r4sjp7b7hox6
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Thu 2008-02-07 14:28:24 +1100
message:
Emit warnings when forcing a reconnect.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
------------------------------------------------------------
revno: 3213.1.5
revision-id:andrew.bennetts at canonical.com-20080207032013-v117r4sjp7b7hox6
parent: andrew.bennetts at canonical.com-20080206060156-3im8420jsztowpe5
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Thu 2008-02-07 14:20:13 +1100
message:
Move NEWS entry into NOTES WHEN UPGRADING.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
------------------------------------------------------------
revno: 3213.1.4
revision-id:andrew.bennetts at canonical.com-20080206060156-3im8420jsztowpe5
parent: andrew.bennetts at canonical.com-20080206054511-6kcr4dhuvvgv9xu4
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Wed 2008-02-06 17:01:56 +1100
message:
Add NEWS entry.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
------------------------------------------------------------
revno: 3213.1.3
revision-id:andrew.bennetts at canonical.com-20080206054511-6kcr4dhuvvgv9xu4
parent: andrew.bennetts at canonical.com-20080206053838-i0i8qhr2708c5b99
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Wed 2008-02-06 16:45:11 +1100
message:
Fix typo in comment.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
------------------------------------------------------------
revno: 3213.1.2
revision-id:andrew.bennetts at canonical.com-20080206053838-i0i8qhr2708c5b99
parent: andrew.bennetts at canonical.com-20080206035225-q572lt8uhrpl22dj
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Wed 2008-02-06 16:38:38 +1100
message:
Add test for reconnection if get_parent_map is unknown by the server.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
------------------------------------------------------------
revno: 3213.1.1
revision-id:andrew.bennetts at canonical.com-20080206035225-q572lt8uhrpl22dj
parent: pqm at pqm.ubuntu.com-20080205071430-7b9vl83ebnsd6i0g
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: request-body-recovery
timestamp: Wed 2008-02-06 14:52:25 +1100
message:
Recover (by reconnecting) if the server turns out not to understand the new requests in 1.2 that send bodies.
modified:
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/smart/medium.py medium.py-20061103051856-rgu2huy59fkz902q-1
bzrlib/tests/test_remote.py test_remote.py-20060720103555-yeeg2x51vn0rbtdp-2
=== modified file 'NEWS'
--- a/NEWS 2008-02-07 05:25:02 +0000
+++ b/NEWS 2008-02-07 06:59:48 +0000
@@ -7,6 +7,15 @@
IN DEVELOPMENT
--------------
+ NOTES WHEN UPGRADING:
+
+ * Fetching via the smart protocol may need to reconnect once during a fetch
+ if the remote server is running Bazaar 1.1 or earlier, because the client
+ attempts to use more efficient requests that confuse older servers. You
+ may be required to re-enter a password or passphrase when this happens.
+ This won't happen if the server is upgraded to Bazaar 1.2.
+ (Andrew Bennetts)
+
CHANGES:
* Fetching via bzr+ssh will no longer fill ghosts by default (this is
=== modified file 'bzrlib/remote.py'
--- a/bzrlib/remote.py 2008-02-06 01:45:28 +0000
+++ b/bzrlib/remote.py 2008-02-07 03:47:24 +0000
@@ -42,7 +42,7 @@
zero_ninetyone,
)
from bzrlib.revision import NULL_REVISION
-from bzrlib.trace import mutter, note
+from bzrlib.trace import mutter, note, warning
# Note: RemoteBzrDirFormat is in bzrdir.py
@@ -777,19 +777,24 @@
"""See bzrlib.Graph.get_parent_map()."""
# Hack to build up the caching logic.
ancestry = self._parents_map
- missing_revisions = set(key for key in keys if key not in ancestry)
+ if ancestry is None:
+ # Repository is not locked, so there's no cache.
+ missing_revisions = set(keys)
+ ancestry = {}
+ else:
+ missing_revisions = set(key for key in keys if key not in ancestry)
if missing_revisions:
parent_map = self._get_parent_map(missing_revisions)
if 'hpss' in debug.debug_flags:
mutter('retransmitted revisions: %d of %d',
- len(set(self._parents_map).intersection(parent_map)),
+ len(set(ancestry).intersection(parent_map)),
len(parent_map))
- self._parents_map.update(parent_map)
+ ancestry.update(parent_map)
present_keys = [k for k in keys if k in ancestry]
if 'hpss' in debug.debug_flags:
self._requested_parents.update(present_keys)
mutter('Current RemoteRepository graph hit rate: %d%%',
- 100.0 * len(self._requested_parents) / len(self._parents_map))
+ 100.0 * len(self._requested_parents) / len(ancestry))
return dict((k, ancestry[k]) for k in present_keys)
def _response_is_unknown_method(self, response, verb):
@@ -812,6 +817,13 @@
def _get_parent_map(self, keys):
"""Helper for get_parent_map that performs the RPC."""
+ medium = self._client.get_smart_medium()
+ if not medium._remote_is_at_least_1_2:
+ # We already found out that the server can't understand
+ # Repository.get_parent_map requests, so just fetch the whole
+ # graph.
+ return self.get_revision_graph()
+
keys = set(keys)
if NULL_REVISION in keys:
keys.discard(NULL_REVISION)
@@ -831,14 +843,18 @@
# TODO: Manage this incrementally to avoid covering the same path
# repeatedly. (The server will have to on each request, but the less
# work done the better).
- start_set = set(self._parents_map)
+ parents_map = self._parents_map
+ if parents_map is None:
+ # Repository is not locked, so there's no cache.
+ parents_map = {}
+ start_set = set(parents_map)
result_parents = set()
- for parents in self._parents_map.itervalues():
+ for parents in parents_map.itervalues():
result_parents.update(parents)
stop_keys = result_parents.difference(start_set)
included_keys = start_set.intersection(result_parents)
start_set.difference_update(included_keys)
- recipe = (start_set, stop_keys, len(self._parents_map))
+ recipe = (start_set, stop_keys, len(parents_map))
body = self._serialise_search_recipe(recipe)
path = self.bzrdir._path_for_remote_call(self._client)
for key in keys:
@@ -848,12 +864,18 @@
response = self._client.call_with_body_bytes_expecting_body(
verb, args, self._serialise_search_recipe(recipe))
if self._response_is_unknown_method(response, verb):
- # Server that does not support this method, get the whole graph.
- response = self._client.call_expecting_body(
- 'Repository.get_revision_graph', path, '')
- if response[0][0] not in ['ok', 'nosuchrevision']:
- reponse[1].cancel_read_body()
- raise errors.UnexpectedSmartServerResponse(response[0])
+ # Server does not support this method, so get the whole graph.
+ # Worse, we have to force a disconnection, because the server now
+ # doesn't realise it has a body on the wire to consume, so the
+ # only way to recover is to abandon the connection.
+ warning(
+ 'Server is too old for fast get_parent_map, reconnecting. '
+ '(Upgrade the server to Bazaar 1.2 to avoid this)')
+ medium.disconnect()
+ # To avoid having to disconnect repeatedly, we keep track of the
+ # fact the server doesn't understand remote methods added in 1.2.
+ medium._remote_is_at_least_1_2 = False
+ return self.get_revision_graph()
elif response[0][0] not in ['ok']:
reponse[1].cancel_read_body()
raise errors.UnexpectedSmartServerResponse(response[0])
@@ -1009,25 +1031,37 @@
return self._real_repository.has_signature_for_revision_id(revision_id)
def get_data_stream_for_search(self, search):
+ medium = self._client.get_smart_medium()
+ if not medium._remote_is_at_least_1_2:
+ self._ensure_real()
+ return self._real_repository.get_data_stream_for_search(search)
REQUEST_NAME = 'Repository.stream_revisions_chunked'
path = self.bzrdir._path_for_remote_call(self._client)
body = self._serialise_search_recipe(search.get_recipe())
response, protocol = self._client.call_with_body_bytes_expecting_body(
REQUEST_NAME, (path,), body)
+ if self._response_is_unknown_method((response, protocol), REQUEST_NAME):
+ # Server does not support this method, so fall back to VFS.
+ # Worse, we have to force a disconnection, because the server now
+ # doesn't realise it has a body on the wire to consume, so the
+ # only way to recover is to abandon the connection.
+ warning(
+ 'Server is too old for streaming pull, reconnecting. '
+ '(Upgrade the server to Bazaar 1.2 to avoid this)')
+ medium.disconnect()
+ # To avoid having to disconnect repeatedly, we keep track of the
+ # fact the server doesn't understand this remote method.
+ medium._remote_is_at_least_1_2 = False
+ self._ensure_real()
+ return self._real_repository.get_data_stream_for_search(search)
+
if response == ('ok',):
return self._deserialise_stream(protocol)
if response == ('NoSuchRevision', ):
# We cannot easily identify the revision that is missing in this
# situation without doing much more network IO. For now, bail.
raise NoSuchRevision(self, "unknown")
- elif (response == ('error', "Generic bzr smart protocol error: "
- "bad request '%s'" % REQUEST_NAME) or
- response == ('error', "Generic bzr smart protocol error: "
- "bad request u'%s'" % REQUEST_NAME)):
- protocol.cancel_read_body()
- self._ensure_real()
- return self._real_repository.get_data_stream_for_search(search)
else:
raise errors.UnexpectedSmartServerResponse(response)
=== modified file 'bzrlib/smart/medium.py'
--- a/bzrlib/smart/medium.py 2008-01-22 06:56:59 +0000
+++ b/bzrlib/smart/medium.py 2008-02-06 03:52:25 +0000
@@ -386,6 +386,10 @@
def __init__(self):
self._current_request = None
+ # Be optimistic: we assume the remote end can accept new remote
+ # requests until we get an error saying otherwise. (1.2 adds some
+ # requests that send bodies, which confuses older servers.)
+ self._remote_is_at_least_1_2 = True
def accept_bytes(self, bytes):
self._accept_bytes(bytes)
=== modified file 'bzrlib/tests/test_remote.py'
--- a/bzrlib/tests/test_remote.py 2008-02-06 01:45:28 +0000
+++ b/bzrlib/tests/test_remote.py 2008-02-07 03:47:24 +0000
@@ -140,7 +140,7 @@
self.responses = responses
self._calls = []
self.expecting_body = False
- _SmartClient.__init__(self, FakeMedium(fake_medium_base))
+ _SmartClient.__init__(self, FakeMedium(fake_medium_base, self._calls))
def call(self, method, *args):
self._calls.append(('call', method, args))
@@ -162,8 +162,20 @@
class FakeMedium(object):
- def __init__(self, base):
+ def __init__(self, base, client_calls):
self.base = base
+ self.connection = FakeConnection(client_calls)
+ self._client_calls = client_calls
+
+
+class FakeConnection(object):
+
+ def __init__(self, client_calls):
+ self._remote_is_at_least_1_2 = True
+ self._client_calls = client_calls
+
+ def disconnect(self):
+ self._client_calls.append(('disconnect medium',))
class TestVfsHas(tests.TestCase):
@@ -660,6 +672,27 @@
client._calls)
repo.unlock()
+ def test_get_parent_map_reconnects_if_unknown_method(self):
+ error_msg = (
+ "Generic bzr smart protocol error: "
+ "bad request 'Repository.get_parent_map'")
+ responses = [
+ (('error', error_msg), ''),
+ (('ok',), '')]
+ transport_path = 'quack'
+ repo, client = self.setup_fake_client_and_repository(
+ responses, transport_path)
+ rev_id = 'revision-id'
+ parents = repo.get_parent_map([rev_id])
+ self.assertEqual(
+ [('call_with_body_bytes_expecting_body',
+ 'Repository.get_parent_map', ('quack/', rev_id), '\n\n0'),
+ ('disconnect medium',),
+ ('call_expecting_body', 'Repository.get_revision_graph',
+ ('quack/', ''))],
+ client._calls)
+
+
class TestRepositoryGetRevisionGraph(TestRemoteRepository):
More information about the bazaar-commits
mailing list