Rev 3343: Return the correct knit serialisation method in _StreamAccess. in file:///home/pqm/archives/thelove/bzr/%2Btrunk/
Canonical.com Patch Queue Manager
pqm at pqm.ubuntu.com
Mon Apr 7 08:53:00 BST 2008
At file:///home/pqm/archives/thelove/bzr/%2Btrunk/
------------------------------------------------------------
revno: 3343
revision-id:pqm at pqm.ubuntu.com-20080407075250-phs53xnslo8boaeo
parent: pqm at pqm.ubuntu.com-20080407061951-fgvwd4v010ibwl3u
parent: andrew.bennetts at canonical.com-20080407055951-1o7h3qmwgww5uea0
committer: Canonical.com Patch Queue Manager <pqm at pqm.ubuntu.com>
branch nick: +trunk
timestamp: Mon 2008-04-07 08:52:50 +0100
message:
Return the correct knit serialisation method in _StreamAccess.
(Andrew Bennetts, Martin Pool, Robert Collins)
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
bzrlib/tests/test_knit.py test_knit.py-20051212171302-95d4c00dd5f11f2b
------------------------------------------------------------
revno: 3340.1.1
revision-id:andrew.bennetts at canonical.com-20080407055951-1o7h3qmwgww5uea0
parent: pqm at pqm.ubuntu.com-20080407012644-p1nash3ycyh2f5n8
parent: andrew.bennetts at canonical.com-20080407052803-2imiyqyze5guma7d
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: 208418-1.4
timestamp: Mon 2008-04-07 15:59:51 +1000
message:
Add NEWS entry.
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
bzrlib/tests/test_knit.py test_knit.py-20051212171302-95d4c00dd5f11f2b
------------------------------------------------------------
revno: 3287.6.6
revision-id:andrew.bennetts at canonical.com-20080407052803-2imiyqyze5guma7d
parent: andrew.bennetts at canonical.com-20080404065803-w0pnxdeh1lgvfxiz
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: 208418-test
timestamp: Mon 2008-04-07 15:28:03 +1000
message:
Tweaks suggested by Robert's review.
modified:
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
------------------------------------------------------------
revno: 3287.6.5
revision-id:andrew.bennetts at canonical.com-20080404065803-w0pnxdeh1lgvfxiz
parent: andrew.bennetts at canonical.com-20080404065447-2nttn6n2rwonf0u2
parent: andrew.bennetts at canonical.com-20080404065630-cklwnxi7g103wqnc
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: 208418-test
timestamp: Fri 2008-04-04 17:58:03 +1100
message:
Merge Martin's version of the fix for bug 208418.
modified:
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
------------------------------------------------------------
revno: 3287.6.3.1.3
revision-id:andrew.bennetts at canonical.com-20080404065630-cklwnxi7g103wqnc
parent: andrew.bennetts at canonical.com-20080404054155-x0zddn049lc3zcj7
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: 208418-knit-parsing
timestamp: Fri 2008-04-04 17:56:30 +1100
message:
Don't accidentally mutate the backing_index's cache of the options.
modified:
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
------------------------------------------------------------
revno: 3287.6.3.1.2
revision-id:andrew.bennetts at canonical.com-20080404054155-x0zddn049lc3zcj7
parent: mbp at sourcefrog.net-20080404043344-ewtd0ck2mm0l7sx9
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: 208418-knit-parsing
timestamp: Fri 2008-04-04 16:41:55 +1100
message:
Fix trivial bug in get_options.
modified:
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
------------------------------------------------------------
revno: 3287.6.3.1.1
revision-id:mbp at sourcefrog.net-20080404043344-ewtd0ck2mm0l7sx9
parent: pqm at pqm.ubuntu.com-20080320092314-y4i0bpy37v8i1mc5
committer: Martin Pool <mbp at sourcefrog.net>
branch nick: 208418-knit-parsing
timestamp: Fri 2008-04-04 15:33:44 +1100
message:
#2008418: (with spiv) Avoid interpreting fulltexts as line deltas when pulling knits.
The knit._StreamAccess class is used for some cases of pulling from a knit to pack
repository; it is used to build a Knit that reads both from a network stream
and a local repository. We were in some cases giving back a full text
from the local repository but incorrectly marking it as a delta.
Also, mark the flag that controls this more clearly as from_backing_knit.
modified:
bzrlib/knit.py knit.py-20051212171256-f056ac8f0fbe1bd9
------------------------------------------------------------
revno: 3287.6.4
revision-id:andrew.bennetts at canonical.com-20080404065447-2nttn6n2rwonf0u2
parent: pqm at pqm.ubuntu.com-20080320092314-y4i0bpy37v8i1mc5
committer: Andrew Bennetts <andrew.bennetts at canonical.com>
branch nick: 208418-test
timestamp: Fri 2008-04-04 17:54:47 +1100
message:
Failing test for bug 208418.
modified:
bzrlib/tests/test_knit.py test_knit.py-20051212171302-95d4c00dd5f11f2b
=== modified file 'NEWS'
--- a/NEWS 2008-04-07 04:44:56 +0000
+++ b/NEWS 2008-04-07 07:52:50 +0000
@@ -95,6 +95,11 @@
* Don't ask for a password if there is no real terminal.
(Alexander Belchenko, #69851)
+ * Fix a bug causing a ValueError crash in ``parse_line_delta_iter`` when
+ fetching revisions from a knit to pack repository or vice versa using
+ bzr:// (including over http or ssh).
+ (#208418, Andrew Bennetts, Martin Pool, Robert Collins)
+
* Implement handling of basename parameter for DefaultMail. (James Westby)
* Launchpad locations (lp: URLs) can be pulled. (Aaron Bentley, #181945)
=== modified file 'bzrlib/knit.py'
--- a/bzrlib/knit.py 2008-04-01 02:41:25 +0000
+++ b/bzrlib/knit.py 2008-04-07 05:59:51 +0000
@@ -2225,13 +2225,17 @@
def get_raw_records(self, memos_for_retrieval):
"""Get the raw bytes for a records.
- :param memos_for_retrieval: An iterable containing the (thunk_flag,
- index, start, end) memo for retrieving the bytes.
- :return: An iterator over the bytes of the records.
+ :param memos_for_retrieval: An iterable of memos from the
+ _StreamIndex object identifying bytes to read; for these classes
+ they are (from_backing_knit, index, start, end) and can point to
+ either the backing knit or streamed data.
+ :return: An iterator yielding a byte string for each record in
+ memos_for_retrieval.
"""
# use a generator for memory friendliness
- for thunk_flag, version_id, start, end in memos_for_retrieval:
- if version_id is self.stream_index:
+ for from_backing_knit, version_id, start, end in memos_for_retrieval:
+ if not from_backing_knit:
+ assert version_id is self.stream_index
yield self.data[start:end]
continue
# we have been asked to thunk. This thunking only occurs when
@@ -2242,21 +2246,19 @@
# as desired. However, for now, this is sufficient.
if self.orig_factory.__class__ != KnitPlainFactory:
raise errors.KnitCorrupt(
- self, 'Bad thunk request %r' % version_id)
+ self, 'Bad thunk request %r cannot be backed by %r' %
+ (version_id, self.orig_factory))
lines = self.backing_knit.get_lines(version_id)
line_bytes = ''.join(lines)
digest = sha_string(line_bytes)
+ # the packed form of the fulltext always has a trailing newline,
+ # even if the actual text does not, unless the file is empty. the
+ # record options including the noeol flag are passed through by
+ # _StreamIndex, so this is safe.
if lines:
if lines[-1][-1] != '\n':
lines[-1] = lines[-1] + '\n'
line_bytes += '\n'
- orig_options = list(self.backing_knit._index.get_options(version_id))
- if 'fulltext' not in orig_options:
- if 'line-delta' not in orig_options:
- raise errors.KnitCorrupt(self,
- 'Unknown compression method %r' % orig_options)
- orig_options.remove('line-delta')
- orig_options.append('fulltext')
# We want plain data, because we expect to thunk only to allow text
# extraction.
size, bytes = self.backing_knit._data._record_to_data(version_id,
@@ -2314,8 +2316,9 @@
:return: A dict of version_id:(index_memo, compression_parent,
parents, record_details).
index_memo
- opaque structure to pass to read_records to extract the raw
- data
+ opaque memo that can be passed to _StreamAccess.read_records
+ to extract the raw data; for these classes it is
+ (from_backing_knit, index, start, end)
compression_parent
Content that this record is built upon, may be None
parents
@@ -2333,23 +2336,22 @@
continue
parent_ids = self.get_parents_with_ghosts(version_id)
noeol = ('no-eol' in self.get_options(version_id))
+ index_memo = self.get_position(version_id)
+ from_backing_knit = index_memo[0]
+ if from_backing_knit:
+ # texts retrieved from the backing knit are always full texts
+ method = 'fulltext'
if method == 'fulltext':
compression_parent = None
else:
compression_parent = parent_ids[0]
- index_memo = self.get_position(version_id)
result[version_id] = (index_memo, compression_parent,
parent_ids, (method, noeol))
return result
def get_method(self, version_id):
"""Return compression method of specified version."""
- try:
- options = self._by_version[version_id][0]
- except KeyError:
- # Strictly speaking this should check in the backing knit, but
- # until we have a test to discriminate, this will do.
- return self.backing_index.get_method(version_id)
+ options = self.get_options(version_id)
if 'fulltext' in options:
return 'fulltext'
elif 'line-delta' in options:
@@ -2365,7 +2367,17 @@
try:
return self._by_version[version_id][0]
except KeyError:
- return self.backing_index.get_options(version_id)
+ options = list(self.backing_index.get_options(version_id))
+ if 'fulltext' in options:
+ pass
+ elif 'line-delta' in options:
+ # Texts from the backing knit are always returned from the stream
+ # as full texts
+ options.remove('line-delta')
+ options.append('fulltext')
+ else:
+ raise errors.KnitIndexUnknownMethod(self, options)
+ return tuple(options)
def get_parent_map(self, version_ids):
"""Passed through to by KnitVersionedFile.get_parent_map."""
@@ -2393,8 +2405,10 @@
coordinates into that (as index_memo's are opaque outside the
index and matching access class).
- :return: a tuple (thunk_flag, index, start, end). If thunk_flag is
- False, index will be self, otherwise it will be a version id.
+ :return: a tuple (from_backing_knit, index, start, end) that can
+ be passed e.g. to get_raw_records.
+ If from_backing_knit is False, index will be self, otherwise it
+ will be a version id.
"""
try:
start, end = self._by_version[version_id][1]
=== modified file 'bzrlib/tests/test_knit.py'
--- a/bzrlib/tests/test_knit.py 2008-04-01 02:41:25 +0000
+++ b/bzrlib/tests/test_knit.py 2008-04-07 05:59:51 +0000
@@ -1924,6 +1924,37 @@
errors.KnitDataStreamUnknown,
target.insert_data_stream, data_stream)
+ def test_insert_data_stream_bug_208418(self):
+ """You can insert a stream with an incompatible format, even when:
+ * the stream has a line-delta record,
+ * whose parent is in the target, also stored as a line-delta
+
+ See <https://launchpad.net/bugs/208418>.
+ """
+ base_lines = split_lines(TEXT_1)
+ # Make the target
+ target = self.make_test_knit(name='target', annotate=True)
+ target.add_lines('version-1', [], base_lines)
+ target.add_lines('version-2', ['version-1'], base_lines + ['a\n'])
+ # The second record should be a delta.
+ self.assertEqual('line-delta', target._index.get_method('version-2'))
+
+ # Make a source, with a different format, but the same data
+ source = self.make_test_knit(name='source', annotate=False)
+ source.add_lines('version-1', [], base_lines)
+ source.add_lines('version-2', ['version-1'], base_lines + ['a\n'])
+ # Now add another record, which should be stored as a delta against
+ # version-2.
+ source.add_lines('version-3', ['version-2'], base_lines + ['b\n'])
+ self.assertEqual('line-delta', source._index.get_method('version-3'))
+
+ # Make a stream of the new version
+ data_stream = source.get_data_stream(['version-3'])
+ # And insert into the target
+ target.insert_data_stream(data_stream)
+ # No errors should have been raised.
+
+
# * test that a stream of "already present version, then new version"
# inserts correctly.
More information about the bazaar-commits
mailing list