Rev 3801: Merge back the InterDifferingSerializer code. in http://bzr.arbash-meinel.com/branches/bzr/brisbane/merge_dev
John Arbash Meinel
john at arbash-meinel.com
Sun Dec 7 17:41:00 GMT 2008
At http://bzr.arbash-meinel.com/branches/bzr/brisbane/merge_dev
------------------------------------------------------------
revno: 3801
revision-id: john at arbash-meinel.com-20081207174043-xjxckelplsqdey73
parent: john at arbash-meinel.com-20081207171419-u9bv8vntqimktxt0
parent: john at arbash-meinel.com-20081207173303-ydamo2rxs3ngjhw0
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: merge_dev
timestamp: Sun 2008-12-07 11:40:43 -0600
message:
Merge back the InterDifferingSerializer code.
renamed:
bzrlib/tests/per_repository/test_add_inventory_delta.py => bzrlib/tests/per_repository/test_add_inventory_by_delta.py test_add_inventory_d-20081013002626-rut81igtlqb4590z-1
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/__init__.py __init__.py-20050309040759-33e65acf91bbcd5d
bzrlib/commit.py commit.py-20050511101309-79ec1a0168e0e825
bzrlib/inventory.py inventory.py-20050309040759-6648b84ca2005b37
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/tests/interrepository_implementations/__init__.py __init__.py-20060220054744-baf49a1f88f17b1a
bzrlib/tests/per_repository/__init__.py __init__.py-20060131092037-9564957a7d4a841b
bzrlib/tests/per_repository/test_commit_builder.py test_commit_builder.py-20060606110838-76e3ra5slucqus81-1
bzrlib/tests/per_repository/test_add_inventory_by_delta.py test_add_inventory_d-20081013002626-rut81igtlqb4590z-1
------------------------------------------------------------
revno: 3735.135.10
revision-id: john at arbash-meinel.com-20081207173303-ydamo2rxs3ngjhw0
parent: john at arbash-meinel.com-20081205222549-lri0j1a3wv37wtax
parent: john at arbash-meinel.com-20081207172622-r3hrmb872nwmezeu
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Sun 2008-12-07 11:33:03 -0600
message:
Merge in the new add_inventory_by_delta and handle the new return values.
modified:
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/tests/per_repository/test_add_inventory_by_delta.py test_add_inventory_d-20081013002626-rut81igtlqb4590z-1
------------------------------------------------------------
revno: 3735.136.1
revision-id: john at arbash-meinel.com-20081207172622-r3hrmb872nwmezeu
parent: john at arbash-meinel.com-20081205172501-a0g7ho4sl29q6dz9
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: add_inventory_by_delta
timestamp: Sun 2008-12-07 11:26:22 -0600
message:
Change the return of add_inventory_by_delta to also return the Inventory.
modified:
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
------------------------------------------------------------
revno: 3735.135.9
revision-id: john at arbash-meinel.com-20081205222549-lri0j1a3wv37wtax
parent: john at arbash-meinel.com-20081205221928-kzstz04ngqrxpb12
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Fri 2008-12-05 16:25:49 -0600
message:
Use a last-modified-revision test.
This avoids copying the same text revisions multiple times.
modified:
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
------------------------------------------------------------
revno: 3735.135.8
revision-id: john at arbash-meinel.com-20081205221928-kzstz04ngqrxpb12
parent: john at arbash-meinel.com-20081205221847-hs9mh3yuinxq7w29
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Fri 2008-12-05 16:19:28 -0600
message:
Bring in the CHK inter-differing-serializer fetch code.
Refactor it into several helper functions which makes the flow a bit
clearer and reduces the indentation.
modified:
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
------------------------------------------------------------
revno: 3735.135.7
revision-id: john at arbash-meinel.com-20081205221847-hs9mh3yuinxq7w29
parent: john at arbash-meinel.com-20081205221809-d3c4cz1jyv9p1y1h
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Fri 2008-12-05 16:18:47 -0600
message:
Add an Inter test that actually uses InterDifferingSerializer
modified:
bzrlib/tests/interrepository_implementations/__init__.py __init__.py-20060220054744-baf49a1f88f17b1a
------------------------------------------------------------
revno: 3735.135.6
revision-id: john at arbash-meinel.com-20081205221809-d3c4cz1jyv9p1y1h
parent: john at arbash-meinel.com-20081205172501-a0g7ho4sl29q6dz9
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Fri 2008-12-05 16:18:09 -0600
message:
Bring in Inventory._make_delta
modified:
bzrlib/inventory.py inventory.py-20050309040759-6648b84ca2005b37
------------------------------------------------------------
revno: 3735.135.5
revision-id: john at arbash-meinel.com-20081205172501-a0g7ho4sl29q6dz9
parent: john at arbash-meinel.com-20081205164010-05sx88jxi50q819a
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: add_inventory_by_delta
timestamp: Fri 2008-12-05 11:25:01 -0600
message:
Change record_delete() to return the delta.
Add direct tests for CB.get_basis_delta(), to ensure that it returns a
valid delta, and that it errors if the client hasn't called will_record_deletes.
modified:
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/tests/per_repository/test_commit_builder.py test_commit_builder.py-20060606110838-76e3ra5slucqus81-1
------------------------------------------------------------
revno: 3735.135.4
revision-id: john at arbash-meinel.com-20081205164010-05sx88jxi50q819a
parent: john at arbash-meinel.com-20081205162905-12c9k3esfetyes4a
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: add_inventory_by_delta
timestamp: Fri 2008-12-05 10:40:10 -0600
message:
This didn't land in 1.9 or 1.10, so make the minimum version 1.11
modified:
bzrlib/__init__.py __init__.py-20050309040759-33e65acf91bbcd5d
------------------------------------------------------------
revno: 3735.135.3
revision-id: john at arbash-meinel.com-20081205162905-12c9k3esfetyes4a
parent: john at arbash-meinel.com-20081205160704-ti2a80z9tvqehwws
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Fri 2008-12-05 10:29:05 -0600
message:
Hide the .basis_delta variable, and require callers to use .get_basis_delta()
This allows us to check that the callers were sure they would be
generating a proper delta, by using CommitBuilder.record_delete() correctly.
modified:
bzrlib/commit.py commit.py-20050511101309-79ec1a0168e0e825
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
------------------------------------------------------------
revno: 3735.135.2
revision-id: john at arbash-meinel.com-20081205160704-ti2a80z9tvqehwws
parent: john at arbash-meinel.com-20081205160233-f4c61by1u3kf5sqj
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Fri 2008-12-05 10:07:04 -0600
message:
Rename add_inventory_delta to add_inventory_by_delta.
renamed:
bzrlib/tests/per_repository/test_add_inventory_delta.py => bzrlib/tests/per_repository/test_add_inventory_by_delta.py test_add_inventory_d-20081013002626-rut81igtlqb4590z-1
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/tests/per_repository/__init__.py __init__.py-20060131092037-9564957a7d4a841b
bzrlib/tests/per_repository/test_add_inventory_by_delta.py test_add_inventory_d-20081013002626-rut81igtlqb4590z-1
------------------------------------------------------------
revno: 3735.135.1
revision-id: john at arbash-meinel.com-20081205160233-f4c61by1u3kf5sqj
parent: pqm at pqm.ubuntu.com-20081205135154-uwqcpl3lruah9fo3
parent: robertc at robertcollins.net-20081013044331-ibghe8j4no7uhb55
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: differ_serializer
timestamp: Fri 2008-12-05 10:02:33 -0600
message:
Merge in the add_inventory_delta code.
added:
bzrlib/tests/per_repository/test_add_inventory_delta.py test_add_inventory_d-20081013002626-rut81igtlqb4590z-1
modified:
NEWS NEWS-20050323055033-4e00b5db738777ff
bzrlib/__init__.py __init__.py-20050309040759-33e65acf91bbcd5d
bzrlib/commit.py commit.py-20050511101309-79ec1a0168e0e825
bzrlib/remote.py remote.py-20060720103555-yeeg2x51vn0rbtdp-1
bzrlib/repository.py rev_storage.py-20051111201905-119e9401e46257e3
bzrlib/tests/per_repository/__init__.py __init__.py-20060131092037-9564957a7d4a841b
bzrlib/tests/per_repository/test_commit_builder.py test_commit_builder.py-20060606110838-76e3ra5slucqus81-1
bzrlib/tests/per_repository/test_repository.py test_repository.py-20060131092128-ad07f494f5c9d26c
-------------- next part --------------
=== modified file 'NEWS'
--- a/NEWS 2008-12-07 16:46:56 +0000
+++ b/NEWS 2008-12-07 17:40:43 +0000
@@ -35,6 +35,11 @@
API CHANGES:
+ * The logic in commit now delegates inventory basis calculations to
+ the ``CommitBuilder`` object; this requires that the commit builder
+ in use has been updated to support the new ``recording_deletes`` and
+ ``record_delete`` methods. (Robert Collins)
+
TESTING:
INTERNALS:
@@ -45,6 +50,15 @@
returned from each pack in turn, in forward I/O order.
(John Arbash Meinel)
+ * New method ``bzrlib.repository.Repository.add_inventory_by_delta``
+ allows adding an inventory via an inventory delta, which can be
+ more efficient for some repository types. (Robert Collins)
+
+ * Repository ``CommitBuilder`` objects can now accumulate an inventory
+ delta. To enable this functionality call ``builder.recording_deletes``
+ and additionally call ``builder.record_delete`` when a delete
+ against the basis occurs. (Robert Collins)
+
bzr 1.10 2008-12-05
-------------------
=== modified file 'bzrlib/__init__.py'
--- a/bzrlib/__init__.py 2008-12-07 16:46:56 +0000
+++ b/bzrlib/__init__.py 2008-12-07 17:40:43 +0000
@@ -53,8 +53,8 @@
version_info = (1, 11, 0, 'dev', 0)
-# API compatibility version: bzrlib is currently API compatible with 1.7.
-api_minimum_version = (1, 9, 0)
+# API compatibility version: bzrlib is currently API compatible with 1.11.
+api_minimum_version = (1, 11, 0)
def _format_version_tuple(version_info):
=== modified file 'bzrlib/commit.py'
--- a/bzrlib/commit.py 2008-12-07 16:46:56 +0000
+++ b/bzrlib/commit.py 2008-12-07 17:40:43 +0000
@@ -352,9 +352,9 @@
entries_title="Directory")
self.builder = self.branch.get_commit_builder(self.parents,
self.config, timestamp, timezone, committer, revprops, rev_id)
-
+
try:
- self.builder.recording_deletes()
+ self.builder.will_record_deletes()
# find the location being committed to
if self.bound_branch:
master_location = self.master_branch.base
@@ -412,7 +412,7 @@
# Make the working tree up to date with the branch
self._set_progress_stage("Updating the working tree")
self.work_tree.update_basis_by_delta(self.rev_id,
- self.builder.basis_delta)
+ self.builder.get_basis_delta())
self.reporter.completed(new_revno, self.rev_id)
self._process_post_hooks(old_revno, new_revno)
finally:
=== modified file 'bzrlib/inventory.py'
--- a/bzrlib/inventory.py 2008-12-04 17:52:42 +0000
+++ b/bzrlib/inventory.py 2008-12-07 17:40:43 +0000
@@ -1265,9 +1265,27 @@
def has_id(self, file_id):
return (file_id in self._byid)
+ def _make_delta(self, old):
+ """Make an inventory delta from two inventories."""
+ old_ids = set(old)
+ new_ids = set(self)
+ adds = new_ids - old_ids
+ deletes = old_ids - new_ids
+ common = old_ids.intersection(new_ids)
+ delta = []
+ for file_id in deletes:
+ delta.append((old.id2path(file_id), None, file_id, None))
+ for file_id in adds:
+ delta.append((None, self.id2path(file_id), file_id, self[file_id]))
+ for file_id in common:
+ if old[file_id] != self[file_id]:
+ delta.append((old.id2path(file_id), self.id2path(file_id),
+ file_id, self[file_id]))
+ return delta
+
def remove_recursive_id(self, file_id):
"""Remove file_id, and children, from the inventory.
-
+
:param file_id: A file_id to remove.
"""
to_find_delete = [self._byid[file_id]]
=== modified file 'bzrlib/remote.py'
--- a/bzrlib/remote.py 2008-12-07 16:46:56 +0000
+++ b/bzrlib/remote.py 2008-12-07 17:40:43 +0000
@@ -776,10 +776,10 @@
self._ensure_real()
return self._real_repository.add_inventory(revid, inv, parents)
- def add_inventory_delta(self, basis_revision_id, delta, new_revision_id,
- parents):
+ def add_inventory_by_delta(self, basis_revision_id, delta, new_revision_id,
+ parents):
self._ensure_real()
- return self._real_repository.add_inventory_delta(basis_revision_id,
+ return self._real_repository.add_inventory_by_delta(basis_revision_id,
delta, new_revision_id, parents)
def add_revision(self, rev_id, rev, inv=None, config=None):
=== modified file 'bzrlib/repository.py'
--- a/bzrlib/repository.py 2008-12-07 16:46:56 +0000
+++ b/bzrlib/repository.py 2008-12-07 17:40:43 +0000
@@ -67,10 +67,10 @@
class CommitBuilder(object):
"""Provides an interface to build up a commit.
- This allows describing a tree to be committed without needing to
+ This allows describing a tree to be committed without needing to
know the internals of the format of the repository.
"""
-
+
# all clients should supply tree roots.
record_root_entry = True
# the default CommitBuilder does not manage trees whose root is versioned.
@@ -119,7 +119,11 @@
self._generate_revision_if_needed()
self.__heads = graph.HeadsCache(repository.get_graph()).heads
- self.basis_delta = []
+ self._basis_delta = []
+ # API compatibility, older code that used CommitBuilder did not call
+ # .record_delete(), which means the delta that is computed would not be
+ # valid. Callers that will call record_delete() should call
+ # .will_record_deletes() to indicate that.
self._recording_deletes = False
def _validate_unicode_text(self, text, context):
@@ -253,38 +257,55 @@
if ie.file_id not in basis_inv:
# add
result = (None, path, ie.file_id, ie)
- self.basis_delta.append(result)
+ self._basis_delta.append(result)
return result
elif ie != basis_inv[ie.file_id]:
# common but altered
# TODO: avoid tis id2path call.
result = (basis_inv.id2path(ie.file_id), path, ie.file_id, ie)
- self.basis_delta.append(result)
+ self._basis_delta.append(result)
return result
else:
# common, unaltered
return None
+ def get_basis_delta(self):
+ """Return the complete inventory delta versus the basis inventory.
+
+ This has been built up with the calls to record_delete and
+ record_entry_contents. The client must have already called
+ will_record_deletes() to indicate that they will be generating a
+ complete delta.
+
+ :return: An inventory delta, suitable for use with apply_delta, or
+ Repository.add_inventory_by_delta, etc.
+ """
+ if not self._recording_deletes:
+ raise AssertionError("recording deletes not activated.")
+ return self._basis_delta
+
def record_delete(self, path, file_id):
"""Record that a delete occured against a basis tree.
This is an optional API - when used it adds items to the basis_delta
being accumulated by the commit builder. It cannot be called unless the
- method recording_deletes() has been called to inform the builder that a
- delta is being supplied.
+ method will_record_deletes() has been called to inform the builder that
+ a delta is being supplied.
:param path: The path of the thing deleted.
:param file_id: The file id that was deleted.
"""
if not self._recording_deletes:
raise AssertionError("recording deletes not activated.")
- self.basis_delta.append((path, None, file_id, None))
+ delta = (path, None, file_id, None)
+ self._basis_delta.append(delta)
+ return delta
- def recording_deletes(self):
+ def will_record_deletes(self):
"""Tell the commit builder that deletes are being notified.
This enables the accumulation of an inventory delta; for the resulting
- commit to be valid deletes against the basis MUST be recorded via
+ commit to be valid, deletes against the basis MUST be recorded via
builder.record_delete().
"""
self._recording_deletes = True
@@ -358,7 +379,7 @@
else:
# add
delta = (None, path, ie.file_id, ie)
- self.basis_delta.append(delta)
+ self._basis_delta.append(delta)
return delta, False, None
else:
# we don't need to commit this, because the caller already
@@ -689,12 +710,12 @@
return self._inventory_add_lines(revision_id, parents,
inv_lines, check_content=False)
- def add_inventory_delta(self, basis_revision_id, delta, new_revision_id,
- parents):
+ def add_inventory_by_delta(self, basis_revision_id, delta, new_revision_id,
+ parents):
"""Add a new inventory expressed as a delta against another revision.
-
+
:param basis_revision_id: The inventory id the delta was created
- against.
+ against. (This does not have to be a direct parent.)
:param delta: The inventory delta (see Inventory.apply_delta for
details).
:param new_revision_id: The revision id that the inventory is being
@@ -705,9 +726,10 @@
graph access, as well as for those that pun ancestry with delta
compression.
- :returns: The validator(which is a sha1 digest, though what is sha'd is
- repository format specific) of the serialized inventory and
- the resulting inventory.
+ :returns: (validator, new_inv)
+ The validator(which is a sha1 digest, though what is sha'd is
+ repository format specific) of the serialized inventory, and the
+ resulting inventory.
"""
if not self.is_in_write_group():
raise AssertionError("%r not in write group" % (self,))
@@ -722,7 +744,7 @@
basis_inv = basis_tree.inventory
basis_inv.apply_delta(delta)
basis_inv.revision_id = new_revision_id
- return (self.add_inventory(new_revision_id, basis_inv, parents),
+ return (self.add_inventory(new_revision_id, basis_inv, parents),
basis_inv)
finally:
basis_tree.unlock()
@@ -3249,11 +3271,94 @@
return False
return True
+ def _fetch_batch(self, revision_ids, basis_id, basis_tree):
+ """Fetch across a few revisions.
+
+ :param revision_ids: The revisions to copy
+ :param basis_id: The revision_id of basis_tree
+ :param basis_tree: A tree that is not in revision_ids which should
+ already exist in the target.
+ :return: (basis_id, basis_tree) A new basis to use now that these trees
+ have been copied.
+ """
+ # Walk though all revisions; get inventory deltas, copy referenced
+ # texts that delta references, insert the delta, revision and
+ # signature.
+ text_keys = set()
+ pending_deltas = []
+ pending_revisions = []
+ for tree in self.source.revision_trees(revision_ids):
+ current_revision_id = tree.get_revision_id()
+ delta = tree.inventory._make_delta(basis_tree.inventory)
+ for old_path, new_path, file_id, entry in delta:
+ if new_path is not None:
+ if not (new_path or self.target.supports_rich_root()):
+ # We leave the inventory delta in, because that
+ # will have the deserialised inventory root
+ # pointer.
+ continue
+ # TODO: Do we need:
+ # "if entry.revision == current_revision_id" ?
+ if entry.revision == current_revision_id:
+ text_keys.add((file_id, entry.revision))
+ revision = self.source.get_revision(current_revision_id)
+ pending_deltas.append((basis_id, delta,
+ current_revision_id, revision.parent_ids))
+ pending_revisions.append(revision)
+ basis_id = current_revision_id
+ basis_tree = tree
+ # Copy file texts
+ from_texts = self.source.texts
+ to_texts = self.target.texts
+ to_texts.insert_record_stream(from_texts.get_record_stream(
+ text_keys, self.target._fetch_order,
+ not self.target._fetch_uses_deltas))
+ # insert deltas
+ for delta in pending_deltas:
+ self.target.add_inventory_by_delta(*delta)
+ # insert signatures and revisions
+ for revision in pending_revisions:
+ try:
+ signature = self.source.get_signature_text(
+ revision.revision_id)
+ self.target.add_signature_text(revision.revision_id,
+ signature)
+ except errors.NoSuchRevision:
+ pass
+ self.target.add_revision(revision.revision_id, revision)
+ return basis_id, basis_tree
+
+ def _fetch_all_revisions(self, revision_ids, pb):
+ """Fetch everything for the list of revisions.
+
+ :param revision_ids: The list of revisions to fetch. Must be in
+ topological order.
+ :param pb: A ProgressBar
+ :return: None
+ """
+ basis_id, basis_tree = self._get_basis(revision_ids[0])
+ batch_size = 100
+ for offset in range(0, len(revision_ids), batch_size):
+ self.target.start_write_group()
+ try:
+ pb.update('Transferring revisions', offset,
+ len(revision_ids))
+ batch = revision_ids[offset:offset+batch_size]
+ basis_id, basis_tree = self._fetch_batch(batch,
+ basis_id, basis_tree)
+ except:
+ self.target.abort_write_group()
+ raise
+ else:
+ self.target.commit_write_group()
+
@needs_write_lock
def fetch(self, revision_id=None, pb=None, find_ghosts=False):
"""See InterRepository.fetch()."""
revision_ids = self.target.search_missing_revision_ids(self.source,
revision_id, find_ghosts=find_ghosts).get_keys()
+ if not revision_ids:
+ return 0, 0
revision_ids = tsort.topo_sort(
self.source.get_graph().get_parent_map(revision_ids))
if not revision_ids:
@@ -3262,77 +3367,39 @@
# texts that delta references, insert the delta, revision and
# signature.
first_rev = self.source.get_revision(revision_ids[0])
+ if pb is None:
+ my_pb = ui.ui_factory.nested_progress_bar()
+ pb = my_pb
+ else:
+ my_pb = None
+ try:
+ self._fetch_all_revisions(revision_ids, pb)
+ finally:
+ if my_pb is not None:
+ my_pb.finished()
+ return len(revision_ids), 0
+
+ def _get_basis(self, first_revision_id):
+ """Get a revision and tree which exists in the target.
+
+ This assumes that first_revision_id is selected for transmission
+ because all other ancestors are already present. If we can't find an
+ ancestor we fall back to NULL_REVISION since we know that is safe.
+
+ :return: (basis_id, basis_tree)
+ """
+ first_rev = self.source.get_revision(first_revision_id)
try:
basis_id = first_rev.parent_ids[0]
# only valid as a basis if the target has it
self.target.get_revision(basis_id)
# Try to get a basis tree - if its a ghost it will hit the
# NoSuchRevision case.
- basis_tree = self.source.revision_tree(first_rev.parent_ids[0])
+ basis_tree = self.source.revision_tree(basis_id)
except (IndexError, errors.NoSuchRevision):
basis_id = _mod_revision.NULL_REVISION
basis_tree = self.source.revision_tree(basis_id)
- if pb is None:
- my_pb = ui.ui_factory.nested_progress_bar()
- pb = my_pb
- else:
- my_pb = None
- try:
- batch_size = 100
- for offset in range(len(revision_ids) / 100 + 1):
- self.target.start_write_group()
- try:
- pb.update('Transferring revisions', offset*batch_size,
- len(revision_ids))
- batch = revision_ids[offset*batch_size:(offset+1)*batch_size]
- text_keys = set()
- pending_deltas = []
- pending_revisions = []
- for tree in self.source.revision_trees(batch):
- current_revision_id = tree.get_revision_id()
- delta = tree.inventory._make_delta(basis_tree.inventory)
- for old_path, new_path, file_id, entry in delta:
- if new_path is not None:
- if not (new_path or self.target.supports_rich_root()):
- # We leave the inventory delta in, because that
- # will have the deserialised inventory root
- # pointer.
- continue
- text_keys.add((file_id, entry.revision))
- revision = self.source.get_revision(current_revision_id)
- pending_deltas.append((basis_id, delta,
- current_revision_id, revision.parent_ids))
- pending_revisions.append(revision)
- basis_id = current_revision_id
- basis_tree = tree
- # Copy file texts
- from_texts = self.source.texts
- to_texts = self.target.texts
- to_texts.insert_record_stream(from_texts.get_record_stream(
- text_keys, self.target._fetch_order,
- not self.target._fetch_uses_deltas))
- # insert deltas
- for delta in pending_deltas:
- self.target.add_inventory_delta(*delta)
- # insert signatures and revisions
- for revision in pending_revisions:
- try:
- signature = self.source.get_signature_text(
- revision.revision_id)
- self.target.add_signature_text(revision.revision_id,
- signature)
- except errors.NoSuchRevision:
- pass
- self.target.add_revision(revision.revision_id, revision)
- except:
- self.target.abort_write_group()
- raise
- else:
- self.target.commit_write_group()
- finally:
- if my_pb is not None:
- my_pb.finished()
- return len(revision_ids), 0
+ return basis_id, basis_tree
class InterOtherToRemote(InterRepository):
=== modified file 'bzrlib/tests/interrepository_implementations/__init__.py'
--- a/bzrlib/tests/interrepository_implementations/__init__.py 2008-04-30 20:09:39 +0000
+++ b/bzrlib/tests/interrepository_implementations/__init__.py 2008-12-05 22:18:47 +0000
@@ -122,6 +122,9 @@
result.append((InterKnitRepo,
pack_repo.RepositoryFormatKnitPack3(),
knitrepo.RepositoryFormatKnit3()))
+ result.append((InterKnitRepo,
+ pack_repo.RepositoryFormatKnitPack3(),
+ pack_repo.RepositoryFormatKnitPack4()))
return result
=== modified file 'bzrlib/tests/per_repository/__init__.py'
--- a/bzrlib/tests/per_repository/__init__.py 2008-12-07 16:46:56 +0000
+++ b/bzrlib/tests/per_repository/__init__.py 2008-12-07 17:40:43 +0000
@@ -858,8 +858,8 @@
result.addTests(basic_tests)
prefix = 'bzrlib.tests.per_repository.'
test_repository_modules = [
- 'test_add_inventory_delta',
'test_add_fallback_repository',
+ 'test_add_inventory_by_delta',
'test_break_lock',
'test_check',
# test_check_reconcile is intentionally omitted, see below.
=== renamed file 'bzrlib/tests/per_repository/test_add_inventory_delta.py' => 'bzrlib/tests/per_repository/test_add_inventory_by_delta.py'
--- a/bzrlib/tests/per_repository/test_add_inventory_delta.py 2008-12-03 23:10:26 +0000
+++ b/bzrlib/tests/per_repository/test_add_inventory_by_delta.py 2008-12-07 17:40:43 +0000
@@ -14,13 +14,13 @@
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
-"""Tests for Repository.add_inventory_delta."""
+"""Tests for Repository.add_inventory_by_delta."""
from bzrlib import errors, revision
from bzrlib.tests.per_repository import TestCaseWithRepository
-class TestAddInventoryDelta(TestCaseWithRepository):
+class TestAddInventoryByDelta(TestCaseWithRepository):
def _get_repo_in_write_group(self, path='repository'):
repo = self.make_repository(path)
@@ -28,13 +28,13 @@
self.addCleanup(repo.unlock)
repo.start_write_group()
return repo
-
+
def test_basis_missing_errors(self):
repo = self._get_repo_in_write_group()
try:
self.assertRaises(errors.NoSuchRevision,
- repo.add_inventory_delta, "missing-revision", [], "new-revision",
- ["missing-revision"])
+ repo.add_inventory_by_delta, "missing-revision", [],
+ "new-revision", ["missing-revision"])
finally:
repo.abort_write_group()
@@ -42,7 +42,7 @@
repo = self.make_repository('repository')
repo.lock_write()
self.addCleanup(repo.unlock)
- self.assertRaises(AssertionError, repo.add_inventory_delta,
+ self.assertRaises(AssertionError, repo.add_inventory_by_delta,
"missing-revision", [], "new-revision", ["missing-revision"])
def make_inv_delta(self, old, new):
@@ -80,8 +80,8 @@
repo_direct.commit_write_group()
repo_delta = self._get_repo_in_write_group('delta')
try:
- delta_validator, result = repo_delta.add_inventory_delta(revision.NULL_REVISION,
- delta, revid, [])
+ delta_validator, inv = repo_delta.add_inventory_by_delta(
+ revision.NULL_REVISION, delta, revid, [])
except:
repo_delta.abort_write_group()
raise
=== modified file 'bzrlib/tests/per_repository/test_commit_builder.py'
--- a/bzrlib/tests/per_repository/test_commit_builder.py 2008-12-07 16:46:56 +0000
+++ b/bzrlib/tests/per_repository/test_commit_builder.py 2008-12-07 17:40:43 +0000
@@ -147,7 +147,7 @@
parent_tree = tree.basis_tree()
parent_tree.lock_read()
self.addCleanup(parent_tree.unlock)
- builder = tree.branch.get_commit_builder([parent_tree.inventory])
+ builder = tree.branch.get_commit_builder([old_revision_id])
try:
ie = inventory.make_entry('directory', '', None,
tree.get_root_id())
@@ -159,10 +159,9 @@
# should be in the delta
got_new_revision = ie.revision != old_revision_id
if got_new_revision:
- self.assertEqual(
- ('', '', ie.file_id, ie),
- delta)
- self.assertEqual(delta, builder.basis_delta[-1])
+ self.assertEqual(('', '', ie.file_id, ie), delta)
+ # The delta should be tracked
+ self.assertEqual(delta, builder._basis_delta[-1])
else:
self.assertEqual(None, delta)
# Directories do not get hashed.
@@ -192,6 +191,63 @@
# but thats all the current contract guarantees anyway.
self.assertEqual(rev_id, tree.branch.repository.get_inventory(rev_id).revision_id)
+ def test_get_basis_delta(self):
+ tree = self.make_branch_and_tree(".")
+ self.build_tree(["foo"])
+ tree.add(["foo"], ["foo-id"])
+ old_revision_id = tree.commit("added foo")
+ tree.lock_write()
+ try:
+ self.build_tree(['bar'])
+ tree.add(['bar'], ['bar-id'])
+ basis = tree.branch.repository.revision_tree(old_revision_id)
+ basis.lock_read()
+ self.addCleanup(basis.unlock)
+ builder = tree.branch.get_commit_builder([old_revision_id])
+ total_delta = []
+ try:
+ parent_invs = [basis.inventory]
+ builder.will_record_deletes()
+ if builder.record_root_entry:
+ ie = basis.inventory.root.copy()
+ delta, _, _ = builder.record_entry_contents(ie, parent_invs,
+ '', tree, tree.path_content_summary(''))
+ if delta is not None:
+ total_delta.append(delta)
+ delta = builder.record_delete("foo", "foo-id")
+ total_delta.append(delta)
+ new_bar = inventory.make_entry('file', 'bar',
+ parent_id=tree.get_root_id(), file_id='bar-id')
+ delta, _, _ = builder.record_entry_contents(new_bar, parent_invs,
+ 'bar', tree, tree.path_content_summary('bar'))
+ total_delta.append(delta)
+ # All actions should have been recorded in the basis_delta
+ self.assertEqual(total_delta, builder.get_basis_delta())
+ builder.finish_inventory()
+ builder.commit('delete foo, add bar')
+ except:
+ tree.branch.repository.abort_write_group()
+ raise
+ finally:
+ tree.unlock()
+
+ def test_get_basis_delta_without_notification(self):
+ tree = self.make_branch_and_tree(".")
+ old_revision_id = tree.commit('')
+ tree.lock_write()
+ try:
+ parent_tree = tree.basis_tree()
+ parent_tree.lock_read()
+ self.addCleanup(parent_tree.unlock)
+ builder = tree.branch.get_commit_builder([old_revision_id])
+ # It is an error to expect builder.get_basis_delta() to be correct,
+ # if you have not also called will_record_deletes() to indicate you
+ # will be calling record_delete() when appropriate
+ self.assertRaises(AssertionError, builder.get_basis_delta)
+ tree.branch.repository.abort_write_group()
+ finally:
+ tree.unlock()
+
def test_record_delete(self):
tree = self.make_branch_and_tree(".")
self.build_tree(["foo"])
@@ -205,15 +261,16 @@
basis = tree.branch.repository.revision_tree(rev_id)
builder = tree.branch.get_commit_builder([rev_id])
try:
- builder.recording_deletes()
+ builder.will_record_deletes()
if builder.record_root_entry is True:
parent_invs = [basis.inventory]
del basis.inventory.root.children['foo']
builder.record_entry_contents(basis.inventory.root,
parent_invs, '', tree, tree.path_content_summary(''))
- builder.record_delete("foo", "foo-id")
- self.assertEqual(("foo", None, "foo-id", None),
- builder.basis_delta[-1])
+ # the delta should be returned, and recorded in _basis_delta
+ delta = builder.record_delete("foo", "foo-id")
+ self.assertEqual(("foo", None, "foo-id", None), delta)
+ self.assertEqual(delta, builder._basis_delta[-1])
builder.finish_inventory()
rev_id2 = builder.commit('delete foo')
except:
@@ -417,7 +474,7 @@
def mini_commit(self, tree, name, new_name, records_version=True,
delta_against_basis=True, expect_fs_hash=False):
"""Perform a miniature commit looking for record entry results.
-
+
:param tree: The tree to commit.
:param name: The path in the basis tree of the tree being committed.
:param new_name: The path in the tree being committed.
@@ -476,7 +533,8 @@
new_entry = builder.new_inventory[file_id]
if delta_against_basis:
expected_delta = (name, new_name, file_id, new_entry)
- self.assertEqual(expected_delta, builder.basis_delta[-1])
+ # The delta should be recorded
+ self.assertEqual(expected_delta, builder._basis_delta[-1])
else:
expected_delta = None
self.assertEqual(expected_delta, delta)
More information about the bazaar-commits
mailing list