Rev 3893: Use the 'chunked' interface to keep memory consumption minimal during revision_trees() in http://bzr.arbash-meinel.com/branches/bzr/1.11/get_record_stream_chunked

[John Arbash Meinel]

At http://bzr.arbash-meinel.com/branches/bzr/1.11/get_record_stream_chunked

------------------------------------------------------------
revno: 3893
revision-id: john at arbash-meinel.com-20081211010104-3tcii2strejk5252
parent: john at arbash-meinel.com-20081211005616-szoqqeabcyahy39u
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: get_record_stream_chunked
timestamp: Wed 2008-12-10 19:01:04 -0600
message:
  Use the 'chunked' interface to keep memory consumption minimal during revision_trees()
-------------- next part --------------
=== modified file 'bzrlib/repository.py'
--- a/bzrlib/repository.py	2008-12-10 04:34:21 +0000
+++ b/bzrlib/repository.py	2008-12-11 01:01:04 +0000
@@ -1680,14 +1680,15 @@
     def _iter_inventory_xmls(self, revision_ids):
         keys = [(revision_id,) for revision_id in revision_ids]
         stream = self.inventories.get_record_stream(keys, 'unordered', True)
-        texts = {}
+        text_chunks = {}
         for record in stream:
             if record.storage_kind != 'absent':
-                texts[record.key] = record.get_bytes_as('fulltext')
+                text_chunks[record.key] = record.get_bytes_as('chunked')
             else:
                 raise errors.NoSuchRevision(self, record.key)
         for key in keys:
-            yield texts[key], key[-1]
+            chunks = text_chunks.pop(key)
+            yield ''.join(chunks), key[-1]
 
     def deserialise_inventory(self, revision_id, xml):
         """Transform the xml into an inventory object.