Rev 3714: A bit of a Frankenstein now, but it drops memory dramatically. in http://bzr.arbash-meinel.com/branches/bzr/1.8-dev/lighter_log_file

[John Arbash Meinel]

At http://bzr.arbash-meinel.com/branches/bzr/1.8-dev/lighter_log_file

------------------------------------------------------------
revno: 3714
revision-id: john at arbash-meinel.com-20080918164436-tlhfuf010iz9gf5e
parent: john at arbash-meinel.com-20080918162150-vi5xcsfwrgaql0it
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: lighter_log_file
timestamp: Thu 2008-09-18 11:44:36 -0500
message:
  A bit of a Frankenstein now, but it drops memory dramatically.
  
  Now we build up new nodes into a list, and cache them as tuples.
  If we end up computing the frozenset() for a node, we then cache it
  as such.
  Memory consumption is now only 101MB, though processing time is 53s.
-------------- next part --------------
=== modified file 'bzrlib/log.py'
--- a/bzrlib/log.py	2008-09-18 16:21:50 +0000
+++ b/bzrlib/log.py	2008-09-18 16:44:36 +0000
@@ -578,7 +578,7 @@
         parents = parent_map[rev]
         rev_ancestry = None
         if text_key in modified_text_versions:
-            rev_ancestry = set([rev])
+            rev_ancestry = [rev]
         for parent in parents:
             if parent not in ancestry:
                 # parent is a Ghost, which won't be present in
@@ -594,6 +594,8 @@
                 # revisions. If it doesn't then we don't need to create a new
                 # set.
                 parent_ancestry = frozenset(ancestry[parent])
+                # We had to compute the frozenset, so just cache it
+                ancestry[parent] = parent_ancestry
                 new_revisions = parent_ancestry.difference(rev_ancestry)
                 if new_revisions:
                     # We need to create a non-frozen set so that we can
@@ -605,13 +607,19 @@
         if rev_ancestry is None:
             ancestry[rev] = ()
         else:
-            ancestry[rev] = tuple(rev_ancestry)
+            if isinstance(rev_ancestry, list):
+                ancestry[rev] = tuple(rev_ancestry)
+            else:
+                ancestry[rev] = rev_ancestry
 
     def is_merging_rev(r):
         parents = parent_map[r]
         if len(parents) > 1:
             leftparent = parents[0]
-            left_ancestry = frozenset(ancestry[leftparent])
+            left_ancestry = ancestry[leftparent]
+            left_ancestry = frozenset(left_ancestry)
+            # It doesn't help to cache this left_ancestry set because we've
+            # won't be accessing it again.
             for rightparent in parents[1:]:
                 right_ancestry = ancestry[rightparent]
                 if not left_ancestry.issuperset(right_ancestry):