Rev 6: Handle when there are parents, but no compression parents. in http://bzr.arbash-meinel.com/plugins/xdelta_repo

[John Arbash Meinel]

At http://bzr.arbash-meinel.com/plugins/xdelta_repo

------------------------------------------------------------
revno: 6
revision-id: john at arbash-meinel.com-20090220213824-4uk44qqjij1t0wmm
parent: john at arbash-meinel.com-20090220205915-jwcsa57f0v9x7nnv
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: xdelta_repo
timestamp: Fri 2009-02-20 15:38:24 -0600
message:
  Handle when there are parents, but no compression parents.
  
  Also change the encoding buffer size, to make the default 10% larger than
  the input text, or 1MB, whichever is larger. This avoids OOM errors, albiet
  probably not optimally.
-------------- next part --------------
=== modified file 'xdelta_repo.py'
--- a/xdelta_repo.py	2009-02-20 20:59:15 +0000
+++ b/xdelta_repo.py	2009-02-20 21:38:24 +0000
@@ -161,7 +161,10 @@
                 compression_parents = None
             else:
                 parents = entry[3][0]
-                compression_parents = entry[3][1]
+                if len(entry[3]) > 1:
+                    compression_parents = entry[3][1]
+                else:
+                    compression_parents = None
             value = entry[2]
             method = 'group'
             result[key] = (self._node_to_position(entry),
@@ -502,9 +505,16 @@
             else:
                 refs = (record.parents,)
                 source = None
-            # XXX: Eventually we'll want to include 'source'
-            comp_bytes = xd3.encode_memory(total_bytes, source=source,
-                                           flags=xd3.SEC_DJW)
+            # Give at least a 1MB buffer, or 10% larger than the input,
+            # whichever is larger
+            out_buf_size = max(len(total_bytes) * 1.1, 1*1024*1024)
+            try:
+                comp_bytes = xd3.encode_memory(total_bytes, source=source,
+                                               flags=xd3.SEC_DJW,
+                                               output_buf_size=out_buf_size)
+            except Exception, e:
+                # we probably ran out of memory
+                import pdb; pdb.set_trace()
             index, start, length = self._access.add_raw_records(
                 [(None, len(comp_bytes))], comp_bytes)[0]
             # For now, we have no compression parents