Rev 94: Change the default loader from the regex to json if available. in

John Arbash Meinel john at
Tue Oct 13 22:13:26 BST 2009


revno: 94
revision-id: john at
parent: john at
committer: John Arbash Meinel <john at>
branch nick: trunk
timestamp: Tue 2009-10-13 16:13:20 -0500
  Change the default loader from the regex to json if available.
  In my own testing, using simplejson + speedups is both a faster parser than
  the regex, *and* results in less memory. Probably mostly because the random
  unicode escapes get translated into a single byte in memory, and integers are
  integers rather than strings.
-------------- next part --------------
=== modified file 'meliae/'
--- a/meliae/	2009-10-13 21:02:35 +0000
+++ b/meliae/	2009-10-13 21:13:20 +0000
@@ -294,7 +294,7 @@
         return [o for o in self.objs.itervalues() if o.type_str == type_str]
-def load(source, using_json=False, show_prog=True):
+def load(source, using_json=None, show_prog=True):
     """Load objects from the given source.
     :param source: If this is a string, we will open it as a file and read all
@@ -302,7 +302,10 @@
         out, so the object should be an iterator of json lines.
     :param using_json: Use simplejson rather than the regex. This allows
         arbitrary ordered json dicts to be parsed but still requires per-line
-        layout.
+        layout. Set to 'False' to indicate you want to use the regex, set to
+        'True' to force using simplejson. None will probe to see if simplejson
+        is available, and use it if it is. (With _speedups built, simplejson
+        parses faster and more accurately than the regex.)
     cleanup = None
     if isinstance(source, str):
@@ -315,6 +318,8 @@
         input_size = sum(map(len, source))
         input_size = 0
+    if using_json is None:
+        using_json = (simplejson is not None)
         return _load(source, using_json, show_prog, input_size)

More information about the bazaar-commits mailing list