[MERGE] add performance tweaks.

[Robert Collins]

On Sun, 2006-06-11 at 22:34 +1000, Robert Collins wrote:
> I'm calling this a set of tweaks rather than major improvements - I see
> a consistent 300ms improvement with them:
> 
> test_one_add_kernel_like_tree OK  5056ms/ 9680ms
> 
> to
> 
> test_one_add_kernel_like_tree OK  4736ms/11013ms
> 
> In summary this removes a lot of basename calls, switches the directory
> walking to use walkdirs, adds slots to inventory objects - without doing
> the complete workover that Jan is working on (I hope this patch doesn't
> conflict with his work, but the constructor was starting to show up in
> my profiles).
> 
> Key hotspots remaining:
>  * encoding the output (2/18)
>  * fileid generation   (1/18)
>  * ignore checking     (3/18)
>  * serialization.      (8/18)
> 
> 
> My replacement xml serialiser halves the serialisation cost, but is not
> compatible with 0.8 [unless the bug 49224 bugfix is applied], and I've
> not yet profiled the cost of doing entity replacement to make a
> ascii-safe bytestream.
> 
> The ignore checking - I think dropping the number of defaults is
> probably the right thing to do. Checking a modest number (say
> CVS, .svn, .hg, .darcs, *~) by default, or allowing users to shrink the
> default, should allow this to be fast where needed - I suspect we will
> have a high time investment to get significantly better than we have
> now. (We can get 20% saved by using a non-matching form of the regex by
> default).
> 
> Fileid generation remains somewhat high. I plan to play with having a
> generator and passing in the fileid to make_entry.
>  10824 0  1.1408      0.2294   bzrlib.inventory:1222(make_entry)
> +10824 0  0.4944      0.2212   +bzrlib.workingtree:134(gen_file_id)
> +10240 0  0.3935      0.1611   +bzrlib.inventory:650(__init__)
>   +584 0  0.0235      0.0092   +bzrlib.inventory:540(__init__)
> 
> Lastly, the encoding : we pay a surprisingly high cost here:
>  10824 0  2.5402      0.3942   bzrlib.add:66(__call__)
> +32472 0  1.8984      0.9094   +codecs:174(write)
> 
> I'm going to reduce the 3 times call multiplier and see if that reduces
> the visible overhead. If that doesn't, we may want to make quiet the
> default ;).

And it does:
 10824 0 1.1788      0.2725   bzrlib.add:66(__call__)
+10824 0 0.6563      0.3208   +codecs:174(write)
+10824 0 0.2499      0.1393   +bzrlib.osutils:70(quotefn)

giving:
test_one_add_kernel_like_tree OK  4587ms/16230ms

--- bzrlib/add.py       2006-06-10 18:52:59 +0000
+++ bzrlib/add.py       2006-06-11 12:23:39 +0000
@@ -63,7 +63,7 @@
         if should_print is not None:
             self.should_print = should_print

-    def __call__(self, inv, parent_ie, path, kind):
+    def __call__(self, inv, parent_ie, path, kind,
_quote=bzrlib.osutils.quotefn):
         """Add path to inventory.

         The default action does nothing.
@@ -74,9 +74,7 @@
         """
         if not self.should_print:
             return
-        self._to_file.write('added ')
-        self._to_file.write(bzrlib.osutils.quotefn(path.raw_path))
-        self._to_file.write('\n')
+        self._to_file.write('added %s\n' % _quote(path.raw_path))


 # TODO: jam 20050105 These could be used for compatibility


-Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060611/102e25d4/attachment.pgp