[MERGE] add performance tweaks.

[Robert Collins]

I'm calling this a set of tweaks rather than major improvements - I see
a consistent 300ms improvement with them:

test_one_add_kernel_like_tree OK  5056ms/ 9680ms

to

test_one_add_kernel_like_tree OK  4736ms/11013ms

In summary this removes a lot of basename calls, switches the directory
walking to use walkdirs, adds slots to inventory objects - without doing
the complete workover that Jan is working on (I hope this patch doesn't
conflict with his work, but the constructor was starting to show up in
my profiles).

Key hotspots remaining:
 * encoding the output (2/18)
 * fileid generation   (1/18)
 * ignore checking     (3/18)
 * serialization.      (8/18)


My replacement xml serialiser halves the serialisation cost, but is not
compatible with 0.8 [unless the bug 49224 bugfix is applied], and I've
not yet profiled the cost of doing entity replacement to make a
ascii-safe bytestream.

The ignore checking - I think dropping the number of defaults is
probably the right thing to do. Checking a modest number (say
CVS, .svn, .hg, .darcs, *~) by default, or allowing users to shrink the
default, should allow this to be fast where needed - I suspect we will
have a high time investment to get significantly better than we have
now. (We can get 20% saved by using a non-matching form of the regex by
default).

Fileid generation remains somewhat high. I plan to play with having a
generator and passing in the fileid to make_entry.
 10824 0  1.1408      0.2294   bzrlib.inventory:1222(make_entry)
+10824 0  0.4944      0.2212   +bzrlib.workingtree:134(gen_file_id)
+10240 0  0.3935      0.1611   +bzrlib.inventory:650(__init__)
  +584 0  0.0235      0.0092   +bzrlib.inventory:540(__init__)

Lastly, the encoding : we pay a surprisingly high cost here:
 10824 0  2.5402      0.3942   bzrlib.add:66(__call__)
+32472 0  1.8984      0.9094   +codecs:174(write)

I'm going to reduce the 3 times call multiplier and see if that reduces
the visible overhead. If that doesn't, we may want to make quiet the
default ;).

Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: add-tweaks.patch
Type: text/x-patch
Size: 30859 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060611/4d95cd03/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060611/4d95cd03/attachment.pgp