[RFC][PERFORMANCE][PYREX][TIRED OF THESE YET?] *fast* walkdirs.

[Robert Collins]

The attached patch combines two things: 
 * Proof of concept work for using pyrex for selected modules, to
provide a C version and a python version. Its likely that the C version
would be platform specific in some cases, and not in other.
 * A demonstration with pyrex of binding to readdir to get directory
type information. When available this provides a 50% speedup to the time
to walk the directory tree - even when the whole tree is in hot cache -
for the 10K kernel-like-tree scenario.

Obviously having C modules should be done with care given their
downsides - loops in the C module will ignore Ctrl-C unless you
explicitly check for it, bugs are harder to analyse, etc etc.

However, performance wise, this take bzrlib.osutils.walkdirs from 157ms
to 77ms for me (os.walk is 210ms).

Given that this is a mandatory part of operations like 'status',
'commit', 'add', 'delete' (if we have an automatic mode for delete), I
think this is probably worth including.

I was going to play with pyrex a bit to evaluate it - and this is the
result of the evaluation.

Cheers,
Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pyrex-fast-walkdirs.patch
Type: text/x-patch
Size: 39101 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060610/990edeb0/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060610/990edeb0/attachment.pgp