[RFC][PERFORMANCE][PYREX][TIRED OF THESE YET?] *fast* walkdirs.

[Robert Collins]

On Wed, 2006-06-14 at 12:17 +1000, Martin Pool wrote:
> On 10/06/2006, at 3:15 AM, Alexander Belchenko wrote:
> 
> > Robert Collins пишет:
> >> The attached patch combines two things:  * Proof of concept work  
> >> for using pyrex for selected modules, to
> >> provide a C version and a python version. Its likely that the C  
> >> version
> >> would be platform specific in some cases, and not in other.
> >>  * A demonstration with pyrex of binding to readdir to get directory
> >> type information. When available this provides a 50% speedup to  
> >> the time
> >> to walk the directory tree - even when the whole tree is in hot  
> >> cache -
> >> for the 10K kernel-like-tree scenario.
> >> Obviously having C modules should be done with care given their
> >> downsides - loops in the C module will ignore Ctrl-C unless you
> >> explicitly check for it, bugs are harder to analyse, etc etc.
> >
> > May I suggest to keep this new C-modules in separate branch, not in  
> > bzr.dev until it goes mature?
> >
> > Because having C-modules require to compiling it and this is create  
> > some difficulties on windows (where C-compilers is not default part  
> > of installation).
> >
> > As compromise I suggest to make new C-extensions optional and give  
> > a chance to bzr to works even without it. I did not look at your  
> > patch so probably you already provide this option.
> 
> If possible I'd like them to actually be plugins loaded through the  
> regular mechanism.  They may need to patch themselves into bzr and  
> perhaps it's not always possible, but something to aim for.

I think that that increases the chance of skew in a single users install
between their C and python versions, and it does not change the ease or
difficulty of testing both implementations : either the thing they
replace has a factory where we can control the one used, or it acts like
a singleton, and we have to just test one during a test run.

The current test branch runs tests completely, then builds the
extensions and runs again. Its not completely ideal, but it achieves my
key goals:
 * Ensure that a python version exists and works.
 * Ensure that the C versions when built still pass tests.

-Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060615/b86c708a/attachment.pgp