[MERGE] readdir, take 2.

[Martin Pool]

On Tue, Sep 2, 2008 at 3:20 PM, Robert Collins
<robertc at robertcollins.net> wrote:
> On Tue, 2008-09-02 at 15:06 +1000, Martin Pool wrote:
>>
>>
>> One of the ideas of the test-not-run framework, which I believe you
>> wanted, was that it would be obvious if things are not being tested.
>> Showing test stipple may remind people to do whatever is necessary to
>> get them built.  I believe at the moment the makefile will still pass
>> even if the extensions were not built.
>
> We achieved the things I wanted quite a while back - we put up a wiki
> page in Montreal, and it got implemented. The current stuff I feel quite
> dissatisfied with. Lets not segue off into that though.

At first this seemed like a nonsequiter, but I guess you're saying
there are things which are implemented now that were not in the
original spec?  As far as I know everything has been heading towards
it.

> PQM, which is our key arbiter for this sort of thing, is much more of a
> concern to me than people watching the test suite. When it spews output
> it becomes rapidly noise and people stop watching the test output
> altogether.

The plan as far as I'm aware is that selftest will gain an option
which tells it to fail rather than warn if there are any missing
dependencies, and pqm will use this.

>> At any rate I'm reluctant to add now tests that can silently not run
>> because if we do want to tighten it up later it may be hard to find
>> them.  But if it's not obvious how to do it the other way nevermind
>> for now.
>
> Well, I can require the extension which will cause test failures on
> windows and any platform without readdir.

No, I think failures would be excessive.  That's one reason why we
introduced results that are in between failure and success, so that we
can be more strict in some cases.

For example test_btree_index uses a Feature to indicate whether the
compiled extension is there or not.

-- 
Martin <http://launchpad.net/~mbp/>