[MERGE] test framework distinguishes skips

[Martin Pool]

On  6 Jul 2006, Aaron Bentley <aaron.bentley at utoronto.ca> wrote:
> > Nice function, but maybe not quite the right name.  How about
> > @test_requires or @test_dependency?
> 
> Yeah, I did think about that.  It's a bit awkward, because the parameter
> is an exception, e.g. ParamikoNotPresent.  I thought
> @dependency_failure(ParamikoNotPresent) made more sense than
> @test_requires(ParamikoNotPresent).
> 
> But the exception does actually name the missing dependency, so it would
> be possible to do @test_requires("paramiko").  Would you prefer that?

Or even just skip_on_exception(ParamikoNotPresent)?

> Even expected failures should be more visible than they are now.  I only
> recently discovered that one of my machines was skipping a bunch of
> tests because its console is ISO-8859-1.

It'd be good to have 0 skips when running it on a typical developer's
machine, where we can expect to have all the dependencies installed.
Then if something does start to skip we can look into why.

However we may end up with disjoint sets of things which can only be
tested on particular platforms.  Separating the skips as you have should
help a lot: personally I would like to run the suite with the constraint
of "it's ok to skip things that require another OS, but everything else
must pass."  (Hm, there may still be restrictions about default encoding
or locale, but perhaps they'd be ok in utf-8?)

> > For some of these, like lsprof, I'd think you could just check the
> > dependency in the test class setup, can't you?
> 
> The two lsprof test cases each import lsprof to get their effect.  They
> share a TestCase with many other tests, so if we're to do handle the
> dependency in setUp, we should split the lsprof tests out into their own
> TestCase.  Using the decorator on the tests seems simpler to me, but we
> can do it that way if you want.

I had in mind to split them out into a separate test case, but I didn't
look very much at the overall context, so if you want to use the
decoraotr that's OK with me.

> Glad you like it.  Now what's your favourite colour of sheep :-)?

That's more robert's department. :-)

-- 
Martin