[MERGE] test framework distinguishes skips
John Arbash Meinel
john at arbash-meinel.com
Fri Jul 7 15:51:28 BST 2006
Martin Pool wrote:
> On 6 Jul 2006, Aaron Bentley <aaron.bentley at utoronto.ca> wrote:
>>> Nice function, but maybe not quite the right name. How about
>>> @test_requires or @test_dependency?
>> Yeah, I did think about that. It's a bit awkward, because the parameter
>> is an exception, e.g. ParamikoNotPresent. I thought
>> @dependency_failure(ParamikoNotPresent) made more sense than
>> @test_requires(ParamikoNotPresent).
>>
>> But the exception does actually name the missing dependency, so it would
>> be possible to do @test_requires("paramiko"). Would you prefer that?
>
> Or even just skip_on_exception(ParamikoNotPresent)?
I probably like this one the most. But it may be too generic. While I
realize that ParamikoNotPresent is a dependency exception, the decorator
looks like it is useful for any exception that would cause you to skip.
I'm not against a decorator like that. But the point was to narrow down
the focus, and make it clear that we are skipping because of a missing
dependency.
Maybe just:
skip_on_dependency_exception(ParamikoNotPresent)
Or
skip_on_exception(ParamikoNotPresent, dependency=True)
Then we parameterize TestSkipped to include *why* the test was skipped.
Which can be 'dependency missing', 'unicode failure', etc.
I also kind of which this type of decorator would let you pass in a
message string to give more information:
skip_on_exception(ParamikoNotPresent, msg='Test requires paramiko')
But if we can include all that information in the parameters, we don't
need another parameter for msg.
>
>> Even expected failures should be more visible than they are now. I only
>> recently discovered that one of my machines was skipping a bunch of
>> tests because its console is ISO-8859-1.
>
> It'd be good to have 0 skips when running it on a typical developer's
> machine, where we can expect to have all the dependencies installed.
> Then if something does start to skip we can look into why.
>
> However we may end up with disjoint sets of things which can only be
> tested on particular platforms. Separating the skips as you have should
> help a lot: personally I would like to run the suite with the constraint
> of "it's ok to skip things that require another OS, but everything else
> must pass." (Hm, there may still be restrictions about default encoding
> or locale, but perhaps they'd be ok in utf-8?)
Well, the full test suite runs with 'LANG=C', which causes lots of
things to skip because of Unicode issues.
Right now, I believe all tests are run if you have a Linux box with
LANG=*.UTF-8
Except for things that are skipped because they are adapted to formats
that don't support them. (Old-format branches that can't be initialized,
LocalTransport doesn't have 'bogus_url' support, and really can't, etc).
But those shouldn't really be TestSkipped, since they are invalid tests.
However, for a while, I think SFTP didn't have bogus_url support, and it
really should (as well for ftp, etc).
So those should be TestSkipped() indicating that we should fix them.
While LocalTransport should be a TestNotPossible().
John
=:->
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060707/4212b416/attachment.pgp
More information about the bazaar
mailing list