[MERGE] Automatic discovery of tests
Martin Pool
mbp at sourcefrog.net
Fri Nov 9 14:10:58 GMT 2007
I do have some good news for you: there are already already ways to
mark tests as ExpectedFailure, or requiresFeature, or NotApplicable.
Maybe a good way to begin would be just to go through and mark all the
currently problematic tests as known failures?
> That's where the problem is *today*. Ideally the test suite
> should be able to say: you lack symlink support you can't do
> that, your locale is badly configured fix it, your filesystem
> can't do that, etc.
done
There is some documentation in HACKING about this; if it's not good
please let me know.
> It may possible to go further in that direction by forbidding the
> use of 'import os' in bzr and forcing the use of osutils (and I
> don't think performance is a concern here if we just provide
> bindings).
That would be reasonable to add to test_source, assuming there's no
valid/necessary uses of it.
> - offer a way to test a server against bzr (see problems
> encountered with ftp or http servers)
That would be nice, to have a way to say 'transport tests should use
this url for an existing server. Then you could give it a file url
for a peculiar filesystem.
> - have the test suite run only the relevant tests (I change a bit
> of code, run the 3 (40, 1027...) impacted tests
That would be nice, but seems hard to determine mechanically. (In
theory you could go from code coverage data.) However, in practice
using --first plus the name of the module you modified works quite
well.
> I'm annoyed by the 'UserWarning: file bla bla was not explicitly
> unlocked' noise.
Is this really hard? It annoys me too but seems like it should be a
one-line fix if someone tracks it down.
I really like having the test suite working well on Ubuntu and would
like everyone to have that.
--
Martin
More information about the bazaar
mailing list