[MERGE] Automatic discovery of tests

Martin Pool mbp at sourcefrog.net
Fri Nov 9 14:10:58 GMT 2007


I do have some good news for you: there are already already ways to
mark tests as ExpectedFailure, or requiresFeature, or NotApplicable.

Maybe a good way to begin would be just to go through and mark all the
currently problematic tests as known failures?
> That's where the problem is *today*. Ideally the test suite
> should be able to say: you lack symlink support you can't do
> that, your locale is badly configured fix it, your filesystem
> can't do that, etc.

done

There is some documentation in HACKING about this; if it's not good
please let me know.

> It may possible to go further in that direction by forbidding the
> use of 'import os' in bzr and forcing the use of osutils (and I
> don't think performance is a concern here if we just provide
> bindings).

That would be reasonable to add to test_source, assuming there's no
valid/necessary uses of it.

> - offer a way to test a server against bzr (see problems
>   encountered with ftp or http servers)

That would be nice, to have a way to say 'transport tests should use
this url for an existing server.  Then you could give it a file url
for a peculiar filesystem.

> - have the test suite run only the relevant tests (I change a bit
>   of code, run the 3 (40, 1027...) impacted tests

That would be nice, but seems hard to determine mechanically.  (In
theory you could go from code coverage data.)  However, in practice
using --first plus the name of the module you modified works quite
well.

> I'm annoyed by the 'UserWarning: file bla bla was not explicitly
> unlocked' noise.

Is this really hard?  It annoys me too but seems like it should be a
one-line fix if someone tracks it down.

I really like having the test suite working well on Ubuntu and would
like everyone to have that.

-- 
Martin



More information about the bazaar mailing list