Generic selftest questions

[Martin Pool]

2009/6/19 Maritza Mendez <martitzam at gmail.com>:
>
> Since I am seeing issues (which I log at launchpad when I have enough info)
> which I do not understand, I am running 'bzr --no-plugins selftest' on clean
> installs of the latest packaged builds (PPA for Ubuntu and Windows default
> installer for WinXP).  And I am finding failures so I've decided I need to
> start from clean installs.  The issues I am seeing are too many to dump on
> the mailing list and too poorly understood by me to file good bug reports.
> Some of the issues may be silly things like not having a port open for bzr
> server.  Both januty and WinXP running bzr 1.15-1 (both with python 2.6)
> will finish 'bzr --no-plugins selftest'.  On linux, I get the hang in
> test_http reported here a couple days ago.  On WinXP, I get "80 err, 21
> fail, 6 missing" and ultimately a hang on a different part of test_http
> (TestBadProtocol).
>
> Is there a log of known failures by platform?

There is not a separate log.  The goal is that on every platform,
every test will either pass or at least give a known failure.  So if
you identify that a particular test always fails on Windows, the best
option is probably to file a bug for it and submit a patch that turns
it into a KnownFailure pointing to that bug.

If it's not at all clear what's going wrong then possible just raising
KnownFailure with a comment that the cause is unclear is better than
filing a low-content bug.

Then we should get to a state where at least we know that no new
failures are occurring.  Vincent is working on setting up a build farm
to repeatedly test on Windows.

As you say, just reporting lots of failures is too much detail to be useful.

> What should my next step be?

Probably trying to find the easiest bug to fix, or one that you can
fix with some help from somebody here.  These can be hard to
collaboratively debug if they hang and it can't be reproduced on
someone else's machine, so one that actually gives a specific failure
is a good start - or perhaps strace or running pdb can give you enough
data about what's going wrong that someone else can help.

> I want to provide feedback but only if it is useful.

I do appreciate it.

-- 
Martin <http://launchpad.net/~mbp/>