Generic selftest questions

[Maritza Mendez]

Thanks Martin.  My plan for windows is to do clean production installs
on three or four different machines and cross-correlate selftest logs
to see if anything stands out over multiple runs and focus on those.
Not as ambitious as Vincent's plan but someplace to start.

Fyi I spent some time reading http.test and I do not see anything that
shouts concern.  But I'm just getting started.

-M

On 6/18/09, Martin Pool <mbp at sourcefrog.net> wrote:
> 2009/6/19 Maritza Mendez <martitzam at gmail.com>:
>>
>> Since I am seeing issues (which I log at launchpad when I have enough
>> info)
>> which I do not understand, I am running 'bzr --no-plugins selftest' on
>> clean
>> installs of the latest packaged builds (PPA for Ubuntu and Windows
>> default
>> installer for WinXP).  And I am finding failures so I've decided I need
>> to
>> start from clean installs.  The issues I am seeing are too many to dump
>> on
>> the mailing list and too poorly understood by me to file good bug
>> reports.
>> Some of the issues may be silly things like not having a port open for
>> bzr
>> server.  Both januty and WinXP running bzr 1.15-1 (both with python 2.6)
>> will finish 'bzr --no-plugins selftest'.  On linux, I get the hang in
>> test_http reported here a couple days ago.  On WinXP, I get "80 err, 21
>> fail, 6 missing" and ultimately a hang on a different part of test_http
>> (TestBadProtocol).
>>
>> Is there a log of known failures by platform?
>
> There is not a separate log.  The goal is that on every platform,
> every test will either pass or at least give a known failure.  So if
> you identify that a particular test always fails on Windows, the best
> option is probably to file a bug for it and submit a patch that turns
> it into a KnownFailure pointing to that bug.
>
> If it's not at all clear what's going wrong then possible just raising
> KnownFailure with a comment that the cause is unclear is better than
> filing a low-content bug.
>
> Then we should get to a state where at least we know that no new
> failures are occurring.  Vincent is working on setting up a build farm
> to repeatedly test on Windows.
>
> As you say, just reporting lots of failures is too much detail to be
> useful.
>
>> What should my next step be?
>
> Probably trying to find the easiest bug to fix, or one that you can
> fix with some help from somebody here.  These can be hard to
> collaboratively debug if they hang and it can't be reproduced on
> someone else's machine, so one that actually gives a specific failure
> is a good start - or perhaps strace or running pdb can give you enough
> data about what's going wrong that someone else can help.
>
>> I want to provide feedback but only if it is useful.
>
> I do appreciate it.
>
> --
> Martin <http://launchpad.net/~mbp/>
>