[RFC] parsable test suite log
Robert Collins
robertc at robertcollins.net
Wed Mar 26 07:42:18 GMT 2008
On Wed, 2008-03-26 at 18:24 +1100, Martin Pool wrote:
> On Tue, Mar 25, 2008 at 2:21 PM, Robert Collins
> <robertc at robertcollins.net> wrote:
> > I'd like to do some machine processing of test suite runs.
> >
> > In the first instance, I'd like to get a list of failed tests from the
> > previous run. The workflow I'm anticipating is:
> > a) write code
> > run some tests
> > loop a until code completed
> > b) run all tests with --record
>
> What would --record be?
>
> How about just --subunit telling it to put the output in that style?
Well, I wasn't proposing changing the output style; rather forking the
result calls so that both our repoer, and a subunit reporter both get
invoked.
> > c) take list of failures, fix one
> > run just the failures
> > loop c until no failures
> > loop b until no failures
> >
> > Now, I have a serialisation format I prepared earlier: subunit. I could
> > plug this in trivially. subunits wire protocol records test runs and
> > could allow some nice introspection stuff in future.
> >
> > Alternatively, I can do a custom serialisation format like vila proposed
> > for his fast-loading stuff; I'd rather not though.
> >
> > Note that this isn't changing loading facilities at all - no shelf or
> > anything; its just about being able to get info about the prior test run
> > via python, rather than copy n paste.
>
> That would be good.
>
> (The following is based on last time I read about subunit; it might be wrong.)
>
> There is some structure within our test output that isn't contemplated
> by subunit; specifically we have a traceback and some log output.
> Maybe it doesn't matter, we should just dump it as one text result and
> let the subunit client figure it out.
Yes.
> I would love to have a gui where i can click a test to (re)run it and
> click a line of output to go there in my editor.
'tribunal'.
-Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080326/675a1074/attachment.pgp
More information about the bazaar
mailing list