[RFC] parsable test suite log

[Robert Collins]

On Wed, 2008-03-26 at 18:24 +1100, Martin Pool wrote:
> On Tue, Mar 25, 2008 at 2:21 PM, Robert Collins
> <robertc at robertcollins.net> wrote:
> > I'd like to do some machine processing of test suite runs.
> >
> >  In the first instance, I'd like to get a list of failed tests from the
> >  previous run. The workflow I'm anticipating is:
> >  a) write code
> >  run some tests
> >  loop a until code completed
> >  b) run all tests with --record
> 
> What would --record be?
> 
> How about just --subunit telling it to put the output in that style?

Well, I wasn't proposing changing the output style; rather forking the
result calls so that both our repoer, and a subunit reporter both get
invoked.

> >  c) take list of failures, fix one
> >  run just the failures
> >  loop c until no failures
> >  loop b until no failures
> >
> >  Now, I have a serialisation format I prepared earlier: subunit. I could
> >  plug this in trivially. subunits wire protocol records test runs and
> >  could allow some nice introspection stuff in future.
> >
> >  Alternatively, I can do a custom serialisation format like vila proposed
> >  for his fast-loading stuff; I'd rather not though.
> >
> >  Note that this isn't changing loading facilities at all - no shelf or
> >  anything; its just about being able to get info about the prior test run
> >  via python, rather than copy n paste.
> 
> That would be good.
> 
> (The following is based on last time I read about subunit; it might be wrong.)
> 
> There is some structure within our test output that isn't contemplated
> by subunit; specifically we have a traceback and some log output.
> Maybe it doesn't matter, we should just dump it as one text result and
> let the subunit client figure it out.

Yes.

> I would love to have a gui where i can click a test to (re)run it and
> click a line of output to go there in my editor.

'tribunal'. 

-Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080326/675a1074/attachment.pgp