[RFC] parsable test suite log

[Vincent Ladeuil]

>>>>> "robert" == Robert Collins <robertc at robertcollins.net> writes:

    robert> I'd like to do some machine processing of test suite runs.
    robert> In the first instance, I'd like to get a list of failed tests from the
    robert> previous run. The workflow I'm anticipating is:
    robert> a) write code
    robert> run some tests
    robert> loop a until code completed
    robert> b) run all tests with --record
    robert> c) take list of failures, fix one
    robert> run just the failures
    robert> loop c until no failures
    robert> loop b until no failures

Don't we all ? ;-)

    robert> Now, I have a serialisation format I prepared
    robert> earlier: subunit. I could plug this in
    robert> trivially. subunits wire protocol records test runs

What will you serialize ? Test ID, status (OK/FAIL), time to run ?

    robert> and could allow some nice introspection stuff in
    robert> future.

Yum.

    robert> Alternatively, I can do a custom serialisation format
    robert> like vila proposed for his fast-loading stuff; I'd
    robert> rather not though.

bzr selftest -v | grep '(FAIL|ERROR)$' | cut --the-crap > failures

 (not tested) should work *today* but with many caveats.

    robert> Note that this isn't changing loading facilities at
    robert> all - no shelf or anything; its just about being able
    robert> to get info about the prior test run via python,
    robert> rather than copy n paste.

approve

        Vincent