[RFC] parsable test suite log
v.ladeuil+lp at free.fr
Tue Mar 25 07:56:23 GMT 2008
>>>>> "robert" == Robert Collins <robertc at robertcollins.net> writes:
robert> I'd like to do some machine processing of test suite runs.
robert> In the first instance, I'd like to get a list of failed tests from the
robert> previous run. The workflow I'm anticipating is:
robert> a) write code
robert> run some tests
robert> loop a until code completed
robert> b) run all tests with --record
robert> c) take list of failures, fix one
robert> run just the failures
robert> loop c until no failures
robert> loop b until no failures
Don't we all ? ;-)
robert> Now, I have a serialisation format I prepared
robert> earlier: subunit. I could plug this in
robert> trivially. subunits wire protocol records test runs
What will you serialize ? Test ID, status (OK/FAIL), time to run ?
robert> and could allow some nice introspection stuff in
robert> Alternatively, I can do a custom serialisation format
robert> like vila proposed for his fast-loading stuff; I'd
robert> rather not though.
bzr selftest -v | grep '(FAIL|ERROR)$' | cut --the-crap > failures
(not tested) should work *today* but with many caveats.
robert> Note that this isn't changing loading facilities at
robert> all - no shelf or anything; its just about being able
robert> to get info about the prior test run via python,
robert> rather than copy n paste.
More information about the bazaar