Interpreting test results across test runs

[Marc Tardif]

Hi folks,

I would appreciate your feedback about interpreting test results across
multiple test runs. For example, consider the following test results for
the first test run of a given project:

  test1: Pass
  test2: Pass
  test3: Fail

Then, consider the following partial test results for a second test run
of the same project:

  test1: Pass

The second test run only executed the first test and, for the sake of
this example, we can safely assume that this was deliberate and the
test runner process did not crash and burn.

My question is: can we make any reasonable assumptions about the tests
that were not run? This can be a matter of opinion where one extreme
might not make any assumptions at all, whereas another extreme might
assume that test results remain the same until proven otherwise. So,
I'm calling for your opinions on what you consider is reasonable.

Thanks!

-- 
Marc Tardif <marc.tardif at canonical.com>
Freenode: cr3, Jabber: cr3 at jabber.org
1024D/72679CAD 09A9 D871 F7C4 A18F AC08 674D 2B73 740C 7267 9CAD