[Launchpad-dev] Bzr plugin for guessing relevant test modules: Fault line

[John Meinel]

You can use the same logic about hit rate that u did for merge --auto. If
something is hit 1000 times it isn't as relevant as one that is hit 1 time.
Similarly a test that covers 1000 lines of code isn't as interesting as one
that covers 10. Those are also usually faster tests to run. So you can do
"progressive" testing. The first tests are fast and focused, later ones
still cover the area, final ones aren't related. Especially w Robert's work
to use lsprof as the data source.
=:->
On Jun 2, 2011 8:14 PM, "Aaron Bentley" <aaron at aaronbentley.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 11-06-02 12:42 PM, vila wrote:
>>>>>>> Aaron Bentley <aaron at aaronbentley.com> writes:
>> Yup. I've been thinking about the coverage-based approach for quite some
>> time: run each test in isolation, establish which lines are covered,
>> then, from a diff, for each line modified, add the corresponding tests.
>
> The issue I thought of with coverage-based testing is that you might get
> spurious positives. For example, most of our tests will cover
> Branch.last_revision, so if you change that, which ones should you run?
>
>> There is quite a high price to pay upfront but then the increments
>> should reasonably fast to calculate and reasonably accurate. But I won't
>> bet this could replace a full test suite run.
>
> No, but Launchpad's test suite takes several hours to run. Because of
> this, shortcuts that help us find the likely test failures quickly are a
> bigger win for Launchpad than for bzr.
>
> Aaron
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk3nxN0ACgkQ0F+nu1YWqI3aPACghrjV/GLrFfsb6e5Dav+SRnQg
> rgUAn1lvsYHANaN9qb841ZIhlGlHPK8z
> =m/fU
> -----END PGP SIGNATURE-----
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/bazaar/attachments/20110602/dfbf0185/attachment.html>