[merge][rfc] don't re-run tests in LANG=C from 'make check'?

[Vincent Ladeuil]

>>>>> "martin" == Martin Pool <mbp at sourcefrog.net> writes:

    martin> === modified file 'Makefile'
    martin> --- Makefile    2007-12-17 01:45:32 +0000
    martin> +++ Makefile    2008-02-07 05:29:54 +0000
    martin> @@ -32,8 +32,6 @@

    martin>  check: docs extensions
    martin>         $(PYTHON) -Werror -O ./bzr selftest -1v $(tests)
    martin> -       @echo "Running all tests with no locale."
    martin> -       LC_CTYPE= LANG=C LC_ALL= ./bzr selftest -1v $(tests) 2>&1 |
    martin> sed -e 's/^/[ascii] /'


    martin> The second test roughly doubles the pqm merge time
    martin> because of running the test suite twice.

It also doubles the tests done isn't ?

    martin> (Not quite double, because the compilation and the
    martin> merge itself is done only once.)

This is quite negligible in my experience.

    martin> I think this is unnecessary and should be removed:
    martin> the majority of tests are not particularly dependent
    martin> on the locale.

We have had more than a few bug reports regarding locale related
problems though.

    martin> When there is something unicode-dependent, we should
    martin> be trying to test it in a way that does not depend on
    martin> the tester's locale

Parametrize the test for a set of various locales ? Each
developer will have to install the appropriate packages... :-/

    martin> - I would say if this ever fails, we should add a new
    martin> test that is specific to that bug.

We do that don't we ? ;)

    martin> In a way we might as well have all the coverage we
    martin> can get, but the pqm run time does slow things down,
    martin> particularly at a release.

Another way to address that will be to provide a way to tell pqm:
"Trust me, I've run /these/ tests" by some signature (produced by
selftest itself for the revid asking to be merged). Pqm could
then run only the missing tests. 

Just an idea but may be worth discussing.

        Vincent