[apparmor] [patch] utils: add python coverage generation

[Christian Boltz]

Hello,

Am Montag, 27. Oktober 2014 schrieb Steve Beattie:
> On Sun, Oct 26, 2014 at 12:49:12AM +0200, Christian Boltz wrote:
> > [1] hmm - what about   $(wildcard ../aa-* ../apparmor/*.py)   ?
> 
> To wit: relying on the above misses any changes to the actual
> test-*.py files themselves; 

oops ;-)

> one of the primary purposes to generate
> coverage data is to give people an incentive to add tests, and the
> reported coverage percentages provide a feedback mechanism that might
> do so (e.g. "Ooh, I increased the amount covered by 5 percentage
> points!"). Editing/adding tests and seeing no change in the coverage
> report could be a bit demoralizing.

Indeed ;-)

> Index: b/utils/test/Makefile
> ===================================================================
> --- a/utils/test/Makefile
> +++ b/utils/test/Makefile
> @@ -26,13 +26,31 @@ common/Make.rules: $(COMMONDIR)/Make.rul
>  	ln -sf $(COMMONDIR) .
>  endif
> 
> -.PHONY: clean check
> +COVERAGE_OMIT=test-*.py,common_test.py
> +ifneq ($(COVERAGE_OUT), )
> +HTML_COVR_ARGS=-d $(COVERAGE_OUT)
> +endif
> +
> +.PHONY: clean check coverage coverage-report coverage-html

Now that we have correct dependencies, does it still make sense to add 
coverage* to .PHONY?

>  ifndef VERBOSE
> -.SILENT: clean check
> +.SILENT: clean check .coverage coverage coverage-report coverage-html
> endif
> 
>  clean: _clean
> -	rm -rf __pycache__/ common
> +	rm -rf __pycache__/ common .coverage htmlcov
> 
>  check:
>  	export PYTHONPATH=.. ; $(foreach test, $(wildcard test-*.py), 
$(call
> pyalldo, $(test))) +
> +.coverage: $(wildcard ../aa-* ../apparmor/*.py test-*.py)

I'd replace test-*.py with just * in the dependencies to make sure 
changes to the test data files also invalidate the cache.

> +	export PYTHONPATH=.. ; $(foreach test, $(wildcard test-*.py),
> $(PYTHON) -m coverage run --branch -p $(test); ) 

This gives us a total coverage of 18%.

I found a way to improve the coverage to 29%: also run *test.py ;-)
so we should either rename those tests to test-*.py (or everything to 
$something_test.py) or change the Makefile to run them.

> +	$(PYTHON) -m
> coverage combine
> +
> +coverage: .coverage
> +
> +coverage-report: .coverage
> +	$(PYTHON) -m coverage report --omit="$(COVERAGE_OMIT)"
> +
> +coverage-html: .coverage
> +	$(PYTHON) -m coverage html --omit="$(COVERAGE_OMIT)"
> $(HTML_COVR_ARGS) 

With the .coverage dependencies changed to * and the other things 
changes or answered,
Acked-by: Christian Boltz <apparmor at cboltz.de>


Regards,

Christian Boltz
-- 
| Contains fubroutines for fontlinge, www.gesindel.de
           ^   [fontlinge 0.1rc4 - fontlinge::Subs manpage]
Nein, das ist C und ... na schön. Ich habe gelogen. Es ist ein
Tippfehler. Schäm... [Ratti]