[apparmor] [patch] utils: add python coverage generation
Steve Beattie
steve at nxnw.org
Mon Oct 27 19:51:16 UTC 2014
Hello,
On Sun, Oct 26, 2014 at 12:49:12AM +0200, Christian Boltz wrote:
> > To generate the coverage data, in the test subdirectory, do:
> >
> > make coverage
> >
> > This essentially runs make check, using a single python interpreter,
> > and records which lines and branches of the python code were
> > exercised.
> >
> > To view a text based report, after generating the coverage data, do:
> >
> > make coverage-report
>
> Instead of "after generating the coverage data", can't you just use a
> make dependency like
> coverage-report: coverage
> coverage-html: coverage
> ?
>
> (I noticed that you add coverage to .PHONY, and I understand the reason
> - getting the dependencies 100% correct would be hard[1]. Nevertheless I
> hope that we can use a make dependency for coverage-report and coverage-
> html)
>
You are correct that I worried about getting the dependencies correct,
and/or corner cases where potential users would end up looking at
out-of-sync coverage data.
> [1] hmm - what about $(wildcard ../aa-* ../apparmor/*.py) ?
To wit: relying on the above misses any changes to the actual
test-*.py files themselves; one of the primary purposes to generate
coverage data is to give people an incentive to add tests, and the
reported coverage percentages provide a feedback mechanism that might
do so (e.g. "Ooh, I increased the amount covered by 5 percentage
points!"). Editing/adding tests and seeing no change in the coverage
report could be a bit demoralizing.
Anyhow. I went ahead and updated the patch to attempt to do this. I
think I got the dependencies in good shape. Patch follows:
utils: add python coverage generation
This patch adds support for generating test coverage information for the
python utils.
To view a text based report, in the test subdirectory do:
make coverage-report
To generate detailed html reports, do:
make coverage-html
And then point your web browser at
$(YOUR_CURRENT_WORKING_TREE)/utils/test/htmlcov/index.html .
An alternate output location can be specified by setting the
COVERAGE_OUT variable, e.g.
make coverage-html COVERAGE_OUT=/tmp/coverage/
(the output directory does not need to exist beforehand.)
To generate only the coverage data, do:
make coverage
or
make .coverage
(The coverage data generated by python is stored in the .coverage
file.) This essentially runs make check, using a single python
interpreter, and records which lines and branches of the python code
were exercised.
Signed-off-by: Steve Beattie <steve at nxnw.org>
---
utils/test/Makefile | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)
Index: b/utils/test/Makefile
===================================================================
--- a/utils/test/Makefile
+++ b/utils/test/Makefile
@@ -26,13 +26,31 @@ common/Make.rules: $(COMMONDIR)/Make.rul
ln -sf $(COMMONDIR) .
endif
-.PHONY: clean check
+COVERAGE_OMIT=test-*.py,common_test.py
+ifneq ($(COVERAGE_OUT), )
+HTML_COVR_ARGS=-d $(COVERAGE_OUT)
+endif
+
+.PHONY: clean check coverage coverage-report coverage-html
ifndef VERBOSE
-.SILENT: clean check
+.SILENT: clean check .coverage coverage coverage-report coverage-html
endif
clean: _clean
- rm -rf __pycache__/ common
+ rm -rf __pycache__/ common .coverage htmlcov
check:
export PYTHONPATH=.. ; $(foreach test, $(wildcard test-*.py), $(call pyalldo, $(test)))
+
+.coverage: $(wildcard ../aa-* ../apparmor/*.py test-*.py)
+ export PYTHONPATH=.. ; $(foreach test, $(wildcard test-*.py), $(PYTHON) -m coverage run --branch -p $(test); )
+ $(PYTHON) -m coverage combine
+
+coverage: .coverage
+
+coverage-report: .coverage
+ $(PYTHON) -m coverage report --omit="$(COVERAGE_OMIT)"
+
+coverage-html: .coverage
+ $(PYTHON) -m coverage html --omit="$(COVERAGE_OMIT)" $(HTML_COVR_ARGS)
+
--
Steve Beattie
<sbeattie at ubuntu.com>
http://NxNW.org/~steve/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/apparmor/attachments/20141027/26388eba/attachment.pgp>
More information about the AppArmor
mailing list