[apparmor] [patch] parser language tests: force using a features file

[Steve Beattie]

On Thu, Apr 24, 2014 at 12:06:14PM -0700, Seth Arnold wrote:
> On Thu, Apr 24, 2014 at 12:09:42AM -0700, Steve Beattie wrote:
> > A simple python script is included that was used to generate the
> > features file based on the current feature set. I'm not sure how to
> > keep it up to date in an automated fashion as we add more supported
> > features, however. (make check can't just fail on the features
> > directory being different; we want builds and tests to run successfully
> > on older releases where the kernel may not support mediating all the
> > features we include.)
> 
> This looks like a useful start but I wouldn't be surprised if there's more
> pain to be had before we find a solution that makes sense for the
> "optionally supported" mediations.

I agree, more pain here is probably coming.

> The first thing that comes to mind is maintaining two piles of
> "expected" outcomes for each feature, but I really don't care for
> that much.

Interestingly, with the current set of committed language tests (I have
more forthcoming), only one ptrace test failed (ptrace/bad_10.sd). This
is because most of the parsing occurs before determining what mediation
features are supported by the kernel; only stuff like regex conversions
(and the error checks that go with them) occur after the features
check. The parser is (currently) almost entirely static with regards
to what language elements it will accept as input.

That said, less parser code is being exercised by the language tests
if when features are detected as not being supported, as some of the
post processing won't occur. But exercising the post processing code
completely is not the primary intent of the language tests; it's a
better target for the regression test suite.

> > Signed-off-by: Steve Beattie <steve at nxnw.org>
> 
> Anyway, this code as-is looks like a useful improvement while we can
> consider future options.
> 
> Acked-by: Seth Arnold <seth.arnold at canonical.com>

Thanks for the review.

-- 
Steve Beattie
<sbeattie at ubuntu.com>
http://NxNW.org/~steve/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/apparmor/attachments/20140424/0ee9807a/attachment.pgp>