[PATCH] Start the work of adding ACPI compliance tests

[Al Stone]

On 10/27/2015 09:31 PM, Alex Hung wrote:
> Hi Al,
> 
> This is certain a great idea. FWTS has some tests that check multiple
> specs or multiple functions within a spec, such as PCIe ASPM (PCIe and
> ACPI FADT) and Chassis type (SMBIOS and ACPI FADT), but most of them are
> in smaller scope.
> 
> Can you share more information on what you are going to do? For example,
> is a test called "acpi/reduce_hardware" to be created and to be added to
> --acpicompliance for the scenario you provided? If this is the case, can
> we add it to acpitests instead of acpicompliance? Of course, nothing
> stops us from putting a test in both acpitests and acpicompliance, but a
> clear definitions for each will always help. :)

Sure, this is all open work via Linaro, so I'm glad to share and discuss the
plans.  And right, it's not that FWTS is _not_ checking for spec compliance
at all, it's more that I want to focus very clearly on that -- and many of
the tests could be in both sets.

The distinction I make between acpitests and acpicompliance is that, if a
test in the former fails, an OS will not boot and run.  For the latter, if
a test fails, it is because something in the syntax or semantics of the ACPI
tables violate the intent of the specification.  Reduced hardware is a good
example -- there are supposedly platforms out there that have the flag set,
but then still use the fields in the FADT that are supposed to be zero.  If
I were to try to add something to acpitests to verify those fields are zero,
it would not make sense -- the OS will boot properly, and in fact non-zero
values are required for some platforms to boot at all.  However, this is clearly
in violation of the spec, so an acpicompliance test _must_ fail.

The plan is to first add two tests: (1) detailed MADT checking, particularly
for subtable lengths and contents, and (2) reduced hardware checking.  For the
first of these, the Linux kernel checking is minimal and there are multiple
cases where there are spec violations but the platform is supported by Linux;
the result will be a combination of the existing tests and new parts.  For the
second, it's primarily checking table fields for non-zero values.

Where the files live is not that critical to me; I've started putting them in
acpi/compliance just for grins and giggles, if the test is new or dramatically
modified.

Then, the plan is to examine the existing ACPI tests and reuse everything I
can.  I'm honestly not sure how that will ultimately turn out -- being sort of
lazy, I don't want to write new code if I don't have to :).  It may require
some reorganization of some tests, or breaking them out differently.  My
thought was to just approach each test one at a time and do what makes sense.

Perhaps in parallel, or perhaps after that, the plan is to re-examine the tests
from the standpoint of the ARMv8 SBSA/SBBR documents and subset them again to
focus on those.  But, that's after getting some beefy ACPI compliance done....

Does that help clarify?

> Cheers,
> Alex Hung
> 
> 
> On 10/24/2015 05:07 AM, Al Stone wrote:
>> So, I'd like to start both re-using FWTS tests, and adding new tests, that
>> specifically focus on verifying that ACPI tables are in full compliance with
>> the ACPI specification.  As I understand it, ACPI tests in FWTS have in the
>> past looked for problem spots that were previously encountered, so as not to
>> encounter them a second time.  Compliance with the specification may or may
>> not be the same thing, so I'd like to separate them out.
>>
>> For example, reduced hardware mode cannot really be enforced in Linux.  There
>> are reportedly platforms that claim to be reduced hardware but then use the
>> fields in tables that the specification says they should not use.  Since these
>> are already supported, these platforms are not broken, but they do violate the
>> spec.  In that case, I would expect the ACPI tables on such platforms to be
>> able to pass --acpitests, but definitely fail --acpicompliance.
>>
>> There are similar issues with MADT subtables.  For instance, one platform that
>> is supported in Linux uses a subtable ID that is a reserved value, and is not
>> able to boot without that subtable.  So again, it would likely pass when using
>> the --acpitests option, but it should definitely fail with --acpicompliance.
>>
>> If this approach is acceptable, I'll start adding the proper flags to any of
>> the existing tests that can be re-used, and I'll submit additional tests as
>> soon as I get them writtem.
>>
>>
>> Al Stone (1):
>>   Add in the notion of ACPI compliance tests.
>>
>>  README_SOURCE.txt                |  1 +
>>  src/lib/include/fwts_framework.h |  9 +++++----
>>  src/lib/src/fwts_framework.c     | 15 +++++++++++----
>>  3 files changed, 17 insertions(+), 8 deletions(-)
>>
> 
> 


-- 
ciao,
al
-----------------------------------
Al Stone
Software Engineer
Linaro Enterprise Group
al.stone at linaro.org
-----------------------------------