[PATCH] Start the work of adding ACPI compliance tests

[Alex Hung]

Hi Al,

That makes sense to me.

I often get questions like "why is this test fail but I don't see any
impacts".  For example, method _PCL seems not used in Linux kernel but
many BIOS has incorrect implementation.  ACPI compliances is usually
my best answers

Cheers,
Alex Hung

On Thu, Oct 29, 2015 at 12:18 AM, Al Stone <al.stone at linaro.org> wrote:
> On 10/27/2015 09:31 PM, Alex Hung wrote:
>> Hi Al,
>>
>> This is certain a great idea. FWTS has some tests that check multiple
>> specs or multiple functions within a spec, such as PCIe ASPM (PCIe and
>> ACPI FADT) and Chassis type (SMBIOS and ACPI FADT), but most of them are
>> in smaller scope.
>>
>> Can you share more information on what you are going to do? For example,
>> is a test called "acpi/reduce_hardware" to be created and to be added to
>> --acpicompliance for the scenario you provided? If this is the case, can
>> we add it to acpitests instead of acpicompliance? Of course, nothing
>> stops us from putting a test in both acpitests and acpicompliance, but a
>> clear definitions for each will always help. :)
>
> Sure, this is all open work via Linaro, so I'm glad to share and discuss the
> plans.  And right, it's not that FWTS is _not_ checking for spec compliance
> at all, it's more that I want to focus very clearly on that -- and many of
> the tests could be in both sets.
>
> The distinction I make between acpitests and acpicompliance is that, if a
> test in the former fails, an OS will not boot and run.  For the latter, if
> a test fails, it is because something in the syntax or semantics of the ACPI
> tables violate the intent of the specification.  Reduced hardware is a good
> example -- there are supposedly platforms out there that have the flag set,
> but then still use the fields in the FADT that are supposed to be zero.  If
> I were to try to add something to acpitests to verify those fields are zero,
> it would not make sense -- the OS will boot properly, and in fact non-zero
> values are required for some platforms to boot at all.  However, this is clearly
> in violation of the spec, so an acpicompliance test _must_ fail.
>
> The plan is to first add two tests: (1) detailed MADT checking, particularly
> for subtable lengths and contents, and (2) reduced hardware checking.  For the
> first of these, the Linux kernel checking is minimal and there are multiple
> cases where there are spec violations but the platform is supported by Linux;
> the result will be a combination of the existing tests and new parts.  For the
> second, it's primarily checking table fields for non-zero values.
>
> Where the files live is not that critical to me; I've started putting them in
> acpi/compliance just for grins and giggles, if the test is new or dramatically
> modified.
>
> Then, the plan is to examine the existing ACPI tests and reuse everything I
> can.  I'm honestly not sure how that will ultimately turn out -- being sort of
> lazy, I don't want to write new code if I don't have to :).  It may require
> some reorganization of some tests, or breaking them out differently.  My
> thought was to just approach each test one at a time and do what makes sense.
>
> Perhaps in parallel, or perhaps after that, the plan is to re-examine the tests
> from the standpoint of the ARMv8 SBSA/SBBR documents and subset them again to
> focus on those.  But, that's after getting some beefy ACPI compliance done....
>
> Does that help clarify?
>
>> Cheers,
>> Alex Hung
>>
>>
>> On 10/24/2015 05:07 AM, Al Stone wrote:
>>> So, I'd like to start both re-using FWTS tests, and adding new tests, that
>>> specifically focus on verifying that ACPI tables are in full compliance with
>>> the ACPI specification.  As I understand it, ACPI tests in FWTS have in the
>>> past looked for problem spots that were previously encountered, so as not to
>>> encounter them a second time.  Compliance with the specification may or may
>>> not be the same thing, so I'd like to separate them out.
>>>
>>> For example, reduced hardware mode cannot really be enforced in Linux.  There
>>> are reportedly platforms that claim to be reduced hardware but then use the
>>> fields in tables that the specification says they should not use.  Since these
>>> are already supported, these platforms are not broken, but they do violate the
>>> spec.  In that case, I would expect the ACPI tables on such platforms to be
>>> able to pass --acpitests, but definitely fail --acpicompliance.
>>>
>>> There are similar issues with MADT subtables.  For instance, one platform that
>>> is supported in Linux uses a subtable ID that is a reserved value, and is not
>>> able to boot without that subtable.  So again, it would likely pass when using
>>> the --acpitests option, but it should definitely fail with --acpicompliance.
>>>
>>> If this approach is acceptable, I'll start adding the proper flags to any of
>>> the existing tests that can be re-used, and I'll submit additional tests as
>>> soon as I get them writtem.
>>>
>>>
>>> Al Stone (1):
>>>   Add in the notion of ACPI compliance tests.
>>>
>>>  README_SOURCE.txt                |  1 +
>>>  src/lib/include/fwts_framework.h |  9 +++++----
>>>  src/lib/src/fwts_framework.c     | 15 +++++++++++----
>>>  3 files changed, 17 insertions(+), 8 deletions(-)
>>>
>>
>>
>
>
> --
> ciao,
> al
> -----------------------------------
> Al Stone
> Software Engineer
> Linaro Enterprise Group
> al.stone at linaro.org
> -----------------------------------
>
> --
> fwts-devel mailing list
> fwts-devel at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/fwts-devel



-- 
Cheers,
Alex Hung