ACK: [F-OEM-5.6/F-OEM-5.10/F/G][SRU][PATCH 0/2] Improve the test result processing for kselftests/net/pmtu.sh

[Tim Gardner]

Acked-by: Tim Gardner <tim.gardner at canonical.com>

On 5/24/21 12:58 AM, Po-Hsu Lin wrote:
> [Impact]
> The pmtu.sh test in net from ubuntu_kernel_selftests will be marked
> as failed even there is no obvious failure message from the test:
> $ sudo ./pmtu.sh
> TEST: ipv4: PMTU exceptions [ OK ]
> TEST: ipv6: PMTU exceptions [ OK ]
>    vxlan4 not supported
> TEST: IPv4 over vxlan4: PMTU exceptions [SKIP]
>    vxlan4 not supported
> TEST: IPv6 over vxlan4: PMTU exceptions [SKIP]
> TEST: IPv4 over vxlan6: PMTU exceptions [ OK ]
> TEST: IPv6 over vxlan6: PMTU exceptions [ OK ]
>    geneve4 not supported
> TEST: IPv4 over geneve4: PMTU exceptions [SKIP]
>    geneve4 not supported
> TEST: IPv6 over geneve4: PMTU exceptions [SKIP]
> TEST: IPv4 over geneve6: PMTU exceptions [ OK ]
> TEST: IPv6 over geneve6: PMTU exceptions [ OK ]
> TEST: IPv4 over fou4: PMTU exceptions [ OK ]
> TEST: IPv6 over fou4: PMTU exceptions [ OK ]
> TEST: IPv4 over fou6: PMTU exceptions [ OK ]
> TEST: IPv6 over fou6: PMTU exceptions [ OK ]
> TEST: IPv4 over gue4: PMTU exceptions [ OK ]
> TEST: IPv6 over gue4: PMTU exceptions [ OK ]
> TEST: IPv4 over gue6: PMTU exceptions [ OK ]
> TEST: IPv6 over gue6: PMTU exceptions [ OK ]
> TEST: vti6: PMTU exceptions [ OK ]
> TEST: vti4: PMTU exceptions [ OK ]
> TEST: vti4: default MTU assignment [ OK ]
> TEST: vti6: default MTU assignment [ OK ]
> TEST: vti4: MTU setting on link creation [ OK ]
> TEST: vti6: MTU setting on link creation [ OK ]
> TEST: vti6: MTU changes on link changes [ OK ]
>    vxlan4 not supported
> TEST: ipv4: cleanup of cached exceptions [SKIP]
> TEST: ipv6: cleanup of cached exceptions [ OK ]
> TEST: ipv4: list and flush cached exceptions [ OK ]
> TEST: ipv6: list and flush cached exceptions [ OK ]
> $ echo $?
> 1
> 
> This is because the test script treats all non-zero return code as a
> failure, thus it will be marked as FAILED when some sub-test got
> skipped.
> 
> [Fix]
> * ef1220a7d4bbdb selftests: pmtu.sh: use $ksft_skip for skipped return
>   code
> * 2a9d3716b810a4 selftests: pmtu.sh: improve the test result processing
> 
> This patchset will:
>    1. Use the kselftest framework skip code $ksft_skip to replace the
>       hardcoded SKIP return code.
>    2. Improve the result processing, the test will be marked as PASSED
>       if nothing goes wrong and not all the tests were skipped.
> 
> These have already landed in newer releases and this test does not
> exist in B, thus we just need this for F/F-OEM-5.6/F-OEM-5.10/G.
> 
> The first patch needs to be backported (except on F-OEM-5.10) as some
> test cases were not added yet.
> The second one can be cherry-picked.
> 
> [Test]
> Run this test directly on a patched kernel. The skipped test will cause
> failure to this test.
> 
> [Where problems could occur]
> Changes limited to testing tools, it's unlikely to cause any problem
> to kernel functions.
> 
> Po-Hsu Lin (2):
>    selftests: pmtu.sh: use $ksft_skip for skipped return code
>    selftests: pmtu.sh: improve the test result processing
> 
>   tools/testing/selftests/net/pmtu.sh | 67 ++++++++++++++++++++++---------------
>   1 file changed, 40 insertions(+), 27 deletions(-)
> 

-- 
-----------
Tim Gardner
Canonical, Inc