ACK: [B][SRU][PATCH 0/1] Fix the return value for skipped tests in kselftest/net/rtnetlink.sh

[Colin Ian King]

On 11/09/2020 09:55, Po-Hsu Lin wrote:
> BugLink: https://bugs.launchpad.net/bugs/1895258
> 
> == SRU Justification ==
> The rtnetlink.sh in kselftest/net from 4.15 kernel will fail with:
>  selftests: rtnetlink.sh
>  ========================================
>  PASS: policy routing
>  PASS: route get
>  PASS: preferred_lft addresses have expired
>  PASS: tc htb hierarchy
>  PASS: gre tunnel endpoint
>  PASS: bridge setup
>  PASS: ipv6 addrlabel
>  PASS: set ifalias e1dfde61-e5a7-415f-8bd9-924e9796038f for test-dummy0
>  PASS: vrf
>  PASS: vxlan
>  SKIP: fou: iproute2 too old
>  SKIP: macsec: iproute2 too old
>  not ok 1..10 selftests: rtnetlink.sh [FAIL]
> 
> The return value for this script is 1.
> 
> Look into the code, it's the kci_test_encap_fou() test that causes
> this issue:
> 
>       ip fou help 2>&1 |grep -q 'Usage: ip fou'
>       if [ $? -ne 0 ];then
>             echo "SKIP: fou: iproute2 too old"
>             return 1
>       fi
> 
> It should at least return ksft_skip=4 (or 0 with older release that
> does not support this skip code in kselftest framework)
> 
> == Fix ==
> * 57aefc7c226d ("selftests: net: return Kselftest Skip code for
> skipped tests")
> 
> This patch needs to be backported into Bionic, as some newer tests
> does not exist in Bionic.
> 
> == Test ==
> Tested directly on the affected kernel, the test will be marked as:
> not ok 1..10 selftests:  rtnetlink.sh [SKIP]
> 
> == Regression Potential ==
> Low, changes limited to test scripts. And the kselftest framework in
> Bionic can handle this ksft_skip=4 return code properly, tests won't
> be marked as failed because of this.
> 
> Although IMO in this case the test should be marked as PASS instead of
> SKIP in the end, as not all tests were skipped, but that's something
> we could fix with upstream in the future.
> 
> Shuah Khan (Samsung OSG) (1):
>   selftests: net: return Kselftest Skip code for skipped tests
> 
>  tools/testing/selftests/net/netdevice.sh    | 16 ++++++++++------
>  tools/testing/selftests/net/psock_tpacket.c |  4 +++-
>  tools/testing/selftests/net/rtnetlink.sh    | 15 +++++++++------
>  3 files changed, 22 insertions(+), 13 deletions(-)
> 

LGTM. Thanks.

Acked-by: Colin Ian King <colin.king at canonical.com>