[ACT][PATCH 0/1] UBUNTU: SAUCE: ubuntu_lxc: improve test granularity and idempotency

[Po-Hsu Lin]

This is a major change to how we test the lxc package.

In the current version, we test it on Xenial and older release with
tests shipped in the lxc-tests package by an "exercise" shell script
and call this task "exercise". On newer releases, this package no
longer exists and the script has been move to the debian/tests/ of
the lxc source package. In this case we run it with autopkgtests.

The downside of this "exercise" task we're using is that with all
the sub-tests running inside it is hard to do the result review and
hinting. Reviewers will need to navigate through the test report to
see which test case is failing, also, the hinting is not very precise
as we can only hint the "exercise" task instead of hinting the
sub-test directly.

Re-write the test, build and run tests with the commands converted
from debian/tests/exercise script in Ubuntu LXC package source tree
on github [1], run sub-tests one-by-one so they can be exposed on the
test report.

Change tested on Dev Jenkins with an AMD64 KVM instance, overall result
is looking good:
  * The test cases we ran in the current version are fully covered
    by this patch.
  * This patch will have 1 extra python3-api test (bug 1844673)
  * X-hwe: 4 tests failing with bug 1939537, lxc-test-state-server
    disabled, 34 tests passed (current version: 35 passed 4 failed)
  * B: 40 tests passed + python3-api passed (current version: 39 passed)
  * F: 43 tests passed + python3-api passed (current version: 45 passed)
    This is because the focal branch on Github falls behind, issue
    reported and should be resolved soon [2]
  * H: 45 tests passed + python3-api passed (current version: 45 passed)

The only thing we can't achieve for the moment is the IGNORED test,
(for example the lxc-test-state-server test) as we don't have a
skipped stated available. In the current version it will be printed:
    IGNORED: lxc-tests: lxc-test-state-server
    ...
    SUMMARY: pass=39, fail=0, ignored=1

With this patch they will just get neglected.

[1] https://github.com/lxc/lxc-pkg-ubuntu
[2] https://github.com/lxc/lxc-pkg-ubuntu/issues/10

Po-Hsu Lin (1):
  UBUNTU: SAUCE: ubuntu_lxc: improve test granularity and idempotency

 ubuntu_lxc/control       |  56 +++++++++++++-
 ubuntu_lxc/exercise      | 158 ---------------------------------------
 ubuntu_lxc/ubuntu_lxc.py |  86 ++++++++++++++-------
 3 files changed, 112 insertions(+), 188 deletions(-)
 delete mode 100644 ubuntu_lxc/exercise

-- 
2.25.1