Next Release of Checkbox (Week #45 Release Candidate) available in Testing PPA

Fri Nov 19 14:29:23 UTC 2021

Hi,

Thanks to Stanley the below nested parts problem is fixed in the oem 
provider.
Though I've spotted a few more ids that cause validation to fail, a fix 
a available here:

https://code.launchpad.net/~sylvain-pineau/oem-qa-checkbox/+git/oem-qa-checkbox/+merge/412143

Please review/merge and prepare a new version of the provider in the 
oem-qa ppa.
I can sync the release of checkbox to stable with yours.

Sylvain

On 18/11/2021 09:15, Pierre Equoy wrote:
> Hi Sylvain,
>
> I tested this version, then forgot to get back to you...
>
> Anyway, from what I've tested, I would say it works fine when using 
> test plans from the cert-client provider.
>
> *However, the change "Change: reorganise Desktop Preload plans" has an 
> impact on the OEM providers*, because they rely on 
> com.canonical.certification::client-cert-20-04, but this test plan was 
> renamed com.canonical.certification::client-cert-desktop-20-04.
>
> Please make sure the OEM PC QA leads (Erin, Gene, Stanley) are ready 
> to publish an OEM provider that point to the right test plan before 
> pushing this new version of Checkbox to stable.
>
> Cheers,
>
> On Mon, Nov 8, 2021 at 11:33 PM Sylvain Pineau 
> <sylvain.pineau at canonical.com <mailto:sylvain.pineau at canonical.com>> 
> wrote:
>
>     Hello all!
>
>     The next release of Checkbox is available in the testing PPA:
>     https://code.launchpad.net/~checkbox-dev/+archive/testing
>     <https://code.launchpad.net/~checkbox-dev/+archive/testing>
>
>     - checkbox-ng 1.15.0rc1
>     - checkbox-support 0.52.0rc1
>     - plainbox-provider-checkbox 0.60.0rc1
>     - plainbox-provider-certification-client 0.47.0rc1
>     - plainbox-provider-certification-server 0.56.0rc1
>     - plainbox-provider-tpm2 1.18.0rc1
>
>     Please help testing this version.
>
>     Bug fixes
>     https://bit.ly/3CWivgf <https://bit.ly/3CWivgf>
>
>     Changelog
>
>     checkbox-ng:
>     + Bump to v1.15.0rc1
>     + Add: auto retry submissions on non-interactive sessions
>     + Fix: quote the name of report on fail
>     + increment version to v1.15.0.dev0
>
>     checkbox-support:
>     + Bump to v0.52.0rc1
>     + Fix: audio volume unit tests failed
>     + Fix: Unified format string method
>     + Fix: Unable to get volume for mono audio
>     + Add: unique identification of usb devs in sysfs paser
>     + increment version to v0.52.0.dev0
>
>     plainbox-provider-certification-client:
>     + Bump to v0.47.0rc1
>     + Change: snaps are on stable and from Canonical
>     + Change: use automated for
>     client-cert-desktop-{18-04,20-04}-automated
>     + Change: update power-management-reboot-poweroff-cert-full id
>     + Change: use new nested TPM plan for desktop 20.04
>     + Change: 20-04 split stress from automated tests
>     + Change: 18-04 split stress from automated tests
>     + Change: apply the manual/automated split 18-04
>     + Change: apply the manual/automated split 20-04
>     + Change: reorganise Desktop Preload plans
>     + Change: use nested parts from TPM provider
>     + increment version to v0.47.0.dev0
>
>     plainbox-provider-certification-server:
>     + Bump to v0.56.0rc1
>     + Removed unneeded launcher now that testflinger dev work is done lp:
>     #1945352
>     + add server-ethernet-sru nested plan to regression testplan lp:
>     #1944989
>     + lower NVME read threshold to 400 to accept lower-speed NVME on ARM
>     systems lp: #1933191
>     + increment version to v0.56.0.dev0
>     + Just removing a little bit of redundancy
>
>     plainbox-provider-checkbox:
>     + Bump to v0.60.0rc1
>     + Detect usb3 on arm with different driver names
>     + Fix: Remove manual jobs from automated USB-C test plan
>     + Change: debsums & check_prelease desktop blockers
>     + Change: rename check_prerelease script
>     + Change: set miscellanea/check_prerelease to blocker
>     + Change: Use key event to simulate screen off
>     + some architectures don't support fwts s3 while manual test, copy
>     from
>     suspend_advance_auto part that if DUT doesn't support fwts s3, use
>     rtcwake instead.
>     + jobs:graphics: Remove the package requirement on nux-tools to work
>     with snaps
>     + ADD: Reintroduce ethernet tests for server regression testing
>     lp: #1944989
>     + Change: interlaced modes don't generate error
>     + Add: kernel-snap/booted-dtbs-match-current test
>     + Change: depend on debsums executable over package
>     + Return the test result of audio/valid-sof-firmware-sig
>     + Update CPUID.py to add Comet Lake lp: #1927016
>     + add paramters for stress-ng
>     + identify suspend mode for pm-graph test case
>     + Check whether screen orientation is fixed on laptops
>     + identify suspend mode for pm-graph test case
>     + bin/inxi_snapshot: Refresh the inxi script using the last
>     upstream version
>     + Fix: graceful exit on no supported camera res
>     + Fix: do not lose first res detected in camera test
>     + Add support for Rocket Lake CPU
>     + info/systemd-analyze: wait for boot to complete
>     + Added zzstd module to ignore list and re-ordered list
>     alphabetically
>     to make it easier to read and manage. lp: #1940797
>     + Change: bin/kernel_taint_test.py: move report_failures() to
>     main() and
>     code cleanup
>     + Change: make wwan/gsm-connection more verbose
>     + Change: bluetooth/detect checks device resource
>     + add double quote for RTC_DEVICE_FILE
>     + place RTC_DEVICE_FILE in environ field.
>     + add an if statement for judging which rtc device will be used in
>     suspend test. since some IoT projects have 2 rtcs and not always rtc0
>     supports wakeup event.
>     + Increase IPMI process timeout to 15 seconds to account for
>     slower BMCs
>     lp: #1937861
>     + Fix: sim-present and ThreeGppConnection test
>     + bin/kernel_taint_test.py: return 0 for passed cases, 1 for others
>     + Change: Align the test for watchdog config on classic and core image
>     + Add: iwlwifi microcode crash check
>     + increment version to v0.60.0.dev0
>     + Remove efibootmode test from server-regression testplan to avoid
>     failures on old systems we know aren't booting EFI mode
>     + Add: led test for specific AAEON devices
>     + Add: gpio test for specific AAEON devices
>     + Ubuntu core kernel snap failboot rollback test
>     + Change: wireless-cert-manual don't nest automated
>     + Change: full/manual/automated for power-management
>     + Change: ethernet/networking plans follow category
>     + Change: mobilebroadband don't next automated
>     + Add: a fingerprint-automated nested test plan
>     + Change: misc-cert-manual doesn't nest -automated
>     + Change: cpu-cert-manual doesn't nest -automated
>     + Filter snapd/save device from storage tests
>
>     plainbox-provider-tpm2:
>     + Bump to v1.18.0rc1
>     + getekcertificate.sh: skip the test without network connection
>     + Change: clevis jobs now honour tpm2.0 manifest
>     + Change: support testing in tpm2_capabilities.py
>     + Change: include clevis tests in Cert test plan
>     + Add: attachment of tpm event log using fwts
>     + Add: category nested parts for Cert
>     + increment version to v1.18.0.dev0
>
>     Cheers,
>
>     Sylvain Pineau
>
>     _______________________________________________
>     ce-certification-qa mailing list --
>     ce-certification-qa at lists.canonical.com
>     <mailto:ce-certification-qa at lists.canonical.com>
>     To unsubscribe send an email to
>     ce-certification-qa-leave at lists.canonical.com
>     <mailto:ce-certification-qa-leave at lists.canonical.com>
>
>
>
> -- 
> Pierre Equoy
> Lead QA Engineer | Canonical Ltd.
> www.canonical.com <http://www.canonical.com/> | www.ubuntu.com 
> <http://www.ubuntu.com/>