Next Release of Checkbox (Week #05 Release Candidate) available in Testing PPA

Thu Feb 1 08:15:34 UTC 2018

Hello all!

The next release of Checkbox is available in the testing PPA:
https://code.launchpad.net/~checkbox-dev/+archive/testing

- checkbox-ng 1.0.0rc1
- checkbox-support 0.37.0rc1
- plainbox-provider-resource 0.35.0rc1
- plainbox-provider-checkbox 0.42.0rc1
- plainbox-provider-sru 1.10.0rc1
- plainbox-provider-tpm2 1.7.0rc1
- plainbox-provider-certification-client 0.33.0rc1
- plainbox-provider-certification-server 0.37.0rc1

Also note that both tpm2-tools and tpm2-tss have be copied to our
dev/testing/stable ppa from Ivan Hu's test ppa (version 1.1) to allow 
testing
of TPM2 hardware on 16.04.

Please help testing this version (including TPM2 certification tests).

HIGHLIGHTS
==========

- Plainbox is now included in checkbox-ng v1.0, back to ONE checkbox 
package!
- XML submission format is gone, replaced by a tarball.
- New HTML report (JQuery-mobile powered), included in the submission 
tarball.
- LOTS of legacy cleanup.

*Bug fixes*
http://goo.gl/jR438j

*Changelog*

checkbox (checkbox-ng + plainbox):
+ fix clashes of units of the same content from diff. namespaces
+ Refresh checkbox-ng docs with plainbox versions
+ fix provisioning of containers when running test-in-lxc
+ Load exporter units from the checkbox-ng package
+ Cleanup container tests
+ Cleanup translations
+ Run desktop sru test plan as a normal user
+ Update po/POTFILES.in
+ silence EPERM on log_dir when using provider_manager
+ exporter:tar: Add the HTML report to the archive
+ exporter:html: New JQuery Mobile based report
+ provider:exporter: New data files needed for the JQuery Mobile HTML report
+ session:state: New set of helper functions meant to be used in Jinja2 
exporters
+ MANIFEST.in: Cleanup
+ commands:run: Access job_state_map from current state
+ docs: add a description of what 'silent' means in UI config
+ docs: fix the launcher example that causes crash
+ config: add 'hide-automated' as a valid option to ui.output
+ docs: add info about stock_reports = none
+ add logic for handling stock_reports = none
+ launcher: use OneOrTheOtherValidator for stock_reports variable
+ config: add OneOrTheOtherValidator
+ Remove whitelist support
+ Remove local jobs support
+ config: add 'none' as a valid stock_report value
+ sa: add _PianoUI a nearly silent UI
+ ctrl.py: better warning msg when a parameter is missing from the resource
+ expand ~ for paths for file transports
+ Import get_providers from plainbox.impl.providers
+ Remove RFC822 exporter
+ Remove lazymod imports
+ highlevel.py: Remove obsolete comment
+ buildsystems.py: Remove 3.2 workarounds
+ Remove obsolete vendorized argparse modules
+ Remove GzipFile subclass for python3.2
+ Remove legacy session storage support
+ Remove python3.2 storage methods
+ Remove vendor/phablet.py
+ Unvendorize funcsigs
+ test_ctrl: Remove session_dir after testing
+ Unvendorize enum module
+ Remove support for experimental glibc-based command runner
+ Remove python3 version check (3.2 is gone with Ubuntu 12.04)
+ Remove daily-package-testing obsolete tools
+ Remove now unnecessary estimated_duration code and simplify
+ ctrl.py: better warning msg when a parameter is missing from the resource
+ sa: add _PianoUI a nearly silent UI
+ Use _PianoUI for bootstrap job output
+ Use job ID in bootstrap job header
+ Allow jinja2 template syntax for non template jobs
+ unit: Add helper functions to detect if checkbox runs on ubuntu core
+ Remove PatternProxy and dependency on padme
+ unit: Add helper functions to detect if checkbox runs on ubuntu core
+ use tr_purpose() for purpose instead of tr_purpose
+ use os.getgroups and drop importing posix
+ fix method naming for running jobs
+ simpleUI extension
+ basic implementation of checkbox-cli remote-*
+ session: Assistant2 WIP
+ move RootViaSudoWithPass ctrl from converged
+ change import to reflect vendorization
+ vendorize rpyc 3.4.4
+ Remove PatternProxy and dependency on padme
+ Fix consuming configs from SNAP_DATA
+ exporter:xlsx: Only Display 3 lines max in I/O log cells
+ migrate embedded_providers from converged
+ add side-loading of providers from home
+ urwid_ui: fix crash when reusing testplanbrowser
+ enable rerunning jobs that were skipped due to failed deps
+ when rerunning place dependees after the dependent
+ secure: add sudo_broker module
+ fix duplicated rerunning of failed resource job
+ trusted-launcher: Add siblings generated jobs to the job_list used by PTL1
+ fix crash when no tests are available for selection
+ trusted-launcher: Add siblings generated jobs to the job_list used by PTL1
+ Remove XML/HEXR exporter support
+ Remove the XML/HEXR exporter
+ fail properly on bad configs
+ make bare 'checkbox-cli' invocation read configs
+ enable rerunning jobs that were skipped due to failed deps
+ when rerunning place dependees after the dependent
+ unit.py: Add __system_env__ to the parameters used by jinja2 templates
+ session:state: Add the before suspend job as a dependency of its after 
suspend sibling

checkbox-support:
+ snap_utils: add a print conf file method to config
+ Use snapctl -d flag to always get a document
+ prevent broken pipes when zenity terminates before the flush
+ Add nmea_test script, configure entry point
+ udev parser: Use product name to detect CANBUS on classic
+ udev parser: Add the new INFINIBAND category (pci subclass 07)
+ snap_utils: ensure values are stringified
+ Remove the XML submission parser and dependency on lxml
+ Requires requests_unixsocket only for python >= 3.5
+ setup.py: add requests and requests_unixsocket
+ snap_utils:config: add top-level functions to handle config changes
+ snap_utils:config: use ValueError on invalid key
+ remove vendorized mock library
+ use standard mock
+ snap_utils: add function writing checkbox.conf
+ snap_utils: add function to load config_vars from snap
+ snap_utils: add get_snapctl_config function
+ Take all script code out of the lib
+ Refactor test logic into snap_utils directory
+ Add entry points for snap_configuration script
+ Create snap_configuration script

plainbox-provider-resource:
+ Update requirements to depend on checkbox-ng
+ Fix fwts resource by using checkbox-support-fwts_test
+ Restore the wireless_sta_protocol (previously available in snappy 
provider)
+ snapd_resource: return an empty response instead of exiting with 0
+ snapd_resource: Stop raising exception if snapd is not installed
+ udev_resource: Add INFINIBAND to the list of filter options
+ graphics resource: report cmd to switch GPUs
+ override the driver name to amdgpu-pro only for vendor == amd
+ graphics: report how to switch GPUs in the record

plainbox-provider-checkbox:
+ Update requirements to depend on checkbox-ng
+ bin:graphics_env: No need to set provider on AMD DRI3 config
+ Support elder nmcli api when running wifi_nmcli_test
+ Compatible with Trusty to get net dirver info
+ led: tidy category and test plan naming
+ submission: call dev parse command using python3 -m plainbox
+ Skip gpu_test on Artful or later
+ led: make all jobs use category in checkbox namespace
+ led: import jobs/testplan/category from p-p-snappy
+ Create after-suspend manual usb type-C tests and related nested test plans
+ Create test plans for the usb preinserted jobs
+ Add fwts_desktop_diagnosis test to misc-client-cert-blockers test plan
+ firmware/no_ACPI_REV_interface is cert blocker
+ usb-c: Move all usb type C tests to whitelist
+ Create after-suspend manual usb type-C tests and related nested test plans
+ Add fwts_desktop_diagnosis test to misc-client-cert-blockers test plan
+ firmware/no_ACPI_REV_interface is cert blocker
+ Update wireless template jobs to check slot connections only on ubuntu 
core
+ Delete unused wireless_extension job
+ fix detecting usb removal
+ add a step that asks the operator to wait until the watcher is established
+ detect inserted usb drives even if they're not automounted
+ Create test plans for the usb preinserted jobs
+ add after-gpu-switch S3-stress test
+ Purge xml_sanitize from the provider
+ Remove base64 encoding and archiving/compression of collections of files
+ Create test plans for the usb preinserted jobs
+ add after-gpu-switch S3-stress test
+ Purge xml_sanitize from the provider
+ Remove base64 encoding and archiving/compression of collections of files
+ Fixed cpu_offlining test for systems with more than 9 cores
+ add pcieport to reqs of automated gpu switch job
+ Don't run the lxc test as root to avoid creating .config as root, 
which breaks restarting checkbox lp: #1729866
+ add automated switch-to-gpu templated job
+ add a logger command to write stressor info into syslog to more 
quickly identify stressors causing OOPSes lp: #1731560
+ wireless: Use also-after-suspend-manual to create jobs depending on 
manual suspend
+ add fullscreen glxgears test
+ bin/virtualization: added retry mechanism to downloading LXC images 
from web lp: #1729080
+ usb-c: Move all usb type C tests to whitelist
+ wifi_nmcli_test: handle nmcli bug on zesty
+ Update test plans for nesting
+ Remove old wireless tests now covered by after-suspend flag
+ Import wireless tests form p-p-snappy

plainbox-provider-sru:
+ Update requirements to depend on checkbox-ng
+ Remove opencv tests from test plan
+ Remove single log check test
+ Remove base64 encoding
+ Fix dkms_build_validation to reduce false alarms.
+ Use templated wireless jobs
+ Remove obex get and send after suspend

plainbox-provider-tpm2:
+ Update requirements to depend on checkbox-ng

plainbox-provider-certification-client:
+ Update requirements to depend on checkbox-ng

plainbox-provider-certification-server:
+ Update requirements to depend on checkbox-ng
+ Created new launchers that are test specific and do not allow 
modification lp: #1742455
+ Cleared up the bits around the LXD files and URLs necessary for testing.
+ Update LXD precheck to specify that we need an actual URL, even for 
locally hosted files. lp: #1729351
+ PXU files: added call-trace check and syslog attach jobs to all test 
plans lp: #1729067
+ Removed CPU Offlining test from whitelists. lp: #1727336

*FWTS 18.01*
Complete changelog:
https://wiki.ubuntu.com/FirmwareTestSuite/ReleaseNotes/18.01.00

Cheers,

Sylvain Pineau