New Checkbox release available in our Stable PPA! (2019-10-04)

Sylvain Pineau sylvain.pineau at
Fri Oct 4 12:22:54 UTC 2019

Hello all!

A new release of Checkbox is available in the stable PPA:

A special *thank you* *to QA* (Betty, Ray, Scott, Jeff) for their time 
and support
to debug, investigate, collect logs and even record videos of the bugs.

It includes a new version of the following components:

- checkbox-ng 1.5.0
- checkbox-support 0.42.0
- plainbox-provider-resource 0.41.0
- plainbox-provider-checkbox 0.49.0
- plainbox-provider-tpm2 1.11.0
- plainbox-provider-certification-client 0.38.0
- plainbox-provider-certification-server 0.43.0
- plainbox-provider-ipdt 0.2.0
- checkbox-provider-phoronix 0.2.0


This version no longer supports the passwordless solution based on pkexec
and "insecure" policy files.
The new unified job runner only supports sudo and will help having a 
common way
to start checkbox using just the 'remote' systemd unit as a backend runner.
The immediate consequence is that automated and headless invocations now 
to install a sudoer file to keep the passworless feature. Checkbox itself
won't provide this mechanism by default for debian packages installations.
Please add such file prior testing using for example:

$ echo "$USER ALL=(ALL:ALL) NOPASSWD: ALL" | sudo tee 

*Bug fixes*


+ Bump to v0.41.0
+ Bump to v0.41.0rc2
+ wireless_sta_protocol: Add 80211ax support
+ Bump to v0.41.0rc1
+ net_if_mgmnt: move snippets to text files
+ net_if_mgmnt: shutil.which for availability tests
+ net_if_mgmnt: tests of the job script
+ net_if_mgmnt: rework to make testable
+ usb: move call to checkbox-support script
+ net_if_management: init category_scope_manager
+ net_if_management: handle category renderer config
+ net_if_management: add resource job
+ udev_resource: add symlink_uuui to known attrs
+ increment version to v0.41.0.dev0

+ Bump to v1.11.0
+ Bump to v1.11.0rc2
+ Add TPM PCR7 unsealing automated test
+ Bump to v1.11.0rc1
+ Remove redirect &>/dev/null
+ increment version to v1.11.0.dev0

+ Bump to v0.38.0
+ Bump to v0.38.0rc2
+ Add iperf3 test to client-cert
+ add cdts test plan
+ add after-suspend-usb-c-cert-full
+ Bump to v0.38.0rc1
+ add missing "after-suspend-monitor-discrete-gpu-cert-full" test plan
+ Modify client cert 18.04 test plan for ODM testing
+ increment version to v0.38.0.dev0

+ Bump to v1.14.0
+ Bump to v1.14.0rc1
+ add fan-stress-reaction test to desktop SRU TP
+ Add stress-ng tests for cpu and disk
+ increment version to v1.14.0.dev0

+ Bump to v1.5.0
+ Bump to v1.5.0rc3
+ launcher: exit gracefully when TP selected in launcher is missing
+ execution: pep8
+ execution: Restore the get_differential_execution_environment()
+ Bump to v1.5.0rc2
+ don't close stdin_forwarder pipes twice
+ execution: use new way of killing the process
+ introduce a way to trim session state before exporting
+ introduce 'salvages' field to jobs
+ MANIFEST: remove path that doesn't exist anymore
+ execution: don't crash when no command (or it evaluates to nothing)
+ stubbox: remove qml-job-related stuff
+ plainbox: remove qml-shell leftovers
+ handle unsupported plugin types better
+ drop qml job support
+ drop the converged ui types
+ builtin-provider: use newline in collect-manifest prompts
+ Bump to v1.5.0rc1
+ subcommands: don't crash when there are no TPs available
+ run: support custom configs in run subcmd
+ config: remove CheckBoxConfig and rework check-config subcmd
+ config: always load configs from SNAP_DATA
+ sudo_broker: make sure that sudo is usable even when run as root
+ sudo_broker: don't check if sudo asks for pass when run as root
+ docs: add a page about the daemonic slave
+ remote: add sending of kills to the slave
+ docs: update chapter about configs
+ remote: support slave-side configs
+ config: let configs be appended when reading from string
+ config: outline loading of file configs to a function
+ sudo_broker: defer checking if sudo requires password
+ execution: use sane defaults for runner configuration
+ tests: update --help test
+ provider_manager: add test command to
+ subcommands: Ensure submissions generated with `merge-*` commands are 
+ po: Update after ptl removal
+ contrib: remove policykit files
+ resource-caching: display result even when comming from cache
+ result: change disk result access logging to info from warning
+ reesource_caching: move clearing of cache from SA
+ resource-caching: propagate info to the caller whether res is from cache
+ runner: remove unnecessary import
+ execution: fix crash on cachables; use the right run_command call
+ jobs: remove via_job field
+ remove trusted launcher
+ remote: fix a crash when resuming the session after resume
+ remote: relax back the polling rate
+ remote: fix race when background exec hasn't yet started
+ execution: fix crashing when resuming from an autorestart job
+ execution: add unified runner
+ assistant: add support for custom Runner kwargs
+ urwid_ui: fix crash in TestPlanBrowser
+ urwid_ui: add filterer for test plan names
+ docs: add dependency on before suspend job
+ docs: fix errors in job unit page regarding the siblings property
+ urwid_ui: disable mouse input in main loops
+ urwid_ui: sort test plans by name
+ urwid_ui: palette and key fixes
+ urwid_ui: help and ID display for TestPlanBrowser
+ increment version to v1.5.0.dev0

+ Bump to v0.43.0
+ Bump to v0.43.0rc3
+ packaging.pxu: Remove iperf3, now provided via the checkbox provider
+ Bump to v0.43.0rc2
+ Added "timeout 5" to iperf3 test in canonical-certification-precheck
+ Add jumbo frames test to canonical-certification-precheck script.
+ Bump to v0.43.0rc1
+ units/storage-only.pxu: removed old disk/storage_device_* test from plan.
+ minor formatting nit-picks
+ modified jobs in test plans for newly defined sd-preinserted-server 
job. LP: 1832233
+ increment version to v0.43.0.dev0
+ added new regression pxu file and launcher
+ Fix mis-reported max link speeds in canonical-certification-precheck

+ Bump to v0.2.0
+ Bump to v0.2.0rc1
+ Add .bumpversion.cfg
+ Remove the debian folder
+ Add debian packaging files
+ Remove rogue hostapd.conf file
+ Tidy, remove ns year prefix
+ Update container tests
+ Remove provider sub directory
+ Delete GPU provider file
+ Move GPU units to main provider
+ Deleted unused launchers, snapcraft.yaml
+ pmr: don't trigger checkbox-snappy builds on merge
+ Delete erroneous memtest src
+ Update for pmr hooks
+ Binaries are 64bit only currently so alter filters
+ Make container tests safer for running in parallel with 
+ Update requirements to depend on checkbox-ng
+ Update the tests in container
+ remove year prefix
+ Update PMR hook
+ ipdt-gpu: only run jobs when cpu indentified as x86
+ ipdt: only run jobs when cpu indentified as x86

+ Bump to v0.42.0
+ Bump to v0.42.0rc2
+ parsers:udevadm: Improve Metadata Capture device node detection
+ udevadm_data: System with a capture metadata node not under the same 
usb config
+ Bump to v0.42.0rc1
+ parsers/netplan: support providing conf as string
+ parsers/udevadm: devmapper filter uses ID_FS_USAGE
+ lsusb: import script
+ test_udevadm: test for video capture metadata device nodes
+ udevadm_data: System with 4 video device nodes (two being capture 
metadata nodes)
+ parsers:udevadm: Ignore Metadata Capture device node
+ parsers/netplan: new parser for netplan configs
+ test_udevadm: test for symlink_uuid property
+ udevadm_data: DH170 with usb stick rebooted
+ udevadm_data: DH170 usb stick inserted no reboot
+ support partitions list in parser
+ expose symlink_uuid property
+ store symlink udev attributes
+ flake8 fixes for udev parser
+ xrandr: fix crashing when DISPLAY is not defined
+ increment version to v0.42.0.dev0
+ snap_utils:snapd: Add task_timeout and poll_interval init parameters

+ Bump to v0.49.0
+ Bump to v0.49.0rc5
+ network: Add a packaging meta data unit to depend on iperf3
+ disks: Add missing packaging meta-data unit to depend on python3-psutil
+ add after-suspend-auto-switch-card
+ firmware: Remove the no_ACPI_REV_interface test
+ Bump to v0.49.0rc4
+ submission: Expose the full set of DMI devices in raw-devices-dmi 
+ add pig-enabled test plan
+ modify edid-cycle test to use PIG
+ Bump to v0.49.0rc3
+ Work around LXD test failures caused by ownership issues
+ add after-suspend-usb-automated test plan
+ Attempt to get the gateway ip address for the interface and ping it 
before ARPing for available IPs to use in gateway_ping_test
+ alsa_test: add fallback loopback scenario
+ alsa_test: add list-devices subcmd
+ alsa_test: use proper AlsaError
+ alsa_test: fix warnings regarding no return statement
+ dont drain capture PCMs, drop them instead
+ remove unnecessary `more` invocation
+ units:wireless: New 80211ax jobs and nested parts for test plans
+ stress: Add iperf3 test plan
+ ethernet: Add iperf3 reverse test script and job
+ adjust graphic test for UMA platform
+ Fix deprecation warning about platform.linux_distribution() in 
secure_boot_mode test.
+ change palm-rejection test to user-itneract
+ suspend_advanced_auto: fallback to rtcwake if fwts s3 is not supported
+ stress: support customized value over env var
+ Add manual test to check decryption error when TPM data modified
+ wireless: remove superfluous wifi category
+ usb: remove superfluous usb2/usb3 categories
+ power-management: remove superfluous power category
+ bluetooth: remove superfluous bluetooth category
+ wwan: import from p-p-snappy
+ wireless: import from p-p-snappy
+ usb: import from p-p-snappy
+ tpm: import from p-p-snappy
+ suspend: import from p-p-snappy
+ stress: import from p-p-snappy
+ snappy: import from p-p-snappy
+ serial: import from p-p-snappy
+ self: import from p-p-snappy
+ security: import from p-p-snappy
+ rtc: import from p-p-snappy
+ power-management: import from p-p-snappy
+ watchdog: import from p-p-snappy
+ monitor: import from p-p-snappy
+ memory: import from p-p-snappy
+ mediacard: import from p-p-snappy
+ location: import from p-p-snappy
+ kernel-snap: import from p-p-snappy
+ info: import parts_meta_info_attachment
+ i2c: import from p-p-snappy
+ gpio: import from p-p-snappy
+ gadget: import from p-p-snappy
+ cpu: import from p-p-snappy
+ Bump to v0.49.0rc2
+ add auto switch card when the default graphic is discrete gpu
+ add auto switch card when the default graphic is discrete gpu
+ bluetooth: import from p-p-snappy
+ audio: import from p-p-snappy
+ 6lowpan: import from p-p-snappy
+ src: import alsa_test from p-p-snappy
+ Bump to v0.49.0rc1
+ data: import all p-p-snappy data files
+ bin: import all p-p-snappy scripts
+ misc: add test checking fans' reaction to cpu load
+ Fix redundant suspend test
+ modify graphics suffix
+ audio: linein, lineout require desktop or AIO
+ modify bluetooth file transfer test
+ wireless/netplan: wait on interface state routable
+ Add after suspend dvd read tests
+ Remove bluetooth/HID test
+ Remove some touchpad tests
+ Remove battery-info hotkey test
+ remove mediacard/sd related tests
+ modify monitor test plan
+ TEMP: disable requires on net_if_management
+ disk: remove storage_test script
+ disk: storage-device test script modifications
+ To correct the name for after suspend touchscreen test plan
+ usb: use lsusb from checkbox-support
+ bin:pm_test: use the proper fwts.log file to compute sleep/resume times
+ Fix thunderbolt cert blocker test cases
+ update sandybridge fingerprints
+ wifi_client_test_netplan: use gateway_ping_test
+ wifi_client_test_netplan: add log collection
+ wireless: enable net-driver-info for netplan tests
+ wireless: import netplan tests from p-p-snappy
+ pep8 cleanup
+ Replace exception raising with a sys.exit and helpful message
+ wireless: NetworkManager jobs requires update
+ hotkey_tests: add top-level docstring
+ hotkey_tests: add playpause button scenario
+ hotkey_tests: use two devs: one keyboard, one special events
+ Add after suspend thunderbolt/thunderbolt3 test cases
+ add flags to make the after suspend camera test can be run
+ monitor: add missing job for type-c tests
+ monitor: add type-c tests and after-suspend TP for discrete-gpu
+ Add missing suspend usb-c/c-to-ethernet-adapter-insert test
+ Add non-manifest-dependent mediacard test for server LP: #1832233
+ rotation_test: use xrandr bin to rotate the screen
+ monitor: add audio playback type-c to hdmi in test plan
+ monitor: modify jobs contents for more consistent with others
+ audio: add type-c to hdmi audio job
+ bin: let PRIME offloading on AMD GPUs work on any Ubuntu release
+ monitor: add display mode test steps to VGA/DVI/DP/HDMI
+ increment version to v0.49.0.dev0
+ network_device_info: handle missing udev attrs
+ network_info: consolidate jobs on newer info script
+ ethernet: add info_automated_server job
+ wireless: add detect job and manifest
+ network_device_info: refactor with argparse
+ bin:removable_storage_test: Ensure it is a media card reader only if 
this was explicitly requested
+ Added fingerprints for ROME and EPYC CPUs
+ Added CPUID script and jobs for server cert to attempt to identify CPU 
+ wifi_nmcli_test: remove trusty support as EOL
+ add automated EDID test
+ bin:gst_pipeline_test: Use pactl instead of pacmd
+ hotkey_tests: fix crashing on no match
+ hotkey_tests: add mute scenario
+ hotkey_tests: remove return values from contextmanager
+ hotkey_tests: add volume down key test
+ touchscreen: add after-suspend jobs lp:1811933
+ check other netplan dirs, and instead of E/N/I for non-netplan systems 
(e.g. desktop) query NetworkManager instead
+ Added job to attach netplan or E/N/I to test runs for server. lp: #1829538
+ add journalctl log collection
+ hotkey_tests: make volume monitoring more generic
+ hotkey_tests: add alt+f2 scenario
+ hotkey_tests: give terminal_hotkey test time to process the cmd
+ touchscreen: remove multitouch-dash cert test plan
+ networking: add ethernet/hotplug* to cert-blockers
+ monitor: test-plan ordering changes lp #1812160

+ Bump to v0.2.0
+ Bump to v0.2.0rc4
+ remove tiobench
+ remove FFTW tests
+ use precise xorg-dev package
+ Bump to v0.2.0rc3
+ remove memory bandwidth test
+ Bump to v0.2.0rc2
+ Add tests for the pmr hook
+ chmod +x on pmr-merge-hook
+ add necessary packaging meta-data
+ don't run all test combinations
+ Bump to v0.2.0rc1
+ Add .pmr-merge-hook
+ Add .bumpversion.cfg
+ add phoronix category and annotate the jobs
+ add README
+ add proposed list of PTS tests
+ install tests if not using preloading
+ remove tests-results if copied from preload path
+ add a report generating and attaching job
+ use pts wrapper when running jobs
+ add a wrapper that checks if the pts test succeeded
+ add basic hierarchy of units
+ add configuration for seamless batch runs
+ vendorize phoronix-test-suite


Sylvain Pineau

More information about the Checkbox-devel mailing list