Next Release of Checkbox (Week #40 Release Candidate #4) available in Testing PPA

Sat Sep 28 09:52:08 UTC 2019

Hello all!

The next release of Checkbox is available in the testing PPA:
https://code.launchpad.net/~checkbox-dev/+archive/testing

Components with new fixes/updates:
- plainbox-provider-checkbox 0.49.0rc4
- checkbox-support 0.42.0rc2
- checkbox-provider-phoronix 0.2.0rc3

Components in testing w/o new updates (See [1],[2] and [3] for their 
changelog):
- checkbox-ng 1.5.0rc3
- plainbox-provider-resource 0.41.0rc2
- plainbox-provider-tpm2 1.11.0rc2
- plainbox-provider-certification-client 0.38.0rc2
- plainbox-provider-certification-server 0.43.0rc2
- plainbox-provider-sru 1.14.0rc1
- plainbox-provider-ipdt 0.2.0rc1

Please help testing this version.

HIGHLIGHTS
----------

This upcoming version no longer supports the passwordless solution based on
pkexec and "insecure" policy files.
The new unified job runner only supports sudo and will help having a 
common way
to start checkbox using just the 'remote' systemd unit as a backend runner.
The immediate consequence is that automated and headless invocations now 
have
to install a sudoer file to keep the passworless feature. Checkbox itself
won't provide this mechanism by default for debian packages installations.
Please add such file prior testing using for example:

$ echo "$USER ALL=(ALL:ALL) NOPASSWD: ALL" | sudo tee 
/etc/sudoers.d/checkbox

Bug fixes
https://bugs.launchpad.net/checkbox-project/+bugs?field.milestone%3Alist=87675&field.milestone%3Alist=87681&field.milestone%3Alist=87682&field.milestone%3Alist=87677&field.milestone%3Alist=87678&field.milestone%3Alist=86955&field.milestone%3Alist=87680&field.milestone%3Alist=87676&orderby=status&start=0

Changelog

checkbox-provider-phoronix:
+ Bump to v0.2.0rc3
*+ remove memory bandwidth test*
+ Bump to v0.2.0rc2
+ Add tests for the pmr hook
+ chmod +x on pmr-merge-hook
+ add necessary packaging meta-data
+ don't run all test combinations
+ Bump to v0.2.0rc1
+ Add .pmr-merge-hook
+ Add .bumpversion.cfg
+ add phoronix category and annotate the jobs
+ add README
+ add proposed list of PTS tests
+ install tests if not using preloading
+ remove tests-results if copied from preload path
+ add a report generating and attaching job
+ use pts wrapper when running jobs
+ add a wrapper that checks if the pts test succeeded
+ add basic hierarchy of units
+ add configuration for seamless batch runs
+ vendorize phoronix-test-suite

checkbox-support:
+ Bump to v0.42.0rc2
*+ parsers:udevadm: Improve Metadata Capture device node detection*
+ udevadm_data: System with a capture metadata node not under the same 
usb config
+ Bump to v0.42.0rc1
+ parsers/netplan: support providing conf as string
+ parsers/udevadm: devmapper filter uses ID_FS_USAGE
+ lsusb: import lsusb.py script
+ test_udevadm: test for video capture metadata device nodes
+ udevadm_data: System with 4 video device nodes (two being capture 
metadata nodes)
+ parsers:udevadm: Ignore Metadata Capture device node
+ parsers/netplan: new parser for netplan configs
+ test_udevadm: test for symlink_uuid property
+ udevadm_data: DH170 with usb stick rebooted
+ udevadm_data: DH170 usb stick inserted no reboot
+ test_udevadm.py: support partitions list in parser
+ udevadmp.py: expose symlink_uuid property
+ udevadm.py: store symlink udev attributes
+ udevadm.py: flake8 fixes for udev parser
+ xrandr: fix crashing when DISPLAY is not defined
+ increment version to v0.42.0.dev0
+ snap_utils:snapd: Add task_timeout and poll_interval init parameters

plainbox-provider-checkbox:
+ Bump to v0.49.0rc4
*+ submission: Expose the full set of DMI devices in raw-devices-dmi 
attachment*
+ add pig-enabled test plan
+ modify edid-cycle test to use PIG
+ Bump to v0.49.0rc3
+ Work around LXD test failures caused by ownership issues
+ add after-suspend-usb-automated test plan
+ Attempt to get the gateway ip address for the interface and ping it 
before ARPing for available IPs to use in gateway_ping_test
+ alsa_test: add fallback loopback scenario
+ alsa_test: add list-devices subcmd
+ alsa_test: use proper AlsaError
+ alsa_test: fix warnings regarding no return statement
+ dont drain capture PCMs, drop them instead
+ remove unnecessary `more` invocation
+ units:wireless: New 80211ax jobs and nested parts for test plans
+ stress: Add iperf3 test plan
+ ethernet: Add iperf3 reverse test script and job
+ adjust graphic test for UMA platform
+ Fix deprecation warning about platform.linux_distribution() in 
secure_boot_mode test.
+ change palm-rejection test to user-itneract
+ suspend_advanced_auto: fallback to rtcwake if fwts s3 is not supported
+ stress: support customized value over env var
+ Add manual test to check decryption error when TPM data modified
+ wireless: remove superfluous wifi category
+ usb: remove superfluous usb2/usb3 categories
+ power-management: remove superfluous power category
+ bluetooth: remove superfluous bluetooth category
+ wwan: import from p-p-snappy
+ wireless: import from p-p-snappy
+ usb: import from p-p-snappy
+ tpm: import from p-p-snappy
+ suspend: import from p-p-snappy
+ stress: import from p-p-snappy
+ snappy: import from p-p-snappy
+ serial: import from p-p-snappy
+ self: import from p-p-snappy
+ security: import from p-p-snappy
+ rtc: import from p-p-snappy
+ power-management: import from p-p-snappy
+ watchdog: import from p-p-snappy
+ monitor: import from p-p-snappy
+ memory: import from p-p-snappy
+ mediacard: import from p-p-snappy
+ location: import from p-p-snappy
+ kernel-snap: import from p-p-snappy
+ info: import parts_meta_info_attachment
+ i2c: import from p-p-snappy
+ gpio: import from p-p-snappy
+ gadget: import from p-p-snappy
+ cpu: import from p-p-snappy
+ Bump to v0.49.0rc2
+ add auto switch card when the default graphic is discrete gpu
+ add auto switch card when the default graphic is discrete gpu
+ bluetooth: import from p-p-snappy
+ audio: import from p-p-snappy
+ 6lowpan: import from p-p-snappy
+ src: import alsa_test from p-p-snappy
+ Bump to v0.49.0rc1
+ data: import all p-p-snappy data files
+ bin: import all p-p-snappy scripts
+ misc: add test checking fans' reaction to cpu load
+ Fix redundant suspend test
+ modify graphics suffix
+ audio: linein, lineout require desktop or AIO
+ modify bluetooth file transfer test
+ wireless/netplan: wait on interface state routable
+ Add after suspend dvd read tests
+ Remove bluetooth/HID test
+ Remove some touchpad tests
+ Remove battery-info hotkey test
+ remove mediacard/sd related tests
+ modify monitor test plan
+ TEMP: disable requires on net_if_management
+ disk: remove storage_test script
+ disk: storage-device test script modifications
+ To correct the name for after suspend touchscreen test plan
+ usb: use lsusb from checkbox-support
+ bin:pm_test: use the proper fwts.log file to compute sleep/resume times
+ Fix thunderbolt cert blocker test cases
+ cpuid.py: update sandybridge fingerprints
+ wifi_client_test_netplan: use gateway_ping_test
+ wifi_client_test_netplan: add log collection
+ wireless: enable net-driver-info for netplan tests
+ wireless: import netplan tests from p-p-snappy
+ pep8 cleanup
+ Replace exception raising with a sys.exit and helpful message
+ wireless: NetworkManager jobs requires update
+ hotkey_tests: add top-level docstring
+ hotkey_tests: add playpause button scenario
+ hotkey_tests: use two devs: one keyboard, one special events
+ Add after suspend thunderbolt/thunderbolt3 test cases
+ add flags to make the after suspend camera test can be run
+ monitor: add missing job for type-c tests
+ monitor: add type-c tests and after-suspend TP for discrete-gpu
+ Add missing suspend usb-c/c-to-ethernet-adapter-insert test
+ Add non-manifest-dependent mediacard test for server LP: #1832233
+ rotation_test: use xrandr bin to rotate the screen
+ monitor: add audio playback type-c to hdmi in test plan
+ monitor: modify jobs contents for more consistent with others
+ audio: add type-c to hdmi audio job
+ bin: let PRIME offloading on AMD GPUs work on any Ubuntu release
+ monitor: add display mode test steps to VGA/DVI/DP/HDMI
+ increment version to v0.49.0.dev0
+ network_device_info: handle missing udev attrs
+ network_info: consolidate jobs on newer info script
+ ethernet: add info_automated_server job
+ wireless: add detect job and manifest
+ network_device_info: refactor with argparse
+ bin:removable_storage_test: Ensure it is a media card reader only if 
this was explicitly requested
+ Added fingerprints for ROME and EPYC CPUs
+ Added CPUID script and jobs for server cert to attempt to identify CPU 
families
+ wifi_nmcli_test: remove trusty support as EOL
+ add automated EDID test
+ bin:gst_pipeline_test: Use pactl instead of pacmd
+ hotkey_tests: fix crashing on no match
+ hotkey_tests: add mute scenario
+ hotkey_tests: remove return values from contextmanager
+ hotkey_tests: add volume down key test
+ touchscreen: add after-suspend jobs lp:1811933
+ check other netplan dirs, and instead of E/N/I for non-netplan systems 
(e.g. desktop) query NetworkManager instead
+ Added job to attach netplan or E/N/I to test runs for server. lp: #1829538
+ wifi_nmcli_test.py: add journalctl log collection
+ hotkey_tests: make volume monitoring more generic
+ hotkey_tests: add alt+f2 scenario
+ hotkey_tests: give terminal_hotkey test time to process the cmd
+ touchscreen: remove multitouch-dash cert test plan
+ networking: add ethernet/hotplug* to cert-blockers
+ monitor: test-plan ordering changes lp #1812160

Cheers,

Sylvain Pineau

[1] https://lists.ubuntu.com/archives/checkbox-devel/2019-August/000452.html
[2] 
https://lists.ubuntu.com/archives/checkbox-devel/2019-September/000454.html
[3] 
https://lists.ubuntu.com/archives/checkbox-devel/2019-September/000455.html