Next Release of Checkbox (Week #38 Release Candidate) available in Testing PPA

[Sylvain Pineau]

Hello all!

The next release of Checkbox is available in the testing PPA:
https://code.launchpad.net/~checkbox-dev/+archive/testing

- checkbox-ng 1.10.0rc1
- checkbox-provider-gpgpu 0.2.0rc1
- checkbox-support 0.48.0rc1
- plainbox-provider-resource 0.46.0rc1
- plainbox-provider-checkbox 0.55.0rc1
- plainbox-provider-certification-client 0.43.0rc1
- plainbox-provider-certification-server 0.51.0rc1
- plainbox-provider-sru 1.20.0rc1

Please help testing this version.

*Bug fixes*
https://bit.ly/3mpsrHb

*HIGHLIGHTS*

- Checkbox sessions are now stored in /var/tmp/... no matter the 
installation method (classic, snap)
- Lots of shellcheck/flake8 fixes, all scripts now have an extension.
- First release of the GPGPU provider

*Changelog*

checkbox-ng:
+ Bump to v1.10.0rc1
+ add session description to the html exporter
+ add without-session-desc option to Jinja 2 exporter
+ fix Jinja2 exporter initialization
+ poperly forward exporter options to SA
+ infer exporter unit from exporter's name in configs
+ use better defaults for session description
+ make establishing session title more sane
+ resource.py: Fix the condition to favor the split_and_evaluate method
+ session/remote-assistant: auto skip on can't start
+ session/assistant: fix not calling method
+ session/storage: update tests
+ session/storage: session storage simplication
+ fix checkbox.conf not being read in some cases
+ exporter/xlsx: lazy load the xlsxwriter modules
+ exporter/tar: don't include xlsx
+ launcher/stages: no xlsx in submission_files
+ launcher:stages: Add support for the $C3_URL env var
+ provider_manager: Prefer universal_newlines=True, encoding is 3.6+
+ docs: New banner/title png
+ provider_manager: Create a subparser to only run a subset of tests
+ provider_manager: Add Flake8 and inline command ShellCheck test suites
+ provider_manager: shellcheck on bin dir scripts
+ Remove trusty special case from doc build test
+ Make the sphinx build test specify a build dir based on the release name
+ fix crashing when sideloading some providers
+ execution: forward PYTHONPATH too to sudo-ed jobs
+ fix manifest not yielding records if used on root slaves
+ master: fix crash when TP list is empty
+ increment version to v1.10.0.dev0
+ requirement: remove unused dependencies
+ slave: guess the normal_user if not supplied
+ setup.py: remove pycrypto
+ remote_assistant: expose _some_ pubkey for RAPI backwards compat.
+ sudo_broker: remove SudoProvider
+ slave: remove references to sudo and kill sudo_input interaction
+ master: remove password handling from master
+ master: require a passwordless slave
+ slave: assume passwordless sudo and mark API for removal
+ slave: require running as root and passless sudo
+ sudo_broker: remove encryption of sudo pass

checkbox-provider-gpgpu:
+ First release

plainbox-provider-resource:
+ Bump to v0.46.0rc1
+ resource:package: Restore the package version in the resource ouput
+ resource:executable: Remove find command errors msg from job output
+ Add low power idle check nodes if exist
+ Just list the package resource name property
+ Drop duplicates from the executable resource output
+ Remove the old staging/udev job prototype
+ package: remove modalias attribute from records
+ packaging: Remove the dmidecode dependency on riscv64 architecture
+ Fix shellcheck errors in bin/*.sh
+ Fix Flake8 errors in bin/*.py
+ Fix inline shellcheck errors in jobs/resources.pxu
+ requirements: Add flake8 to deb-base
+ Revert "requirements: [TEMP] run only manage.py test -u"
+ requirements: [TEMP] run only manage.py test -u
+ Remove all i18n content
+ bin:xinput_resource -> xinput_resource.py
+ bin:uname_resource -> uname_resource.py
+ bin:udev_resource -> udev_resource.py
+ bin:udev2resource -> udev2resource.py
+ bin:optical_resource -> optical_resource.py
+ bin:module_resource -> module_resource.py
+ bin:mobilebroadband_resource -> mobilebroadband_resource.sh
+ bin:meminfo_resource -> meminfo_resource.py
+ bin:lsb_resource -> lsb_resource.py
+ bin:graphics_card_resource -> graphics_card_resource.py
+ bin:efi_resource -> efi_resource.py
+ bin:dpkg_resource -> dpkg_resource.py
+ bin:dmi_resource -> dmi_resource.py
+ bin:display_resource -> display_resource.py
+ bin:cpuinfo_resource -> cpuinfo_resource.py
+ bin:cdimage_resource -> cdimage_resource.py
+ bin:block_device_resource -> block_device_resource.py
+ requirements: install shellcheck for testing
+ increment version to v0.46.0.dev0

plainbox-provider-certification-client:
+ Bump to v0.43.0rc1
+ Adopt the core stress tests (cold/warm boot and S3)
+ client-cert-odm*: remove laptop focused tests
+ Certification test plans for ODM programme (18)
+ Create certification test plans for ODM programme
+ Reword the descriptions for client cert test plans
+ increment version to v0.43.0.dev0

checkbox-support:
+ Bump to v0.48.0rc1
+ Fix bug if device file does not exist in disk_support.py
+ scripts/audio_settings.py: classic snap support
+ snap_utils/system: fix is_classic_snap
+ Moved disk_support.py up one level
+ Added new disk_support.py file with shared disk functions.
+ increment version to v0.48.0.dev0

plainbox-provider-checkbox:
+ Bump to v0.55.0rc1
+ units: usb: Add note to make sure USB drive has a partition
+ fde_tests.py: update FDE detect for UC20
+ bin/network_device_info.py - added argument to list disconnected 
devices, and exit with a failure code if desired. Modified server 
ethernet info job to use this. lp: #1895048
+ units:mediacard: Fix sdhc-storage summary
+ units: Replace legacy "System Testing" by Checkbox
+ units:suspend: typo (network_wait.py -> network_wait.sh)
+ Add low power idle check to power-management category
+ units:bluetooth: Remove the bluez package.version>= '5.37' requirements
+ fixed minor issues in maas_user_check job, added job to miscellanea 
regression test plan as well.
+ Added new job to check BMC for the maas user account. lp: #1892786
+ bin/maas-version-check: removed the 2.0 version check that is causing 
issues, since we no longer need it anyway. lp: #1892525
+ disk/hdd-parking: add the term which is commonly using by OEM
+ Add job for validating sof firmware signature
+ Exchanged the sequence of graphics log parsing
+ camera: fix typo in camera-automated test plan
+ add leeway for frequency scaling test
+ Update tb3 blocker status
+ pulse_active_port_change.py: classic snap support
+ audio: rename pulse-active-port-change.py
+ units/*: simplify requires fields
+ wireless: NM auto jobs require root for ping
+ Add unit/server-missellaneous-regression to remove commonly failing 
jobs from regression suite. lp: #1889775
+ units/*: remove use of zenity dialogs
+ units:monitor: Remove the two chameleon jobs and scripts
+ bin:disk_read_performance_test.sh: Fix SC1001 for older releases
+ requirements: Run all subtests via manage.py test
+ bin:*.sh: Fix all shellcheck errors
+ net_driver_info: handle empty argument
+ UC buildstamp location
+ fde_tests: fix bad search with line end
+ wifi: fixes for xenial, fix ping args
+ wifi_nmcli_test.py: add ping test, disable ipv6
+ wifi_client_test_netplan: more debug, gateway fix
+ units/*: modify all calls of geteway_ping_test
+ gateway_ping_test.py: fix flake8 warnings
+ bin: rename gateway_ping_test script
+ units:i2c: Fix shellcheck error (detected on xenial/bionic only)
+ requirements: Enable manage.py -i (shellcheck on job inline commands)
+ units: Fix all jobs commands shellcheck errors
+ bin:spindown.sh: Deleted
+ bin:*.py: Fix Flake8 W503 (Xenial version, 2.5.4)
+ requirements: Add manage.py test -f to run Flake8 on Python scripts
+ bin:*.py: Fix all Flake8 errors
+ bin:testlib.py: Deleted
+ Add output support for Tiger Lake CPUID
+ make xorg-version depend on xorg-process
+ Revisions to stress_ng_test script in response to review feedback
+ requirements: [TEMP] run only manage.py test -u
+ Remove all i18n content
+ Remove mention of plainbox in the README
+ p-p-r:bin:udev_resource -> udev_resource.py
+ p-p-r:bin:meminfo_resource -> meminfo_resource.py
+ p-p-r:bin:cpuinfo_resource -> cpuinfo_resource.py
+ Update I18N po/ files
+ bin:xrandr_cycle -> xrandr_cycle.py
+ bin:xen_test: Deleted
+ bin:window_test -> window_test.py
+ bin:wifi_time2reconnect -> wifi_time2reconnect.py
+ bin:wifi_client_test -> wifi_client_test.py
+ bin:volume_test -> volume_test.py
+ bin:virtualization -> virtualization.py
+ bin:udisks2_monitor: Deleted
+ bin:touchpad_test -> touchpad_test.py
+ bin:touchpad_driver_info -> touchpad_driver_info.py
+ bin:tomcat_test: Deleted
+ bin:test_bt_keyboard -> test_bt_keyboard.py
+ bin:stress_ng_test -> stress_ng_test.py
+ bin:spindown -> spindown.sh
+ bin:sources_test -> sources_test.sh
+ bin:sleep_time_check -> sleep_time_check.py
+ bin:sleep_test_log_check -> sleep_test_log_check.py
+ bin:sleep_test -> sleep_test.py
+ bin:samba_test: Deleted
+ bin:rotation_test_using_dbus: Deleted
+ bin:rotation_test -> rotation_test.py
+ bin:resolution_test -> resolution_test.py
+ bin:rendercheck_test: Deleted
+ bin:removable_storage_watcher -> removable_storage_watcher.py
+ bin:removable_storage_test -> removable_storage_test.py
+ bin:pulse-active-port-change -> pulse-active-port-change.py
+ bin:pts_run: Deleted
+ bin:process_wait -> process_wait.py
+ bin:pm_log_check -> pm_log_check.py
+ bin:pm_test.py: Preserve the log_filename (w/o the .py script extension)
+ bin:pm_test -> pm_test.py
+ bin:optical_write_test -> optical_write_test.sh
+ bin:optical_read_test -> optical_read_test.py
+ bin:optical_detect: Deleted
+ bin:nvidia_nvlink_check -> nvidia_nvlink_check.sh
+ bin:network_wait -> network_wait.sh
+ bin:network_restart -> network_restart.py
+ bin:network_reconnect_resume_test -> network_reconnect_resume_test.py
+ bin:network_printer_test -> network_printer_test.sh
+ bin:network_predictable_names -> network_predictable_names.sh
+ bin:network_ntp_test -> network_ntp_test.py
+ bin:network_configs -> network_configs.sh
+ bin:network_check -> network_check.py
+ bin:network -> network.py
+ bin:net_driver_info -> net_driver_info.py
+ bin:mm-test: Deleted
+ bin:memory_test -> memory_test.py
+ bin:memory_info: Deleted
+ bin:memory_compare -> memory_compare.py
+ bin:max_diskspace_used: Deleted
+ bin:manage_compiz_plugin -> manage_compiz_plugin.py
+ bin:maas-version-check -> maas-version-check.sh
+ disk/encryption: support new LUKS in FDE test
+ bin:lsmod_info -> lsmod_info.py
+ bin:lock_screen_watcher -> lock_screen_watcher.py
+ bin:light_sensor_test -> light_sensor_test.sh
+ bin:led_hdd_test -> led_hdd_test.sh
+ bin:lamp_test: Deleted
+ bin:key_test -> key_test.py
+ bin:keyboard_test -> keyboard_test.py
+ bin:kernel_taint_test -> kernel_taint_test.py
+ bin:hdd_parking -> hdd_parking.py
+ bin:gst_pipeline_test -> gst_pipeline_test.py
+ bin:graphics_stress_test -> graphics_stress_test.py
+ bin:graphics_modes_info -> graphics_modes_info.py
+ bin:graphics_env -> graphics_env.sh
+ bin:graphics_driver -> graphics_driver.py
+ bin:graphic_memory_info -> graphic_memory_info.py
+ bin:gputest_benchmark -> gputest_benchmark.py
+ bin:gpu_test -> gpu_test.py
+ bin:get_make_and_model -> get_make_and_model.py
+ bin:gcov_tarball -> gcov_tarball.sh
+ bin:fstrim_test -> fstrim_test.py
+ bin:fresh_rate_info -> fresh_rate_info.py
+ bin:frequency_governors_test -> frequency_governors_test.py
+ bin:floppy_test: Deleted
+ bin:efi-pxeboot -> efi-pxeboot.py
+ bin:dns_server_test: Deleted
+ bin:dmitest -> dmitest.py
+ bin:disk_stats_test -> disk_stats_test.sh
+ bin:disk_smart -> disk_smart.py
+ bin:disk_read_performance_test -> disk_read_performance_test.sh
+ bin:disk_info -> disk_info.py
+ bin:disk_cpu_load -> disk_cpu_load.sh
+ bin:cycle_vts -> cycle_vts.sh
+ bin:create_connection -> create_connection.py
+ bin:cpu_topology -> cpu_topology.py
+ bin:cpu_offlining -> cpu_offlining.py
+ bin:connect_wireless -> connect_wireless.sh
+ bin:color_depth_info -> color_depth_info.py
+ bin:cking_suite: Deleted
+ bin:check-prerelease -> check-prerelease.py
+ bin:check_is_laptop -> check_is_laptop.sh
+ bin:camera_test -> camera_test.py
+ bin:bt_connect -> bt_connect.py
+ bin:broadband_info -> broadband_info.sh
+ bin:brightness_test -> brightness_test.py
+ bin:boot_mode_test -> boot_mode_test.py
+ bin:bmc_info -> bmc_info.py
+ bin:bluetooth_transfer_stress -> bluetooth_transfer_stress.sh
+ bin:bluetooth_test -> bluetooth_test.py
+ bin:bluetooth_scan: Deleted
+ bin:battery_test -> battery_test.py
+ bin:audio_test -> audio_test.py
+ bin:audio_settings -> audio_settings.py
+ bin:audio_driver_info -> audio_driver_info.py
+ bin:audio_bluetooth_loopback_test -> audio_bluetooth_loopback_test.sh
+ bin:ansi_parser -> ansi_parser.py
+ bin:alsa_record_playback -> alsa_record_playback.sh
+ bin:alsa_pcm_info -> alsa_pcm_info.py
+ bin:accelerometer_test -> accelerometer_test.py
+ units/image: tests for ODM IoT Certification
+ Add a Wi-Fi connect/disconnect stress job
+ 1. create led-cert-blockers and after-suspend-led-cert-blockers test 
plan. 2. fixed bug in camera/led{name} job. 3. modify the name of the 
led-full test plan.
+ requirements: install shellcheck for testing
+ fixed the typo to unmount disk
+ gateway_ping_test: fix crash when using networkctl
+ Modify stress_ng_test to use new disk library
+ i2c/manifest: more specific naming
+ Modify import & call to Disk library in fstrim_test script, per 
jocave's comment
+ Fix broken fstrim_test module definition
+ Add a test case to check Energy Efficient Ethernet (EEE) status.
+ Removed new fstrim test from inappropriate .pxu files
+ Add test case for Fn key and Fn key led
+ pep8 fixes
+ bin/ipmi_test.py: don't bother trying to load ipmi_powernv on 
non-power hardware lp: #1883761
+ Create the packaging meta data unit to add the dependency.
+ Include miscellanea/debsums in misc-client-cert-automated.
+ Add the test case for debsums.
+ PEP8 compliance fixes
+ bin/kernel_taint_test: Improve output to identify proprietary and 
non-intree drivers. lp: #1876977
+ Added new fstrim test
+ Add thunderbolt3 storage-preinserted test
+ ethernet: static config job networkd/nm support
+ increment version to v0.55.0.dev0
+ Add dock test case for check all IO port at same time
+ Add dock test case for check all IO port at same time
+ modify the description of Line-Out for desktop with internal speaker
+ Remove OBEX browse and get from suspend-reference-cert-blockers
+ Remove GPGPU units as they've been moved to their own provider. LP: 
#1879965

plainbox-provider-sru:
+ Bump to v1.20.0rc1
+ Add thunderbolt test
+ increment version to v1.20.0.dev0

plainbox-provider-certification-server:
+ Bump to v0.51.0rc1
+ units/server-regression.pxu - replace miscellanea test plan with 
slimmer regression version. lp: #1889775
+ Adding nvdimm tests to the memory-only testplan.  This will allow the 
retest of memory for Apache Pass testing
+ Added a change to force the test to run without the option to select tests
+ Adding a nvdimm-test only to test Apache Pass.
+ Update regression test plan to add in everything but network
+ Update subset launchers for network and storage to allow deselecting 
devices for test. lp: #1884106
+ Add new fstrim test to multiple .pxu files
+ increment version to v0.51.0.dev0
+ Remove GPGPU units as they've been moved to their own provider. LP: 
#1879965

Cheers,

Sylvain Pineau