Next Release of Checkbox (Week #04 Release Candidate) available in Testing PPA RC2

[Sylvain Pineau]

Hi,

New RC (#2) are available in the testing ppa for the following packages 
since RC1 was not promoted to stable before the break.
The new packages are mainly published to catch up with recent fixes that 
landed in master since last month.
Other providers that didn't get updated are still in RC1.

- checkbox-ng 1.3.0rc2
- checkbox-support 0.40.0rc2
- plainbox-provider-checkbox 0.47.0rc2
- plainbox-provider-certification-server 0.41.0rc2

Please help testing this version.

*Changelog*

checkbox-ng:
+ subcommands: Add a --nofake option to tp-export
+ docs: document side-loading providers
+ assistant: override providers one by one with side-loading
+ remote: Don't call wait_for_interaction_prompt for manual jobs
+ remote: Display the verification steps for interactive jobs
+ remote: fix interactive jobs
+ remote: fix handling of input pipe when restarting SA
+ remote: remove stale TODO comment
+ remote: sprinkle logging calls all over the master
+ remote: fix crashing when master is disconnected very early on
+ remote: tidy up references to session_change_lock
+ remote: when connecting while bootstrapping restart the session
+ remote: ensure session lock is released when restarting

checkbox-support:
+ scripts:run_watcher: Add support for mediacard events

plainbox-provider-checkbox:
+ units:touchpad: Remove singletouch-automated and multitouch-automated
+ units:touchpad: Remove obsolete touchpad/singletouch-corner
+ add /etc/media-info as an alternative to buildstamp
+ bluetooth: add bluez controller detect job
+ units:graphics: update xorg-process detection method
+ units:mediacard: leverage the checkbox-support utilities to run 
non-udisks2 tests on UC
+ bluetooth: create bluetooth detect job
+ Increase timeout in memory_stress_ng from 150% to 200% of expected run 
time
+ oobm: initial tests for Intel AMT testing
+ ethernet: add after-suspend manual plan (empty)
+ skip system_info_json on dragonboard because it crashes

plainbox-provider-certification-server:
+ Added launchers to make retesting memory and cpu stress easier. LP: 
#1811704

Thx,

Sylvain

-------- Forwarded Message --------
Subject: 	Next Release of Checkbox (Week #49 Release Candidate) 
available in Testing PPA
Date: 	Thu, 6 Dec 2018 14:10:10 +0100
From: 	Sylvain Pineau <sylvain.pineau at canonical.com>
To: 	ce-certification-qa <ce-certification-qa at lists.canonical.com>, 
checkbox-devel at lists.ubuntu.com



Hello all!

The next release of Checkbox is available in the testing PPA:
https://code.launchpad.net/~checkbox-dev/+archive/testing

- checkbox-ng 1.3.0rc1
- checkbox-support 0.40.0rc1
- plainbox-provider-resource 0.39.0rc1
- plainbox-provider-checkbox 0.47.0rc1
- plainbox-provider-sru 1.13.0rc1
- plainbox-provider-tpm2 1.9.0rc1
- plainbox-provider-certification-client 0.36.0rc1
- plainbox-provider-certification-server 0.41.0rc1

Please help testing this version.

*HIGHLIGHTS*

- New checkbox-cli merge commands
   See https://yphus.github.io/test/ubuntu/2018/10/31/merge-commands.html
- Hardware summary available in HTML reports
- Remote API V4
- Show less advice when running provider validation
- New BT LE Beacon test
- Re-introduce FWTS in server test plans

*Bug fixes*
https://goo.gl/pzrtGx

*Changelog*

checkbox-ng:
+ remote: properly restart session when launcher are used
+ remote: add API for handling input from master
+ remote: use daemonic controller
+ add daemonic execution controller
+ add `normal-user` variable to launchers/configs
+ ctrl: accept FO for subprocess' stdin as ctrl's param
+ remote: print the outcome for non-bootstrap jobs
+ launchers:merge*: Make -o a required argument
+ exporters:html: Update unit tests
+ remote: print the outcome for non-bootstrap jobs
+ exporters: Add a new system info section to html reports
+ checkbox-cli: Explicit actions when failing to merge submissions
+ remote: fix unicode errors
+ remote: bump remote api version
+ remote: remove unnecessary return
+ remote: allow monitor_job in TestsSelected state
+ update unit tests for provider_manager
+ unit:testplan: kill advice in testplan
+ unit:file: kill advice for file unit
+ unit:template: kill unit type advice
+ unit:job: kill unnecessary advice
+ remote: track progress through interrupted session
+ remote_assistant: populate done sublist in progress
+ session:assistant: add get_dynamic_done_list()
+ exporters:data:multi-page.html: Remove the $(...).toolbar() call
+ remote: filter tps when defined in launcher
+ plainbox:xparsers: fix doctest failing on py37
+ Increase timeout for remote connection to 10min.
+ exporters:html: Disable autoescape for comments to preserve HTML tags
+ exporter:xlsx: Add category names to tp-export reports
+ checkbox-cli: Refactor merge-reports/submissions code
+ plainbox:test-data: Update HTML reference files
+ checkbox-cli: New merge-submissions command
+ exporters:html: Upgrade existing html exporter with chart.js 2.7.3
+ checkbox-cli: New merge-reports command
+ checkbox-cli:run: Add both title and description options to the command
+ exporters:jinja2: New dump_from_session_manager_list method
+ exporters:html-multi-page: New exporter to record multiple sessions in 
a single HTML
+ exporters:html-multi-page: New Chart.js (2.7.3)
+ exporters:json: Add category_id, attachment-results and category_map
+ session:state: Do not consider resources/attachments for global outcome
+ remote: track progress through interrupted session
+ remote_assistant: populate done sublist in progress
+ session:assistant: add get_dynamic_done_list()
+ exporters:data:multi-page.html: Remove the $(...).toolbar() call
+ remote: filter tps when defined in launcher
+ plainbox:xparsers: fix doctest failing on py37
+ Increase timeout for remote connection to 10min.
+ exporters:html: Disable autoescape for comments to preserve HTML tags
+ exporter:xlsx: Add category names to tp-export reports
+ checkbox-cli: Refactor merge-reports/submissions code
+ plainbox:test-data: Update HTML reference files
+ checkbox-cli: New merge-submissions command
+ exporters:html: Upgrade existing html exporter with chart.js 2.7.3
+ checkbox-cli: New merge-reports command
+ checkbox-cli:run: Add both title and description options to the command
+ exporters:jinja2: New dump_from_session_manager_list method
+ exporters:html-multi-page: New exporter to record multiple sessions in 
a single HTML
+ exporters:html-multi-page: New Chart.js (2.7.3)
+ exporters:json: Add category_id, attachment-results and category_map
+ session:state: Do not consider resources/attachments for global outcome
+ exporters: add resource-results to the json template
+ checkbox-cli: remove --partial option from list-bootstrapped
+ checkbox-cli: default formatting option when using list-bootstrapped
+ ctrl: Don't process templates when there's no resources (i.e no I/O logs)
+ subcommands: Exit properly the run command after a CTRL-C
+ Check to see that the UndesiredJobReadinessInhibitor exists before 
removing it in SessionState

checkbox-support:
+ parsers:udev: Set pid/vid in base16 for canbus and v4l devices (for 
consistency)
+ parsers:udevadm: Report NVDIMM devices as DISK
+ parsers:udevadm: Set bus to mediacard for the PARTITION category (mmc)
+ add hostfs as a proper root partitions in udevadm parser
+ Added bits that are requested for the server suite. Adds new --server 
option to run these.
   Adds --quiet option to suppress lengthy console output. LP: #1801960
+ Fix pattern match for /writable mountpoint to work with core18 also
+ network: removing network script in favour of p-p-c
+ scripts:eddystone_scanner: power on the adapter via bluetoothctl
+ scripts:eddystone_scanner: Use logging
+ scripts:eddystone_scanner: Remove asyncio async/await to run on Python 3.4
+ parsers:udevadm: Add interface (hciX) info to BLUETOOTH devices
+ parsers:udevadm: Fix indentation
+ scripts:eddystone_scanner: New beacon scanner
+ Vendorize https://github.com/frawau/aioblescan
+ parsers:udevadm: Identify CAPTURE devices using the V4L subsystem only
+ Add snap_connect.py scripta and expose
+ scripts:fwts_test: Take into account SKIPPED and WARNING exit summmary
+ scripts:fwts_test: Fix summary parsing to only fail after the last check
+ scripts:fwts_test: set suspend/resume timings to N/A when not found in 
logs
+ scripts:fwts_test: open logs using "error='ignore'"
+ scripts:fwts_test: Get suspend/resume timings from fwts, not syslog
+ scripts:fwts_test: Flush the progress messages to follow them on stdout
+ scripts:fwts_test:Mask the Zenity GTK (harmless) warning
+ parsers:tests: Updates to account for the new HIDRAW category
+ parsers:tests: Drop redondant tests
+ parsers:tests:new udevadm sample to check microchip USB2SPI is 
reported as HIDRAW
+ parsers:udevadm: Add HIDRAW category
+ Full rewrite of run_watcher script to monitor systemd journal instead 
of syslog

plainbox-provider-resource:
+ bin:udev_resource: Add a --short option to the list command
+ bin:snapd_resource: Handle systems with no plugs/slots

plainbox-provider-checkbox:
+ ethernet: allow check static to be used on classic
+ Add a2dp to list of BT blockers LP:#1803645
+ bin/inxi_snapshot: Drop the check for tty
+ Make container tests safer for running in parallel with 
plainbox-provider-checkbox
+ bin:system_info: print error msg coming from subprocesses
+ Modified network script to call iperf3 in multiple threads; necessary 
for testing high-speed (40Gbps and faster) NICs.
+ bin:system_info: Forcing usb data to be collected via sysfs
+ socketcan: add manual test plans for nesting
+ disk: add manual test plans for nesting
+ add missing 'also-after-suspend' flag for job usb3/insert
+ units:miscellaneous: Add kernel_cmdline_attachment to submission-resources
+ units:miscellaneous: Add system_info_json to submission-resources
+ units:submission: Add a new job calling system_info.py
+ bin:system_info.py: New tool to gather system info from inxi and udev 
parser
+ bin:inxi_snapshot: Add inxi system info command line tool
+ added missing include: to testplan
+ Added category_id and summary fields to new jobs
+ Add klog and olog tests to miscellanea for server. Added server 
miscellanea test plan to start nesting in the server suite. LP: #1802152
+ units:stress: Remove stress-cert-automated in stress-cert-manual
+ Added server focused fwts jobs to firmware/jobs.pxu. Added new 
test-plan.pxu with a test plan to nest in server test plans. LP: #1801963
+ Removing purley test
+ bin:dkms_info: Skip malformed modaliases
+ networking: initial ipv6 jobs and devices test plans
+ units: Add EddyStone URL scan tests (before and after S3)
+ network_info: fix scraping ip for ipv6 address
+ network: use local script as most up-to-date
+ check-static: import script to go with test
+ ethernet: import test plans from p-p-s
+ units:misc: Add BLUETOOTH to device check test
+ units:bluetooth: power on is now done by the eddystone_scanner script
+ units:bluetooth: Fail the eddystone scan test if one of the commands 
failed
+ units:bluetooth: Bring down the device after the eddystone scan test
+ units:bluetooth: Ensure the device is up and running before scanning 
for beacon ads
+ units:bluetooth: Add new BT LE beacon test using EddyStone URL 
advertisements
+ units:bluetooth:test-plan.pxu: Fix indentation
+ units:camera: Remove the unnecessary dep on libcheese
+ renamed server info testplan
+ Added a server info-attachment testplan too so that I can start taking 
advantage of nested testplans.
+ Added a second lstopo job to create a small png system map. Modified 
original job.
+ units:cpu: Only run cpu/cstates on Intel
+ added lstopo_attachment to the info-attachment-cert-automated testplan
+ Add new attachment job to run lstopo command
+ Fixed issue that breaks ipmi_test on some systems that require a 
specific channel to be used. LP: #1794926
+ units:misc: CAPTURE devices added to miscellanea/device_check
+ units:camera: Use template jobs to drive V4L devices
+ bin:camera_test: Use GStreamer camerabin instead of libcheese
+ Add category to template job `ethernet/hotplug-*-end-cycle`

plainbox-provider-sru:
+ units: Remove OBEX tests to prefer EddyStone URL scan tests (before 
and after S3)
+ sru.pxu: Use the new camera template jobs

plainbox-provider-tpm2:
+ Use a temp dir for container test setup to make it safe for running in 
parallel on a shared mount
+ Ensure all jobs run as root
+ Move files as checkbox running as root
+ Exclude tpm2_encryptdecrypt which does not seem to be supported so far

plainbox-provider-certification-client:
+ Make container tests safer for running in parallel with 
plainbox-provider-certification-client

plainbox-provider-certification-server:
+ Added olog and klog tests and new nested testplans to server suite 
testplans, LP: #1802152
+ Added nested firmware testplan to Firmware, Full, Functional and SoC 
testplans to re-introduce FWTS. LP: #1801961
+ Removed references to defunct Purley ID test from test plans. LP: #1800699

Cheers,

Sylvain Pineau