New Checkbox release available in our Stable PPA! (2019-01-24)

[Sylvain Pineau]

Hello all!

A new release of Checkbox is available in the stable PPA:
https://launchpad.net/~hardware-certification/+archive/ubuntu/public

It includes a new version of the following components:

- checkbox-ng 1.3.0
- checkbox-support 0.40.0
- plainbox-provider-resource 0.39.0
- plainbox-provider-checkbox 0.47.0
- plainbox-provider-sru 1.13.0
- plainbox-provider-tpm2 1.9.0
- plainbox-provider-certification-client 0.36.0
- plainbox-provider-certification-server 0.41.0

*HIGHLIGHTS*

- New checkbox-cli merge commands
   See https://yphus.github.io/test/ubuntu/2018/10/31/merge-commands.html
- Hardware summary available in HTML reports
- Remote API V4
- Show less advice when running provider validation
- New BT LE Beacon test
- Re-introduce FWTS in server test plans

*Bug fixes*
https://goo.gl/dgjeJK

*Changelog*

checkbox-ng:
+ subcommands: Add a --nofake option to tp-export
+ docs: document side-loading providers
+ assistant: override providers one by one with side-loading
+ remote: Don't call wait_for_interaction_prompt for manual jobs
+ remote: Display the verification steps for interactive jobs
+ remote: fix interactive jobs
+ remote: fix handling of input pipe when restarting SA
+ remote: remove stale TODO comment
+ remote: sprinkle logging calls all over the master
+ remote: fix crashing when master is disconnected very early on
+ remote: tidy up references to session_change_lock
+ remote: when connecting while bootstrapping restart the session
+ remote: ensure session lock is released when restarting
+ remote: properly restart session when launcher are used
+ remote: add API for handling input from master
+ remote: use daemonic controller
+ add daemonic execution controller
+ add `normal-user` variable to launchers/configs
+ ctrl: accept FO for subprocess' stdin as ctrl's param
+ remote: print the outcome for non-bootstrap jobs
+ launchers:merge*: Make -o a required argument
+ exporters:html: Update unit tests
+ remote: print the outcome for non-bootstrap jobs
+ exporters: Add a new system info section to html reports
+ checkbox-cli: Explicit actions when failing to merge submissions
+ remote: fix unicode errors
+ remote: bump remote api version
+ remote: remove unnecessary return
+ remote: allow monitor_job in TestsSelected state
+ update unit tests for provider_manager
+ unit:testplan: kill advice in testplan
+ unit:file: kill advice for file unit
+ unit:template: kill unit type advice
+ unit:job: kill unnecessary advice
+ remote: track progress through interrupted session
+ remote_assistant: populate done sublist in progress
+ session:assistant: add get_dynamic_done_list()
+ exporters:data:multi-page.html: Remove the $(...).toolbar() call
+ remote: filter tps when defined in launcher
+ plainbox:xparsers: fix doctest failing on py37
+ Increase timeout for remote connection to 10min.
+ exporters:html: Disable autoescape for comments to preserve HTML tags
+ exporter:xlsx: Add category names to tp-export reports
+ checkbox-cli: Refactor merge-reports/submissions code
+ plainbox:test-data: Update HTML reference files
+ checkbox-cli: New merge-submissions command
+ exporters:html: Upgrade existing html exporter with chart.js 2.7.3
+ checkbox-cli: New merge-reports command
+ checkbox-cli:run: Add both title and description options to the command
+ exporters:jinja2: New dump_from_session_manager_list method
+ exporters:html-multi-page: New exporter to record multiple sessions in 
a single HTML
+ exporters:html-multi-page: New Chart.js (2.7.3)
+ exporters:json: Add category_id, attachment-results and category_map
+ session:state: Do not consider resources/attachments for global outcome
+ remote: track progress through interrupted session
+ remote_assistant: populate done sublist in progress
+ session:assistant: add get_dynamic_done_list()
+ exporters:data:multi-page.html: Remove the $(...).toolbar() call
+ remote: filter tps when defined in launcher
+ plainbox:xparsers: fix doctest failing on py37
+ Increase timeout for remote connection to 10min.
+ exporters:html: Disable autoescape for comments to preserve HTML tags
+ exporter:xlsx: Add category names to tp-export reports
+ checkbox-cli: Refactor merge-reports/submissions code
+ plainbox:test-data: Update HTML reference files
+ checkbox-cli: New merge-submissions command
+ exporters:html: Upgrade existing html exporter with chart.js 2.7.3
+ checkbox-cli: New merge-reports command
+ checkbox-cli:run: Add both title and description options to the command
+ exporters:jinja2: New dump_from_session_manager_list method
+ exporters:html-multi-page: New exporter to record multiple sessions in 
a single HTML
+ exporters:html-multi-page: New Chart.js (2.7.3)
+ exporters:json: Add category_id, attachment-results and category_map
+ session:state: Do not consider resources/attachments for global outcome
+ exporters: add resource-results to the json template
+ checkbox-cli: remove --partial option from list-bootstrapped
+ checkbox-cli: default formatting option when using list-bootstrapped
+ ctrl: Don't process templates when there's no resources (i.e no I/O logs)
+ subcommands: Exit properly the run command after a CTRL-C
+ Check to see that the UndesiredJobReadinessInhibitor exists before 
removing it in SessionState

checkbox-support:
+ scripts:run_watcher: Add support for mediacard events
+ parsers:udev: Set pid/vid in base16 for canbus and v4l devices (for 
consistency)
+ parsers:udevadm: Report NVDIMM devices as DISK
+ parsers:udevadm: Set bus to mediacard for the PARTITION category (mmc)
+ add hostfs as a proper root partitions in udevadm parser
+ Added bits that are requested for the server suite. Adds new --server 
option to run these.
   Adds --quiet option to suppress lengthy console output. LP: #1801960
+ Fix pattern match for /writable mountpoint to work with core18 also
+ network: removing network script in favour of p-p-c
+ scripts:eddystone_scanner: power on the adapter via bluetoothctl
+ scripts:eddystone_scanner: Use logging
+ scripts:eddystone_scanner: Remove asyncio async/await to run on Python 3.4
+ parsers:udevadm: Add interface (hciX) info to BLUETOOTH devices
+ parsers:udevadm: Fix indentation
+ scripts:eddystone_scanner: New beacon scanner
+ Vendorize https://github.com/frawau/aioblescan
+ parsers:udevadm: Identify CAPTURE devices using the V4L subsystem only
+ Add snap_connect.py scripta and expose
+ scripts:fwts_test: Take into account SKIPPED and WARNING exit summmary
+ scripts:fwts_test: Fix summary parsing to only fail after the last check
+ scripts:fwts_test: set suspend/resume timings to N/A when not found in 
logs
+ scripts:fwts_test: open logs using "error='ignore'"
+ scripts:fwts_test: Get suspend/resume timings from fwts, not syslog
+ scripts:fwts_test: Flush the progress messages to follow them on stdout
+ scripts:fwts_test:Mask the Zenity GTK (harmless) warning
+ parsers:tests: Updates to account for the new HIDRAW category
+ parsers:tests: Drop redondant tests
+ parsers:tests:new udevadm sample to check microchip USB2SPI is 
reported as HIDRAW
+ parsers:udevadm: Add HIDRAW category
+ Full rewrite of run_watcher script to monitor systemd journal instead 
of syslog

plainbox-provider-resource:
+ bin:udev_resource: Add a --short option to the list command
+ bin:snapd_resource: Handle systems with no plugs/slots

plainbox-provider-checkbox:
+ units:touchpad: Remove singletouch-automated and multitouch-automated
+ units:touchpad: Remove obsolete touchpad/singletouch-corner
+ add /etc/media-info as an alternative to buildstamp
+ bluetooth: add bluez controller detect job
+ units:graphics: update xorg-process detection method
+ units:mediacard: leverage the checkbox-support utilities to run 
non-udisks2 tests on UC
+ bluetooth: create bluetooth detect job
+ Increase timeout in memory_stress_ng from 150% to 200% of expected run 
time
+ oobm: initial tests for Intel AMT testing
+ ethernet: add after-suspend manual plan (empty)
+ skip system_info_json on dragonboard because it crashes
+ ethernet: allow check static to be used on classic
+ Add a2dp to list of BT blockers LP:#1803645
+ bin/inxi_snapshot: Drop the check for tty
+ Make container tests safer for running in parallel with 
plainbox-provider-checkbox
+ bin:system_info: print error msg coming from subprocesses
+ Modified network script to call iperf3 in multiple threads; necessary 
for testing high-speed (40Gbps and faster) NICs.
+ bin:system_info: Forcing usb data to be collected via sysfs
+ socketcan: add manual test plans for nesting
+ disk: add manual test plans for nesting
+ add missing 'also-after-suspend' flag for job usb3/insert
+ units:miscellaneous: Add kernel_cmdline_attachment to submission-resources
+ units:miscellaneous: Add system_info_json to submission-resources
+ units:submission: Add a new job calling system_info.py
+ bin:system_info.py: New tool to gather system info from inxi and udev 
parser
+ bin:inxi_snapshot: Add inxi system info command line tool
+ added missing include: to testplan
+ Added category_id and summary fields to new jobs
+ Add klog and olog tests to miscellanea for server.  Added server 
miscellanea test plan to start nesting in the server suite. LP: #1802152
+ units:stress: Remove stress-cert-automated in stress-cert-manual
+ Added server focused fwts jobs to firmware/jobs.pxu. Added new 
test-plan.pxu with a test plan to nest in server test plans. LP: #1801963
+ Removing purley test
+ bin:dkms_info: Skip malformed modaliases
+ networking: initial ipv6 jobs and devices test plans
+ units: Add EddyStone URL scan tests (before and after S3)
+ network_info: fix scraping ip for ipv6 address
+ network: use local script as most up-to-date
+ check-static: import script to go with test
+ ethernet: import test plans from p-p-s
+ units:misc: Add BLUETOOTH to device check test
+ units:bluetooth: power on is now done by the eddystone_scanner script
+ units:bluetooth: Fail the eddystone scan test if one of the commands 
failed
+ units:bluetooth: Bring down the device after the eddystone scan test
+ units:bluetooth: Ensure the device is up and running before scanning 
for beacon ads
+ units:bluetooth: Add new BT LE beacon test using EddyStone URL 
advertisements
+ units:bluetooth:test-plan.pxu: Fix indentation
+ units:camera: Remove the unnecessary dep on libcheese
+ renamed server info testplan
+ Added a server info-attachment testplan too so that I can start taking 
advantage of nested testplans.
+ Added a second lstopo job to create a small png system map. Modified 
original job.
+ units:cpu: Only run cpu/cstates on Intel
+ added lstopo_attachment to the info-attachment-cert-automated testplan
+ Add new attachment job to run lstopo command
+ Fixed issue that breaks ipmi_test on some systems that require a 
specific channel to be used. LP: #1794926
+ units:misc: CAPTURE devices added to miscellanea/device_check
+ units:camera: Use template jobs to drive V4L devices
+ bin:camera_test: Use GStreamer camerabin instead of libcheese
+ Add category to template job `ethernet/hotplug-*-end-cycle`

plainbox-provider-sru:
+ units: Remove OBEX tests to prefer EddyStone URL scan tests (before 
and after S3)
+ sru.pxu: Use the new camera template jobs

plainbox-provider-tpm2:
+ Use a temp dir for container test setup to make it safe for running in 
parallel on a shared mount
+ Ensure all jobs run as root
+ Move files as checkbox running as root
+ Exclude tpm2_encryptdecrypt which does not seem to be supported so far

plainbox-provider-certification-client:
+ Make container tests safer for running in parallel with 
plainbox-provider-certification-client

plainbox-provider-certification-server:
+ Converted info/attachment jobs over to use the nested plan 
server-info-attachment-automated LP: #1812684
+ Added launchers to make retesting memory and cpu stress easier. LP: 
#1811704
+ Added olog and klog tests and new nested testplans to server suite 
testplans, LP: #1802152
+ Added nested firmware testplan to Firmware, Full, Functional and SoC 
testplans to re-introduce FWTS. LP: #1801961
+ Removed references to defunct Purley ID test from test plans. LP: #1800699

Cheers,

Sylvain Pineau