[Bug 2030482] Re: [MIR] s390-tools Rust dependencies (vendored)

[Simon Chopin]

** Description changed:

- TBD by Foundations.
+ [Availability]
+ The package s390-tools is already in Ubuntu main, and is re-reviewed due to signinficant changes in the package (new Rust code-base, including vendored dependencies).
+ The package s390-tools builds for the architectures it is designed to work on.
+ It currently builds and works for architectures: s390x, and to a much more limited extent, amd64, arm64 and ppc64el
+ Link to package https://launchpad.net/ubuntu/+source/s390-tools
  
- This is a heads-up about the upcoming s390-tools package pulling in new
- vendored Rust dependencies.
+ [Rationale]
+ - The package TBDSRC is required in Ubuntu main for hardware enablement on s390x machines
+ - The package TBDSRC will not generally be useful for a large part of
+   our user base, but is important/helpful still because it's necessary for the proper operation
+   of IBM Z mainframe.
+ - There is no other/better way to solve this that is already in main or
+   should go universe->main instead of this.
+ 
+ - The package TBDSRC is required in Ubuntu main no later than Mantic
+ Beta freeze
+ 
+ [Security]
+ - No CVEs/security issues in this software in the past (CVE-2021-25316 doesn't apply)
+ 
+ - no `suid` or `sgid` binaries
+ - There are a lot of binaries in /sbin, which is expected as they are used for machine administration.
+ - Package does install services, timers or recurring jobs
+   * cpacfstatsd -> system statistics
+   * cpi.service -> used to provide system data to the hypervisor
+   * cpuplugd.service -> CPU hotplug
+   * dumpconf.service -> Configures dumps on panics
+   * iucvtty-login at .service, ttyrun-getty at .service -> TTY handling
+   * mon_fsstatd.service, mon_procd.service -> monitoring
+ 
+ Vendored dependencies security history:
+ 
+ - https://github.com/rustsec/advisory-db/blob/main/crates/once_cell/RUSTSEC-2019-0017.md
+ - https://github.com/rustsec/advisory-db/tree/main/crates/openssl
+   -> Note that while the vendored crate is affected by RUSTSEC-2023-0044 the
+      relevant function is never called by the compiled binary, either directly or
+      indirectly.
+ - https://github.com/rustsec/advisory-db/blob/main/crates/serde_yaml/RUSTSEC-2018-0005.md
+ - https://github.com/rustsec/advisory-db/blob/main/crates/socket2/RUSTSEC-2020-0079.md
+ 
+ There doesn't seem to be any specific security features attached to
+ those services.
+ 
+ In addition, there are several udev rules shipped with the software, to
+ deal with s390-specific hardware.
+ 
+ - Packages does not contain extensions to security-sensitive software
+   (filters, scanners, plugins, UI skins, ...)
+ 
+ [Quality assurance - function/usage]
+ - The package works well right after install
+ 
+ [Quality assurance - maintenance]
+ - The package is maintained well in Ubuntu/Upstream and does
+   not have too many, long-term & critical, open bugs
+   - Ubuntu https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug
+     -> mostly feature requests
+   - Upstream's bug tracker: https://github.com/ibm-s390-linux/s390-tools/issues
+   Note that we've completely diverged from Debian, so their package isn't relevant to this MIR.
+   Upstream is heavily involved with the Ubuntu packaging, often providing us with verifications for SRUs
+   and tests of potential packages.
+ - The package does deal with exotic hardware, the Canonical Partners Engineering team has access
+   to the relevant machines to be able to test, fix and verify bugs.
+ 
+ [Quality assurance - testing]
+ - The package does not run a test at build time because no test suite is
+   provided upstream. Things recently changed a bit with the new Rust code
+   having a few tests, but I'm reluctant to enable them as the vendored
+   dependency tree would more than double in size (compressed!)
+ - The package does not run an autopkgtest.
+ 
+ RULE: - If no build tests nor autopkgtests are included, and/or if the package
+ RULE:   requires specific hardware to perform testing, the subscribed team
+ RULE:   must provide a written test plan in a comment to the MIR bug, and
+ RULE:   commit to running that test either at each upload of the package or
+ RULE:   at least once each release cycle. In the comment to the MIR bug,
+ RULE:   please link to the codebase of these tests (scripts or doc of manual
+ RULE:   steps) and attach a full log of these test runs. This is meant to
+ RULE:   assess their validity (e.g. not just superficial).
+ RULE:   If possible such things should stay in universe. Sometimes that is
+ RULE:   impossible due to the way how features/plugins/dependencies work
+ RULE:   but if you are going to ask for promotion of something untestable
+ RULE:   please outline why it couldn't provide its value (e.g. by splitting
+ RULE:   binaries) to users from universe.
+ RULE:   This is a balance that is hard to strike well, the request is that all
+ RULE:   options have been exploited before giving up. Look for more details
+ RULE:   and backgrounds https://github.com/canonical/ubuntu-mir/issues/30
+ RULE:   Just like in the SRU process it is worth to understand what the
+ RULE:   consequences a regression (due to a test miss) would be. Therefore
+ RULE:   if being untestable we ask to outline what consequences this would
+ RULE:   have for the given package. And let us be honest, even if you can
+ RULE:   test you are never sure you will be able to catch all potential
+ RULE:   regressions. So this is mostly to force self-awareness of the owning
+ RULE:   team than to make a decision on.
+ TODO: - The package can not be well tested at build or autopkgtest time
+ TODO:   because TBD. To make up for that:
+ TODO-A:   - We have access to such hardware in the team
+ TODO-B:   - We have allocated budget to get this hardware, but it is not here
+ TODO-B:     yet
+ TODO-C:   - We have checked with solutions-qa and will use their hardware
+ TODO-C:     through testflinger
+ TODO-D:   - We have checked with other team TBD and will use their hardware
+ TODO-D:     through TBD (eg. MAAS)
+ TODO-E:   - We have checked and found a simulator which covers this case
+ TODO-E:     sufficiently for testing, our plan to use it is TBD
+ TODO-F:   - We have engaged with the upstream community and due to that
+ TODO-F:     can tests new package builds via TBD
+ TODO-G:   - We have engaged with our user community and due to that
+ TODO-G:     can tests new package builds via TBD
+ TODO-H:   - We have engaged with the hardware manufacturer and made an
+ TODO-H:     agreement to test new builds via TBD
+ TODO-A-H: - Based on that access outlined above, here are the details of the
+ TODO-A-H:   test plan/automation TBD (e.g. script or repo) and (if already
+ TODO-A-H:   possible) example output of a test run: TBD (logs).
+ TODO-A-H:   We will execute that test plan
+ TODO-A-H1:  on-uploads
+ TODO-A-H2:  regularly (TBD details like frequency: monthly, infra: jira-url)
+ TODO-X:   - We have exhausted all options, there really is no feasible way
+ TODO-X:     to test or recreate this. We are aware of the extra implications
+ TODO-X:     and duties this has for our team (= help SEG and security on
+ TODO-X:     servicing this package, but also more effort on any of your own
+ TODO-X:     bug triage and fixes).
+ TODO-X:     Due to TBD there also is no way to provide this to users from
+ TODO-X:     universe.
+ TODO-X:     Due to the nature, integration and use cases of the package the
+ TODO-X:     consequences of a regression that might slip through most likely
+ TODO-X:     would include
+ TODO-X:     - TBD
+ TODO-X:     - TBD
+ TODO-X:     - TBD
+ 
+ [Quality assurance - packaging]
+ - debian/watch is present and works
+ - debian/control defines a correct Maintainer field
+ 
+ - Recent build logs
+ https://launchpadlibrarian.net/682423862/buildlog_ubuntu-mantic-s390x.s390-tools_2.29.0-0ubuntu1_BUILDING.txt.gz
+ 
+ There is the usual issue of noisy Rust warnings in the dependencies.
+ 
+ - Lintian output is attached. It doesn't look very good, probably due to the
+   fact that since the package basically only fully build on s390x we rarely
+   produce binary packages on development machines, which is where Lintian runs
+   would usually scream at us.
+ 
+ - Lintian overrides are present, but ok because they're about Ubuntu-specific
+   source fields.
+ 
+ - This package does not rely on obsolete or about to be demoted packages.
+ - This package has no python2 or GTK2 dependencies
+ 
+ - The package will be installed by default on s390x, but does not ask debconf
+   questions higher than medium
+ 
+ - Packaging and build is fairly easy, link to debian/rules:
+ https://git.launchpad.net/ubuntu/+source/s390-tools/tree/debian/rules
+ 
+ There's a little bit of complexity due to the signing requirements, the fact
+ that is mostly builds on s390x, and also due to the Rust integration, but it's
+ still mostly straightforward.
+ 
+ [UI standards]
+ - Application is not end-user facing (does not need translation)
+ 
+ [Dependencies]
+ - No further depends or recommends dependencies that are not yet in main
+ 
+ [Standards compliance]
+ - This package correctly follows FHS and Debian Policy
+ 
+ [Maintenance/Owner]
+ - Foundations team is already subscribed to the package. Note that most of the
+   day-to-day work is done by Frank Heimes
+ 
+ - This does not use static builds using static archive from other
+ packages.
+ 
+ - The Foundations team is aware of the implications of vendored code and (as
+   alerted by the security team) commits to provide updates and backports
+   to the security team for any affected vendored code for the lifetime
+   of the release (including ESM).
+ 
+ - This package uses vendored rust code tracked in the Vendored-Sources-Rust field
+   in the package, refreshing that code is outlined in debian/README.source
+ 
+ - This package is rust based and vendors all non language-runtime dependencies.
+   To be noted, upstream has defined a policy regarding which Rust dependencies
+   are acceptable, whic hseems fairly sensible and should reduce the inevitable growth
+   of that dep tree:
+ 
+   https://github.com/ibm-s390-linux/s390-tools/tree/master/rust#what-
+ third-party-crates-can-be-used-for-s390-tools
+ 
+ - The package has been built in the archive more recently than the last
+   test rebuild
  
  Feature request: bug #2030316
  Original s390-tools MIR: bug #1521984

** Attachment added: "lintian"
   https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug/2030482/+attachment/5694998/+files/lintian

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to s390-tools in Ubuntu.
https://bugs.launchpad.net/bugs/2030482

Title:
  [MIR] s390-tools Rust dependencies (vendored)

Status in s390-tools package in Ubuntu:
  Incomplete

Bug description:
  [Availability]
  The package s390-tools is already in Ubuntu main, and is re-reviewed due to signinficant changes in the package (new Rust code-base, including vendored dependencies).
  The package s390-tools builds for the architectures it is designed to work on.
  It currently builds and works for architectures: s390x, and to a much more limited extent, amd64, arm64 and ppc64el
  Link to package https://launchpad.net/ubuntu/+source/s390-tools

  [Rationale]
  - The package TBDSRC is required in Ubuntu main for hardware enablement on s390x machines
  - The package TBDSRC will not generally be useful for a large part of
    our user base, but is important/helpful still because it's necessary for the proper operation
    of IBM Z mainframe.
  - There is no other/better way to solve this that is already in main or
    should go universe->main instead of this.

  - The package TBDSRC is required in Ubuntu main no later than Mantic
  Beta freeze

  [Security]
  - No CVEs/security issues in this software in the past (CVE-2021-25316 doesn't apply)

  - no `suid` or `sgid` binaries
  - There are a lot of binaries in /sbin, which is expected as they are used for machine administration.
  - Package does install services, timers or recurring jobs
    * cpacfstatsd -> system statistics
    * cpi.service -> used to provide system data to the hypervisor
    * cpuplugd.service -> CPU hotplug
    * dumpconf.service -> Configures dumps on panics
    * iucvtty-login at .service, ttyrun-getty at .service -> TTY handling
    * mon_fsstatd.service, mon_procd.service -> monitoring

  Vendored dependencies security history:

  - https://github.com/rustsec/advisory-db/blob/main/crates/once_cell/RUSTSEC-2019-0017.md
  - https://github.com/rustsec/advisory-db/tree/main/crates/openssl
    -> Note that while the vendored crate is affected by RUSTSEC-2023-0044 the
       relevant function is never called by the compiled binary, either directly or
       indirectly.
  - https://github.com/rustsec/advisory-db/blob/main/crates/serde_yaml/RUSTSEC-2018-0005.md
  - https://github.com/rustsec/advisory-db/blob/main/crates/socket2/RUSTSEC-2020-0079.md

  There doesn't seem to be any specific security features attached to
  those services.

  In addition, there are several udev rules shipped with the software,
  to deal with s390-specific hardware.

  - Packages does not contain extensions to security-sensitive software
    (filters, scanners, plugins, UI skins, ...)

  [Quality assurance - function/usage]
  - The package works well right after install

  [Quality assurance - maintenance]
  - The package is maintained well in Ubuntu/Upstream and does
    not have too many, long-term & critical, open bugs
    - Ubuntu https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug
      -> mostly feature requests
    - Upstream's bug tracker: https://github.com/ibm-s390-linux/s390-tools/issues
    Note that we've completely diverged from Debian, so their package isn't relevant to this MIR.
    Upstream is heavily involved with the Ubuntu packaging, often providing us with verifications for SRUs
    and tests of potential packages.
  - The package does deal with exotic hardware, the Canonical Partners Engineering team has access
    to the relevant machines to be able to test, fix and verify bugs.

  [Quality assurance - testing]
  - The package does not run a test at build time because no test suite is
    provided upstream. Things recently changed a bit with the new Rust code
    having a few tests, but I'm reluctant to enable them as the vendored
    dependency tree would more than double in size (compressed!)
  - The package does not run an autopkgtest.

  RULE: - If no build tests nor autopkgtests are included, and/or if the package
  RULE:   requires specific hardware to perform testing, the subscribed team
  RULE:   must provide a written test plan in a comment to the MIR bug, and
  RULE:   commit to running that test either at each upload of the package or
  RULE:   at least once each release cycle. In the comment to the MIR bug,
  RULE:   please link to the codebase of these tests (scripts or doc of manual
  RULE:   steps) and attach a full log of these test runs. This is meant to
  RULE:   assess their validity (e.g. not just superficial).
  RULE:   If possible such things should stay in universe. Sometimes that is
  RULE:   impossible due to the way how features/plugins/dependencies work
  RULE:   but if you are going to ask for promotion of something untestable
  RULE:   please outline why it couldn't provide its value (e.g. by splitting
  RULE:   binaries) to users from universe.
  RULE:   This is a balance that is hard to strike well, the request is that all
  RULE:   options have been exploited before giving up. Look for more details
  RULE:   and backgrounds https://github.com/canonical/ubuntu-mir/issues/30
  RULE:   Just like in the SRU process it is worth to understand what the
  RULE:   consequences a regression (due to a test miss) would be. Therefore
  RULE:   if being untestable we ask to outline what consequences this would
  RULE:   have for the given package. And let us be honest, even if you can
  RULE:   test you are never sure you will be able to catch all potential
  RULE:   regressions. So this is mostly to force self-awareness of the owning
  RULE:   team than to make a decision on.
  TODO: - The package can not be well tested at build or autopkgtest time
  TODO:   because TBD. To make up for that:
  TODO-A:   - We have access to such hardware in the team
  TODO-B:   - We have allocated budget to get this hardware, but it is not here
  TODO-B:     yet
  TODO-C:   - We have checked with solutions-qa and will use their hardware
  TODO-C:     through testflinger
  TODO-D:   - We have checked with other team TBD and will use their hardware
  TODO-D:     through TBD (eg. MAAS)
  TODO-E:   - We have checked and found a simulator which covers this case
  TODO-E:     sufficiently for testing, our plan to use it is TBD
  TODO-F:   - We have engaged with the upstream community and due to that
  TODO-F:     can tests new package builds via TBD
  TODO-G:   - We have engaged with our user community and due to that
  TODO-G:     can tests new package builds via TBD
  TODO-H:   - We have engaged with the hardware manufacturer and made an
  TODO-H:     agreement to test new builds via TBD
  TODO-A-H: - Based on that access outlined above, here are the details of the
  TODO-A-H:   test plan/automation TBD (e.g. script or repo) and (if already
  TODO-A-H:   possible) example output of a test run: TBD (logs).
  TODO-A-H:   We will execute that test plan
  TODO-A-H1:  on-uploads
  TODO-A-H2:  regularly (TBD details like frequency: monthly, infra: jira-url)
  TODO-X:   - We have exhausted all options, there really is no feasible way
  TODO-X:     to test or recreate this. We are aware of the extra implications
  TODO-X:     and duties this has for our team (= help SEG and security on
  TODO-X:     servicing this package, but also more effort on any of your own
  TODO-X:     bug triage and fixes).
  TODO-X:     Due to TBD there also is no way to provide this to users from
  TODO-X:     universe.
  TODO-X:     Due to the nature, integration and use cases of the package the
  TODO-X:     consequences of a regression that might slip through most likely
  TODO-X:     would include
  TODO-X:     - TBD
  TODO-X:     - TBD
  TODO-X:     - TBD

  [Quality assurance - packaging]
  - debian/watch is present and works
  - debian/control defines a correct Maintainer field

  - Recent build logs
  https://launchpadlibrarian.net/682423862/buildlog_ubuntu-mantic-s390x.s390-tools_2.29.0-0ubuntu1_BUILDING.txt.gz

  There is the usual issue of noisy Rust warnings in the dependencies.

  - Lintian output is attached. It doesn't look very good, probably due to the
    fact that since the package basically only fully build on s390x we rarely
    produce binary packages on development machines, which is where Lintian runs
    would usually scream at us.

  - Lintian overrides are present, but ok because they're about Ubuntu-specific
    source fields.

  - This package does not rely on obsolete or about to be demoted packages.
  - This package has no python2 or GTK2 dependencies

  - The package will be installed by default on s390x, but does not ask debconf
    questions higher than medium

  - Packaging and build is fairly easy, link to debian/rules:
  https://git.launchpad.net/ubuntu/+source/s390-tools/tree/debian/rules

  There's a little bit of complexity due to the signing requirements, the fact
  that is mostly builds on s390x, and also due to the Rust integration, but it's
  still mostly straightforward.

  [UI standards]
  - Application is not end-user facing (does not need translation)

  [Dependencies]
  - No further depends or recommends dependencies that are not yet in main

  [Standards compliance]
  - This package correctly follows FHS and Debian Policy

  [Maintenance/Owner]
  - Foundations team is already subscribed to the package. Note that most of the
    day-to-day work is done by Frank Heimes

  - This does not use static builds using static archive from other
  packages.

  - The Foundations team is aware of the implications of vendored code and (as
    alerted by the security team) commits to provide updates and backports
    to the security team for any affected vendored code for the lifetime
    of the release (including ESM).

  - This package uses vendored rust code tracked in the Vendored-Sources-Rust field
    in the package, refreshing that code is outlined in debian/README.source

  - This package is rust based and vendors all non language-runtime dependencies.
    To be noted, upstream has defined a policy regarding which Rust dependencies
    are acceptable, whic hseems fairly sensible and should reduce the inevitable growth
    of that dep tree:

    https://github.com/ibm-s390-linux/s390-tools/tree/master/rust#what-
  third-party-crates-can-be-used-for-s390-tools

  - The package has been built in the archive more recently than the last
    test rebuild

  Feature request: bug #2030316
  Original s390-tools MIR: bug #1521984

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/s390-tools/+bug/2030482/+subscriptions