[Bug 2043625] Re: libsolv: lack of zstd support makes it impossible to create SUSE Tumbleweed/Fedora Rawhide chroots with zypper/dnf

Thu Nov 16 16:36:47 UTC 2023

Hi Luca, I hope this finds you well.

I *really* appreciate you submitting this as a Stable Release Update for
Ubuntu. I see you're a Debian Developer and have already submitted the
appropriate paperwork for a Debian Stable update; thank you very much
for your work on that as well.

As a fellow Debian Developer, I have a baseline level of trust in your
uploads. That being said, the changes are minimal and clean, and you
appropriately attached the bug report to the changelog entry. Simply
speaking regarding packaging, you hit the nail on the head (which is
what I would expect, of course ;) ).

I'm sure you're aware: the Ubuntu Stable Release Updates Team reserves
the right to accept or reject this upload, based on the contents of the
upload and your bug report. Fortunately, unlike Debian, you're able to
edit your report to include additional information.

I would highly suggest expanding on any regression potential. Could a
regression in the underlying libzstd library negatively affect existing
users? How would those regressions present themselves? Try to be overly
verbose. (I attended a talk at the Ubuntu Summit regarding SRUs, and one
of the major points was, "we usually reject uploads with a Regression
Potential of "very minimal." Not my decision, just be aware. :) )

Thanks again for your stellar work!

** Changed in: libsolv (Ubuntu Jammy)
     Assignee: (unassigned) => Luca Boccassi (bluca)

** Changed in: libsolv (Ubuntu Lunar)
     Assignee: (unassigned) => Luca Boccassi (bluca)

** Changed in: libsolv (Ubuntu Mantic)
     Assignee: (unassigned) => Luca Boccassi (bluca)

** Changed in: libsolv (Ubuntu Noble)
     Assignee: (unassigned) => Luca Boccassi (bluca)

** Changed in: libsolv (Ubuntu Jammy)
   Importance: Undecided => High

** Changed in: libsolv (Ubuntu Lunar)
   Importance: Undecided => High

** Changed in: libsolv (Ubuntu Mantic)
   Importance: Undecided => High

** Changed in: libsolv (Ubuntu Noble)
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/2043625

Title:
  libsolv: lack of zstd support makes it impossible to create SUSE
  Tumbleweed/Fedora Rawhide chroots with zypper/dnf

Status in libsolv package in Ubuntu:
  Fix Committed
Status in libsolv source package in Jammy:
  In Progress
Status in libsolv source package in Lunar:
  In Progress
Status in libsolv source package in Mantic:
  In Progress
Status in libsolv source package in Noble:
  Fix Committed

Bug description:
  Both Fedora Rawhide and SUSE Tumbleweed started to compress their
  respective RepoData with zstd. The libsolv version in all Ubuntu
  releases is not build with zstd support, so using zypper/dnf from any
  Ubuntu version to build a new Fedora/SUSE chroot started to fail this
  week.

  Noble will be fixed in the next sync of libsolv from Debian unstable,
  where I enabled libzstd support in 0.7.26-2. I would like SRUs all the
  way back to jammy to fix the CI jobs. The fix is very low risk as it
  enables an already available and optional feature, that is exercised
  only when referring to zstd-compressed repositories, which currently
  cannot work at all without this flag enabled.

  There is an older distro-wide LP tracking libzstd enablement across
  all packages:
  https://bugs.launchpad.net/ubuntu/+source/libsolv/+bug/1923845

  Support for zstd was added upstream in version 0.6.35:
  https://github.com/openSUSE/libsolv/commit/f51d048e6774c180a9fef21b87d7303accae3809

  Fix proposal for Debian stable: https://bugs.debian.org/cgi-
  bin/bugreport.cgi?bug=1056006

  [Impact]

  This affects our CI jobs in the systemd projects on Github Actions,
  which are running on Jammy and using dnf and zypper (with libsolv) to
  build images. e.g.:

  https://pipelinesghubeus21.actions.githubusercontent.com/fW9YnoEKBASdi1Tl6Qk8OxwlrJgIZvKOJ6c0FMLt24imZ5L13v/_apis/pipelines/1/runs/254815/signedlogcontent/8?urlExpires=2023-11-15T23%3A13%3A32.0430115Z&urlSigningMethod=HMACV1&urlSignature=vLbcJy6TM0PlI%2FQQsuRjhqwKLKo00%2B%2FvuURQuAWNpac%3D
  https://pipelinesghubeus21.actions.githubusercontent.com/fW9YnoEKBASdi1Tl6Qk8OxwlrJgIZvKOJ6c0FMLt24imZ5L13v/_apis/pipelines/1/runs/254815/signedlogcontent/4?urlExpires=2023-11-15T23%3A13%3A49.1567762Z&urlSigningMethod=HMACV1&urlSignature=deiHR5B%2FxW9RXe%2FtC5YYYLDcWnCMa1gDGkYciaU3amo%3D

  [Major Changes]

  The libsolv versions in Ubuntu already support zstd, but it is not
  enabled at build time. It is trivially enabled with a build-dependency
  on libzstd and by passing a build configuration flag to cmake.

  [Test Plan]

  Install zypper, create a minimal config file and try to bootstrap a
  minimal SUSE Tumbleweed chroot with these 3 simple commands:

  mkdir -p repos.d img
  cat <<EOF>repos.d/test.repo
  [test.repo]
  name=test.repo
  baseurl=https://download.opensuse.org/tumbleweed/repo/oss/
  gpgcheck=1
  enabled=1
  autorefresh=0
  keeppackages=1
  EOF
  zypper --reposd-dir=$PWD/repos.d --root=$PWD/img --gpg-auto-import-keys install distribution-release filesystem

  If it works, then the fix worked. Without the fix, the following error
  can be seen:

  Building repository 'test.repo' cache .........................................................................[error]
  Error building the cache:
  [test.repo|https://download.opensuse.org/tumbleweed/repo/oss/] Failed to cache repo (1).
  History:
   - 'repo2solv' '-o' '/tmp/img/var/cache/zypp/solv/test.repo/solv' '-X' '/tmp/img/var/cache/zypp/raw/test.repo'
     /tmp/img/var/cache/zypp/raw/test.repo/repodata/d6fbf1152bab99fc7ceacf974422a9799694274b64c36015b10288e6cabadd81e4649b19f52570efc5f3ab5b28817c9561fa8eeca117a05f3caea6c33e48cb69-primary.xml.zst: No such file or directory
     Command exited with status 1.

  Warning: Skipping repository 'test.repo' because of the above error.
  Some of the repositories have not been refreshed because of an error.
  Loading repository data...
  Reading installed packages...
  'distribution-release' not found in package names. Trying capabilities.
  No provider of 'distribution-release' found.
  'filesystem' not found in package names. Trying capabilities.
  No provider of 'filesystem' found.
  Resolving package dependencies...
  Nothing to do.

  [Regression Potential]
  Very minimal. The feature had been supported for many versions prior to the current ones, and when enabled it only affects code paths that deal with zstd compressed artifacts, which currently cannot work at all due to the lack of support. The worst case scenario is that support for zstd does not work in some ways, which is not any worse than status quo.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libsolv/+bug/2043625/+subscriptions