[Bug 1961427] Re: zlib: compressBound() returns an incorrect result on z15

[Steve Langasek]

Hello bugproxy, or anyone else affected,

Accepted zlib into jammy-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/zlib/1:1.2.11.dfsg-2ubuntu9.1 in a
few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
jammy to verification-done-jammy. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-jammy. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: zlib (Ubuntu Jammy)
       Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to zlib in Ubuntu.
https://bugs.launchpad.net/bugs/1961427

Title:
  zlib: compressBound() returns an incorrect result on z15

Status in Ubuntu on IBM z Systems:
  In Progress
Status in bedtools package in Ubuntu:
  Fix Released
Status in htslib package in Ubuntu:
  Invalid
Status in zlib package in Ubuntu:
  Fix Committed
Status in bedtools source package in Focal:
  Invalid
Status in htslib source package in Focal:
  Fix Committed
Status in zlib source package in Focal:
  Fix Committed
Status in bedtools source package in Impish:
  Won't Fix
Status in htslib source package in Impish:
  Won't Fix
Status in zlib source package in Impish:
  Won't Fix
Status in bedtools source package in Jammy:
  Fix Committed
Status in htslib source package in Jammy:
  Invalid
Status in zlib source package in Jammy:
  Fix Committed

Bug description:
  SRU Justification:
  ==================

  [Impact]

  * zlib: compressBound() returns an incorrect result on IBM z15
  hardware.

  * Passing the result of compressBound() to compress() results
    in an error code.

  * This is because compressBound() is not adjusted for DFLTCC.

  [Fix]

  * Adjust compressBound() for DFLTCC like it's already done
    for deflateBound().

  * Since zlib project does not accept patches at the moment,
    the fix has been integrated into the DFLTCC pull request:
    https://github.com/madler/zlib/pull/410
    The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  * The fix extracted out of the above is:
    https://launchpadlibrarian.net/589857296/debdiff_zlib_1.2.11.dfsg-2ubuntu7_to_zlib_1.2.11.dfsg-2ubuntu8_jammy.diff

  * On top of this actual zlib fix, there is another patch needed:
   'Remove compressBound assertions. (PR #1258)' for htslib.

  * But there is a standalone 'htslib' package version, as well as a
    htslib version included in (some) 'bedtools' packages.
    Both need to be patched (see '[Other]' for more details).

  [Test Plan]

  * An IBM z15 system (LPAR, z/VM guest or KVM virtual machine)
    with Ubuntu Server 21.10 (or 22.04).

  * A test can be done  based on the following C test program:
        #include <assert.h>
        #include <stdlib.h>
        #include <zlib.h>
        int main() {
            Bytef in_buf[128], out_buf[1024];
            for (size_t i = 0; i < sizeof(in_buf); i++)
                in_buf[i] = rand();
            uLongf dest_len = compressBound(sizeof(in_buf));
            assert(dest_len <= sizeof(out_buf));
            int ret = compress(out_buf, &dest_len,
                               in_buf, sizeof(in_buf));
            assert(ret == Z_OK);
            }

  * The test needs to be done by IBM, due to the requirements
    for the special z15 hardware.

  * A successful test was just completed, based on the version in jammy-
  proposed, which is at the same code level that the impish version this
  SRU is targeted for.

  [Where problems could occur]

  * If the adjustment of compressBound() for DFLTCC is done
    erroneously the issue can still be present or in worst case
    even affect Z systems other than z15 only.

  * The compression can become errorneous with the new changes,
    e.g. in compressBound.

  * Mistakes in dfltcc_free_window OF and especially DEFLATE_BOUND_COMPLEN,
    (incl. the bit definitions), may cause various and unforseen defects.

  * Any build time issues that might have been introduced by this patch
    can be identified by a test build; this was done and is available here:
    https://launchpad.net/~fheimes/+archive/ubuntu/lp1961427

  [Other Info]

  * Ubuntu jammy, impish and focal are affected by the zlib issue.

  * The 'htslib' version '1.13+ds' (as it is part of I, J and K),
    already includes the patch,
    hence only htslib '1.10.2' in focal needs to be patched.

  * The bedtools version '2.30.0+dfsg' (as it is part of I, J and K),
    requires the patch,
    but version '2.27.1+dfsg' bedtools in focal does not incl. an
    embedded htslib, hence does not need to be (actually can't be)
    patched.

  * Patched version of the affected htslib and bedtools packages
    are build and also available at this PPA:
    https://launchpad.net/~fheimes/+archive/ubuntu/lp1961427  

  __________

  Description:   zlib: compressBound() returns an incorrect result on z15
  Symptom:       Passing the result of compressBound() to compress()
                 results in an error code.
  Problem:       compressBound() is not adjusted for DFLTCC.
  Solution:      Adjust compressBound() for DFLTCC like it's already done
                 for deflateBound(). Since zlib project does not accept
                 patches at the moment, the fix has been integrated into
                 the DFLTCC pull request:
                 https://github.com/madler/zlib/pull/410
                 The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  Reproduction:  z15 only:
                 #include <assert.h>
                 #include <stdlib.h>
                 #include <zlib.h>
                 int main() {
                     Bytef in_buf[128], out_buf[1024];
                     for (size_t i = 0; i < sizeof(in_buf); i++)
                         in_buf[i] = rand();
                     uLongf dest_len = compressBound(sizeof(in_buf));
                     assert(dest_len <= sizeof(out_buf));
                     int ret = compress(out_buf, &dest_len,
                                        in_buf, sizeof(in_buf));
                     assert(ret == Z_OK);
                 }

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1961427/+subscriptions