[Bug 1961427] Re: zlib: compressBound() returns an incorrect result on z15

[Łukasz Zemczak]

Hey Frank! You switched the focal verification tags, but I do not see
any verification of the focal packages. Comment #65 only mentions
versions for jammy. Have the focal packages been tested in any way?

Switching back tags for focal.

** Tags removed: verification-done verification-done-focal
** Tags added: verification-needed verification-needed-focal

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1961427

Title:
  zlib: compressBound() returns an incorrect result on z15

Status in Ubuntu on IBM z Systems:
  Fix Committed
Status in bedtools package in Ubuntu:
  Fix Released
Status in htslib package in Ubuntu:
  Invalid
Status in zlib package in Ubuntu:
  Fix Released
Status in bedtools source package in Focal:
  Invalid
Status in htslib source package in Focal:
  Fix Committed
Status in zlib source package in Focal:
  Fix Committed
Status in bedtools source package in Impish:
  Won't Fix
Status in htslib source package in Impish:
  Won't Fix
Status in zlib source package in Impish:
  Won't Fix
Status in bedtools source package in Jammy:
  Fix Released
Status in htslib source package in Jammy:
  Invalid
Status in zlib source package in Jammy:
  Fix Released

Bug description:
  SRU Justification:
  ==================

  [Impact]

  * zlib: compressBound() returns an incorrect result on IBM z15
  hardware.

  * Passing the result of compressBound() to compress() results
    in an error code.

  * This is because compressBound() is not adjusted for DFLTCC.

  [Fix]

  * Adjust compressBound() for DFLTCC like it's already done
    for deflateBound().

  * Since zlib project does not accept patches at the moment,
    the fix has been integrated into the DFLTCC pull request:
    https://github.com/madler/zlib/pull/410
    The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  * The fix extracted out of the above is:
    https://launchpadlibrarian.net/589857296/debdiff_zlib_1.2.11.dfsg-2ubuntu7_to_zlib_1.2.11.dfsg-2ubuntu8_jammy.diff

  * On top of this actual zlib fix, there is another patch needed:
   'Remove compressBound assertions. (PR #1258)' for htslib.

  * But there is a standalone 'htslib' package version, as well as a
    htslib version included in (some) 'bedtools' packages.
    Both need to be patched (see '[Other]' for more details).

  [Test Plan]

  * An IBM z15 system (LPAR, z/VM guest or KVM virtual machine)
    with Ubuntu Server 21.10 (or 22.04).

  * A test can be done  based on the following C test program:
        #include <assert.h>
        #include <stdlib.h>
        #include <zlib.h>
        int main() {
            Bytef in_buf[128], out_buf[1024];
            for (size_t i = 0; i < sizeof(in_buf); i++)
                in_buf[i] = rand();
            uLongf dest_len = compressBound(sizeof(in_buf));
            assert(dest_len <= sizeof(out_buf));
            int ret = compress(out_buf, &dest_len,
                               in_buf, sizeof(in_buf));
            assert(ret == Z_OK);
            }

  * The test needs to be done by IBM, due to the requirements
    for the special z15 hardware.

  * A successful test was just completed, based on the version in jammy-
  proposed, which is at the same code level that the impish version this
  SRU is targeted for.

  [Where problems could occur]

  * If the adjustment of compressBound() for DFLTCC is done
    erroneously the issue can still be present or in worst case
    even affect Z systems other than z15 only.

  * The compression can become errorneous with the new changes,
    e.g. in compressBound.

  * Mistakes in dfltcc_free_window OF and especially DEFLATE_BOUND_COMPLEN,
    (incl. the bit definitions), may cause various and unforseen defects.

  * Any build time issues that might have been introduced by this patch
    can be identified by a test build; this was done and is available here:
    https://launchpad.net/~fheimes/+archive/ubuntu/lp1961427

  [Other Info]

  * Ubuntu jammy, impish and focal are affected by the zlib issue.

  * The 'htslib' version '1.13+ds' (as it is part of I, J and K),
    already includes the patch,
    hence only htslib '1.10.2' in focal needs to be patched.

  * The bedtools version '2.30.0+dfsg' (as it is part of I, J and K),
    requires the patch,
    but version '2.27.1+dfsg' bedtools in focal does not incl. an
    embedded htslib, hence does not need to be (actually can't be)
    patched.

  * Patched version of the affected htslib and bedtools packages
    are build and also available at this PPA:
    https://launchpad.net/~fheimes/+archive/ubuntu/lp1961427  

  __________

  Description:   zlib: compressBound() returns an incorrect result on z15
  Symptom:       Passing the result of compressBound() to compress()
                 results in an error code.
  Problem:       compressBound() is not adjusted for DFLTCC.
  Solution:      Adjust compressBound() for DFLTCC like it's already done
                 for deflateBound(). Since zlib project does not accept
                 patches at the moment, the fix has been integrated into
                 the DFLTCC pull request:
                 https://github.com/madler/zlib/pull/410
                 The commitid is b25781e735363e04f6c56e21431c47e4afc50b17.

  Reproduction:  z15 only:
                 #include <assert.h>
                 #include <stdlib.h>
                 #include <zlib.h>
                 int main() {
                     Bytef in_buf[128], out_buf[1024];
                     for (size_t i = 0; i < sizeof(in_buf); i++)
                         in_buf[i] = rand();
                     uLongf dest_len = compressBound(sizeof(in_buf));
                     assert(dest_len <= sizeof(out_buf));
                     int ret = compress(out_buf, &dest_len,
                                        in_buf, sizeof(in_buf));
                     assert(ret == Z_OK);
                 }

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1961427/+subscriptions