MRE request for Squid

[Athos Ribeiro]

On Tue, Jan 24, 2023 at 07:43:57PM -0800, Steve Langasek wrote:
>Hi Athos,

Hi Steve,

Thanks for the review.

>On Tue, Jan 24, 2023 at 12:38:05PM -0300, Athos Ribeiro wrote:
>
>> I would like to request a Micro Release Exception for the Squid
>> package in Ubuntu Jammy and Focal. I put together a wiki page
>> describing everything I judged pertinent to the MRE here:
>
>> https://wiki.ubuntu.com/SquidUpdates
>
>> As described in the wiki page above, we recently confirmed with the
>> upstream project that the package is fit for MREs at
>> http://lists.squid-cache.org/pipermail/squid-users/2023-January/025586.html
>
>> While Squid 4 is no longer listed as stable in the upstream project, we
>> can still benefit from this MRE for focal to make sure we ship the
>> latest 4.x release available.
>
>Please explain what this means for it to longer be listed as stable, if
>releases are still happening and you want them to be included under this
>MRE.

As described in
http://www.squid-cache.org/mail-archive/squid-users/201312/0093.html, a
squid major release (N) official upstream support ends when the first
stable release for the next major version (N+1) is released. Whilie this
means that Squid 4.x reached its EOL, as announced in
http://lists.squid-cache.org/pipermail/squid-announce/2021-August/000135.html,
Historically, this does not seem to mean a hard commitment that no other
new 4.x versions will ever be released after the EOL announcement
(http://lists.squid-cache.org/pipermail/squid-announce/2021-October/000137.html
shows that 4.17 was released a couple months after the EOL
announcement).

Including 4.x under this MRE would allow us to at least evaluate picking
up 4.17 under this exception. Although we could do this in a separate,
single exception bug for focal and keep this exception request just
applicable for Jammy if the SRU team sees it is more suitebla in this
case.

>
>> It may also be worth mentioning that the following bugs/discussions led
>> us to push this MRE request forward:
>
>> - https://bugs.launchpad.net/ubuntu/+source/squid/+bug/1975399
>> - https://bugs.launchpad.net/ubuntu/jammy/+source/squid/+bug/1989380
>> - https://lists.ubuntu.com/archives/ubuntu-server/2022-June/009293.html
>
>> Thank you in advance for considering this request, and please let me
>> know if you need more information.
>
>The upstream testing is described as:
>
>  Squid contains an extensive testsuite that is executed during the Ubuntu
>  package build time on all supported architectures.
>
>That seems like a pretty boilerplate description that could be applied to
>many upstream projects, but doesn't really tell us whether we should have
>confidence in the upstream testsuite.
>
>What kind of tests are included?  Many build-time testsuites include only
>unit tests.  Are there integration tests as well?  (Important for a server)
>Are there any relevant statistics about unit test coverage?

The downstream test suite (autopkgtest) does contain simple integration
tests to ensure the server runs, listens and responds properly.

The upstream tests are composed of unit tests checking isolated
behaviors and performing code checks. While the test suite is available
and the code base is (apparently) extensive, I could not find any
benchmarks or coverage data for that.

>  TODO-B: list each non passing test, explain why that is ok in this case
>
>Why would there be any non-passing tests?  Isn't it a build failure if there
>are failing tests?
>
>If the package build doesn't fail on a test failure, then I don't consider
>that an adequate test plan for an MRE.

Currently, the upstream tests ran during build time will not block the package
build upon failures. Only the DEP8 test failures will.

I agree such failures should halt the build to be part of this test plan
here. This will require additional changes in a first SRU so failures
will block the builds in these cases. I will start with lunar now.

Would staging such changes (by usung block-proposed tags) in the stable
releases be enough here? Then, the first MRE uploads (or any other SRUs
that may land before those) should pick those test changes up and any
new SRUs would be subject to the "new build time test rules".

Thoughts?

-- 
Athos Ribeiro