MRE request for Squid

[Athos Ribeiro]

On Mon, Jan 30, 2023 at 03:07:29PM -0800, Steve Langasek wrote:

Hi,

>On Fri, Jan 27, 2023 at 12:17:04AM -0300, Athos Ribeiro wrote:
>> On Tue, Jan 24, 2023 at 07:43:57PM -0800, Steve Langasek wrote:
>
>> > On Tue, Jan 24, 2023 at 12:38:05PM -0300, Athos Ribeiro wrote:
>
>> > > I would like to request a Micro Release Exception for the Squid
>> > > package in Ubuntu Jammy and Focal. I put together a wiki page
>> > > describing everything I judged pertinent to the MRE here:
>
>> > > https://wiki.ubuntu.com/SquidUpdates
>
>> > > As described in the wiki page above, we recently confirmed with the
>> > > upstream project that the package is fit for MREs at
>> > > http://lists.squid-cache.org/pipermail/squid-users/2023-January/025586.html
>
>> > > While Squid 4 is no longer listed as stable in the upstream project, we
>> > > can still benefit from this MRE for focal to make sure we ship the
>> > > latest 4.x release available.
>
>> > Please explain what this means for it to longer be listed as stable, if
>> > releases are still happening and you want them to be included under this
>> > MRE.
>
>> As described in
>> http://www.squid-cache.org/mail-archive/squid-users/201312/0093.html, a
>> squid major release (N) official upstream support ends when the first
>> stable release for the next major version (N+1) is released. Whilie this
>> means that Squid 4.x reached its EOL, as announced in
>> http://lists.squid-cache.org/pipermail/squid-announce/2021-August/000135.html,
>> Historically, this does not seem to mean a hard commitment that no other
>> new 4.x versions will ever be released after the EOL announcement
>> (http://lists.squid-cache.org/pipermail/squid-announce/2021-October/000137.html
>> shows that 4.17 was released a couple months after the EOL
>> announcement).
>
>> Including 4.x under this MRE would allow us to at least evaluate picking
>> up 4.17 under this exception. Although we could do this in a separate,
>> single exception bug for focal and keep this exception request just
>> applicable for Jammy if the SRU team sees it is more suitebla in this
>> case.
>
>Ok.

On a second thought, I am removing the request for focal here. Its build
process does not run the test suite as the 5.x (>= 22.04) packages,
which would make the test plans and acceptance criteria different.
Therefore I'd like to proceed with the request for series caring squid
version >= 5.x only (i.e. >= 22.04).

Moreover, I reworded the request to explicitly include the interim
releases to comply with the SRU processes. For that, I replaced the
mentions to LTS releases with entries to stable releases, included
kinetic in the request, and mentioned that new stable releases will also
be under the MRE as long as the Squid release policy is not changed
regarding the formats and commitments described in the document.

>> > > It may also be worth mentioning that the following bugs/discussions led
>> > > us to push this MRE request forward:
>> >
>> > > - https://bugs.launchpad.net/ubuntu/+source/squid/+bug/1975399
>> > > - https://bugs.launchpad.net/ubuntu/jammy/+source/squid/+bug/1989380
>> > > - https://lists.ubuntu.com/archives/ubuntu-server/2022-June/009293.html
>> >
>> > > Thank you in advance for considering this request, and please let me
>> > > know if you need more information.
>> >
>> > The upstream testing is described as:
>> >
>> >  Squid contains an extensive testsuite that is executed during the Ubuntu
>> >  package build time on all supported architectures.
>> >
>> > That seems like a pretty boilerplate description that could be applied to
>> > many upstream projects, but doesn't really tell us whether we should have
>> > confidence in the upstream testsuite.
>> >
>> > What kind of tests are included?  Many build-time testsuites include only
>> > unit tests.  Are there integration tests as well?  (Important for a server)
>> > Are there any relevant statistics about unit test coverage?
>>
>> The downstream test suite (autopkgtest) does contain simple integration
>> tests to ensure the server runs, listens and responds properly.
>>
>> The upstream tests are composed of unit tests checking isolated
>> behaviors and performing code checks. While the test suite is available
>> and the code base is (apparently) extensive, I could not find any
>> benchmarks or coverage data for that.
>
>I think this information should be included in the MRE wiki page for
>reference.

+1, this information was included in the document.
>
>> >  TODO-B: list each non passing test, explain why that is ok in this case
>> >
>> > Why would there be any non-passing tests?  Isn't it a build failure if there
>> > are failing tests?
>> >
>> > If the package build doesn't fail on a test failure, then I don't consider
>> > that an adequate test plan for an MRE.
>
>> Currently, the upstream tests ran during build time will not block the package
>> build upon failures. Only the DEP8 test failures will.
>
>> I agree such failures should halt the build to be part of this test plan
>> here. This will require additional changes in a first SRU so failures
>> will block the builds in these cases. I will start with lunar now.
>
>> Would staging such changes (by usung block-proposed tags) in the stable
>> releases be enough here? Then, the first MRE uploads (or any other SRUs
>> that may land before those) should pick those test changes up and any
>> new SRUs would be subject to the "new build time test rules".
>
>Yes, that would be ok by me.

I fixed the issue in Lunar and am working on SRUing it to Jammy and
Kinetic (with a block-proposed tag).
https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2004050

Finally, I updated the document at https://wiki.ubuntu.com/SquidUpdates.

Thank you.

-- 
Athos Ribeiro