Archive changes

[Colin Watson]

On Tue, Mar 15, 2016 at 11:15:16PM +0100, Joerg Jaspert wrote [on
debian-devel-announce]:
> I've just activated a few changes to the archive we talk(ed) about for a
> long time. And while it is not exactly the start of this release cycle,
> it should still work out nicely (so one hopes).
> 
> As of now, InRelease/Release files, Packages and Sources no longer
> provide MD5Sum and SHA1sums, only SHA256.
> 
> Additionally I turned off generating gzip compressed versions of those
> files, xz is there.
> 
> To test it, this is limited to experimental. We hope nothing breaks on it,
> but lets try for a few days. If that works out, we should adjust
> unstable, and another short time later coordinate with the release team
> to adjust testing, so it ends up in the next release.

This change has caused quite a bit of fallout, both in Debian and Ubuntu
(and quite probably elsewhere).  On the whole I approve of the direction
of the changes so haven't been lobbying to have them reversed, although
the timing is a little inconvenient!  The main thing Ubuntu developers
may have noticed is that Launchpad is currently failing to import source
packages from Debian, so you can't yet use syncpackage for packages
processed after the changes in the mail quoted above.  I've been working
hard on that in the latter half of this week.  The state of play is:

 * We use debmirror to mirror the bits of the Debian archive we need for
   the import.  This was broken by the removal of gzip and the removal
   of weaker checksums
   (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818479) but is now
   fixed in unstable and xenial, and also backported to the system where
   we run the import.

 * Having upgraded debmirror, we now find that it's doing slightly
   stricter signature checking, so it fails because our
   debian-archive-keyring package is old enough that it only has one of
   the keys used to sign current Debian suites.  I've requested a
   backport, which Canonical staff can track in
   https://portal.admin.canonical.com/89843.

 * The program that actually does the import will also break due to the
   removal of gzip and the removal of weaker checksums.  I've proposed a
   branch to fix this
   (https://code.launchpad.net/~cjwatson/launchpad/gina-stronger-checksums/+merge/289505),
   and we should be able to deploy something like this early next week.

People running xenial may also have noticed that apt is now complaining
on update about weak signatures on PPAs (and perhaps other archives too,
but we have no control over those).  There's a fix for this pending
deployment
(https://code.launchpad.net/~cjwatson/launchpad/digest-algo-sha512/+merge/289052,
which we might amend with
https://code.launchpad.net/~cjwatson/launchpad/digest-algo-sha384/+merge/289479)
which will at least fix the problem when the PPA in question are
republished.  We're also working on some changes to let us go through
and re-sign all existing PPAs, or at least those with xenial
publications.

It never rains but it pours; but with any luck this will be enough
catch-up work for a while once we're finished ...

-- 
Colin Watson                                       [cjwatson at ubuntu.com]