de-crufting of -proposed [Was, Re: Proposed pocket racing uninstallability and SRU verification around release time]

[Steve Langasek]

On Thu, Oct 20, 2016 at 02:32:45PM +0200, Dimitri John Ledkov wrote:

> We literarly have devel-proposed littered with things, and on release
> day people do see those packages even though it has never been
> intented to release them.

If people are "seeing" them on release day, that is still a bug in the
client configuration, not in the contents of -proposed.

> For example:
> https://launchpad.net/ubuntu/+source/jitsi/2.4.4997-1.2ubuntu1/+publishinghistory

> Has been stuck in -proposed for 848 days old and it has not met
> release criteria since utopic.

> Yet, it was published in yakkety-proposed until yesterday, at which
> point it was finally deleted from yakkety-proposed as zesty-proposed
> is initialised.
> However, duing that time it was not an SRU but a broken devel package
> that should be stuck in devel.

> My argument is that we should just remove these packages that have not
> migrated for years. Others in release team say they don't hurt
> anything. Yet, they are visible in a stable-proposed when they should
> not be. Without launchpad engineering, we could be moving those out of
> $stable-proposed into a ppa between release day and next series
> initialisation.

For the record, I don't say that they don't hurt anything.  I consider cruft
hanging out in devel-proposed a problem, because it means people waste time
trying to find the signal in
<http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html>. 
I have heard more than once a recommendation that people find things to work
on on this page by scrolling to "somewhere in the middle".  But we shouldn't
have to tell them that; we should drive the cruft down to zero, so that it's
possible to work from the bottom.

However, removing all of that cruft in an appropriate way takes time on a
per-package basis.  In the example of jitsi, first of all, it's not true
that it has been "stuck in -proposed for 848 days"; unfortunately
proposed-migration sorts by upload date, and so if a package is in the
release pocket, and is then demoted to -proposed, it will sort to the
bottom.  You'll see that the publishing history shows that jitsi *was* in
the release pocket, during utopic, and then was demoted because of a libav
transition.  (Ok, that means it's actually been in -proposed for 776 days
instead of 848 days, kind of an extreme case all the same...) I've argued
that wherever sensible, we should remove packages instead of just demoting
them to -proposed.  I believe the archive team has been much more consistent
about this, at least since around March of this year, so there should be
much less cruft flowing in this way.

But the historical cruft, we still haven't gotten rid of.  And I say that we
should remove packages "whenever sensible".  There are two basic classes
where it's not obviously sensible to remove instead of demote:

 - the package has an Ubuntu delta; removing it will cause the package to be
   re-synced from Debian (now or in the future), and the synced version
   might make its way back into the devel pocket but be buggy in a way
   that's not acceptable to the Ubuntu community.

 - the package was removed as uninstallable from the release pocket because
   one of its dependencies had to be removed, but that dependency could
   theoretically become installable again in the future (e.g. we still have
   the source around, but it currently FTBFS) in which case the package
   should be allowed back into the release pocket - but won't be if we
   make it disappear now.

If each of these has to wait for an archive admin to manually review and
assess the impact of removal, we're going to be a bottleneck.  But both
classes of problem can be addressed upstream in Debian by anyone:

 - Get the Ubuntu delta upstreamed, so the package can be in sync.  If it's
   in sync and still uninstallable / unbuildable, the package can be removed
   from -proposed and we can wait for a fixed package from Debian.

 - If the package is broken in Debian and not fixable / being fixed, help
   Debian get it removed from their archive.  The removal will then
   propagate to -proposed.


In the specific instance of jitsi, the delta isn't one we care about - it's
a build fix for -Wl,--as-needed.  Maybe this is still needed, maybe it's
fixed upstream, I don't know; but either way, the correct fix is for us to
take the new version from Debian (as a merge or as a sync) and see what
happens.  If the new version is still unbuildable in Ubuntu, we can then
drop it from -proposed and wait for a fixed version to arrive from
Debian[1].

I've force-synced the new jitsi now, and it's waiting in the unapproved
queue for zesty to open.

Thanks,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org

[1] Though in the case of build failures caused by -Wl,--as-needed, I
wouldn't hold my breath.  Since Debian doesn't use this flag by default,
such problems are invisible to Debian maintainers unless we report them.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-release/attachments/20161020/8bc93301/attachment.pgp>