Opening backports pre-release

[Evan Broder]

On Wed, Nov 2, 2011 at 2:11 PM, Evan Broder <evan at ebroder.net> wrote:
> Greetings, Tech Board -
>  The backports team has been playing with a proposal for about a year
> now that we'd like to finally submit to you folks.
>
> Currently, during the period between FeatureFreeze and release (about
> 1/3 of the cycle!), it's impossible to upload either new packages or
> new features to Ubuntu (freeze exceptions excluded). Post-release,
> this is an issue already addressed by the backports process. We'd like
> to propose extending the use of backports by opening the backports
> pocket for uploads and Debian syncs at FeatureFreeze, and then copying
> the contents of the backports pocket to the x+1 release when it opens,
> similar to what we do already for SRUs.
>
> We would still enforce the current backports requirements (the package
> must "build, install, and run", any reverse-dependencies must still
> work with the new package, and all uploads must be approved by a
> backporter). These requirements are stronger than those for general
> archive uploads, so we don't believe that there is significant risk to
> post-release archive quality of x+1.
>
> I see a handful of potential issues with this, and I only have answers
> for some of them, so if the TB is generally interested in the idea,
> I'd certainly appreciate feedback and suggestions:
>
>  - Skew between backports (and therefore x+1) and the main archive:
> bugfixes in release not making it into backports, versions in release
> superseding those in backports, etc. In general, I think we would want
> a mechanism to invalidate packages in the backports pocket whenever
> that package is uploaded to the release pocket, and possibly to the
> proposed pocket, though I don't know what that is.
>
>  - Archive admin workload. All backports currently require some action
> from archive admins. For no-source-change uploads, an archive admin
> runs a script on the archive master. For source-change uploads,
> because they are uploaded post-release, they are automatically put
> into the unapproved queue. Additionally, new packages have to go
> through sourceNEW and binNEW. Our proposal would create additional
> work for the archive admins near release time, which may be
> problematic.
>
>  - Upload privileges. I believe that currently anybody on ~ubuntu-dev
> can upload any package to the backports pocket, and backporters can
> approve backports of packages they otherwise would not be able to
> upload. The backports team would prefer to maintain that - would it be
> sufficient to ensure that manual backports uploads always go through
> unapproved?
>
>  - Component isolation. All packages in the backports pocket build
> with all components enabled. This means that if we copy binaries to
> x+1, we could be copying binaries built against packages from the
> wrong component. It might make sense to only pocket copy source
> packages into the x+1 release, instead of all packages.
>
> If the TB approves this, the backports team would gladly take
> responsibility for proposing patches to Launchpad, ubuntu-dev-tools,
> ubuntu-archive-tools, and anything else we've forgotten that needs
> modification to support the new workflow.
>
> I know this is rather late notice, but I'll go ahead and add this to
> tomorrow's meeting agenda in the hopes that there's time to discuss
> it. I'm also generally available for in-person or IRC discussion.

Hello everybody -
   We discussed this during today's TB meeting. Thanks to Stephane,
Colin, and Martin for their input on our ideas. I believe that this
represents the changes to my original proposal that were suggested
today:

 * In order to be sure that the packages in -backports remain in sync
with -release, the backports team will develop a bot to file bugs
against the backports project when a package already in -backports is
uploaded to a non-backports pocket. For now, the backports team will
be responsible for merging these changes when necessary (though we
would obviously welcome collaboration from whoever actually did the
upload)

 * Souce-change uploads to the -backports pocket will continue to land
in the UNAPPROVED queue.

 * Privileges to upload source-change backports will be changed to
match the rest of the archive, as opposed to the current configuration
where any member of MOTU or Core Dev can upload any package. However,
all uploads would still require approval from ubuntu-backporters.

 * When the next release is opened for development, pre-release
backports will be re-uploaded to the new release with their version
number bumped for a rebuild (i.e. there will be no direct source or
binary pocket copies)

 * Somewhat tangentially, Launchpad should ideally be modified to
allow sufficient granularity of queue control that the backports team
could be given permission to accept packages from the UNAPPROVED queue
for backports pocket uploads. Bug #648611 is already scheduled for
escalation due to release team's requirements, but I've also mentioned
the backports use case on that bug.

- Evan