Call for discussion: Migrating from interdiff to diff.gz for new upstream sponsoring

[Emmet Hikory]

Fellow developers,
    I'd like to revisit the migration of the preferred sponsoring
process for new upstream versions from the presentation of interdiff
files to the presentation of diff.gz files, last discussed in the MOTU
Meeting of 21st December.  I present to this list, rather than
ubuntu-motu@ as changes to the universe processes have been shown to
be affecting main, and I suspect that wider discussion will lead to
process convergence.

Overview:

    The current documentation on the wiki indicates that the uploading
of a compressed interdiff is the preferred method of requesting
sponsorship for a new upstream version of a package (where that
version is not yet in Debian).  This proposal suggests that the
presentation of the new candidate diff.gz is sufficient for
reconstructing the candidate new upstream, and should be used in
preference to interdiff for new upstream versions.  No change is
proposed to the requested information for package updates of the same
upstream version or merges from Debian.  This mail is intended to
provoke discussion about this issue.

History,

    During the feisty cycle (and even late edgy), there was a fair
amount of discussion regarding sponsoring processes, and the sponsor
queues became the standard means by which those without upload
permission to a given component would request upload.  At that time,
the apparent consensus in #ubuntu-motu was that interdiff was a useful
tool to present the differences of a new upstream version ahead of a
version in Debian, the process then consisting of attaching an
interdiff to a bug report, and uploading the candidate package to
REVU, or some similar repository from which dget would retrieve the
package.

    During the gutsy cycle, this was documented in the MOTU
Contributing wiki page (1), and was used by a growing number of
contributors, especially when preparing new candidates for universe
inclusion.  There was also significant use of REVU for this, and a
number of complaints about tracking the status of candidates in both
REVU and Malone.  In some cases, the candidates were ignored, and in
others there were colliding uploads.  After several different models,
it was agreed in a succession of MOTU meetings, ending in the hardy
that new candidates for universe would be handled as follows:

NEW packages: all to be uploaded to REVU.  status to be tracked in
REVU.  Malone placeholder bug in place to handle assignment of the
packager.  A specification (2) was started with initial review at UDS
Boston to further reduce confusion in the future.  The REVU sorting
algorithm was adjusted to better focus on new packages needing review.

new upstream candidates: All methods other than interdiff were to be
deprecated, in favor of consistency.  Documentation (3) was added to
the wiki explaining both the processes for generating and testing an
interdiff to be presented for review, and constructing a candidate
locally for review and sponsoring.

bugfixes, patches, etc: All such candidates were to be handled as
debdiffs uploaded to bug reports (with occasional exceptions for Vcs
branches for Vcs-managed ubuntu-local packages).

Issues with the current state of affairs:

a) Despite the documentation, Interdiff remains confusing to some
people.  For constructors, this is often related to confusion over
whether "-p1" should or should not be included as an argument.  For
reviewers, this is often related to unfamiliarity with the process of
constructing a new target candidate.

b) As contributors who have been working in universe for some time
begin to submit candidates to main, these submissions tend to follow
the processes in place in universe.  As the use of interdiffs was
never discussed in depth with the main sponsors, this has led to some
confusion on the part of main sponsors as why they are receiving an
interdiff that is not human-readable.

c) When constructed according to the documentation, the presented
interdiff.gz file contains the complete contents of both the new and
old diff.gz files, and so is both larger and harder to use than the
new candidate diff.gz alone.

Proposal for discussion:
    Rather than requesting the submission of an interdiff, sponsors
should request (and expect) the presentation of a diff.gz file.

Benefits:

a) Easier for contributors: as the diff.gz is a natural result of
preparing a new candidate, and requires no additional manipulation
b) Easier for sponsors, as there is likely to be less difficulty
constructing a new package with a diff.gz than from the interdiff (in
the simple case, untar upstream, apply the diff, build).
c) Less space used by attachments in Malone
d) Less confusion by main sponsors who may not be familiar with the
processes in use by the universe sponsors

Detriments:

a) Increased process change (especially after it was finally all settled)
b) Those familiar with the current process have to learn the new
process, and may need to revise local scripts or aliases

Note:

    This is a call for discussion.  Please share your opinion, and any
additional thoughts.  My intent in drafting this is to seek a process
for new upstream candidates that meets the goals of all of
contributors, universe sponsors, main sponsors, Malone administrators,
and QA initiatives.  The use of diff.gz may not be the best solution,
and counter proposals would be as welcome as support.

1: https://wiki.ubuntu.com/MOTU/Contributing
2: https://blueprints.launchpad.net/ubuntu/+spec/review-process-convergence
3; https://wiki.ubuntu.com/UbuntuDevelopment/Interdiff

-- 
Emmet HIKORY