Call for discussion: Migrating from interdiff to diff.gz for new upstream sponsoring

[Scott Kitterman]

On Wednesday 09 January 2008 07:47, Emmet Hikory wrote:
> Fellow developers,
>     I'd like to revisit the migration of the preferred sponsoring
> process for new upstream versions from the presentation of interdiff
> files to the presentation of diff.gz files, last discussed in the MOTU
> Meeting of 21st December.  I present to this list, rather than
> ubuntu-motu@ as changes to the universe processes have been shown to
> be affecting main, and I suspect that wider discussion will lead to
> process convergence.
>
> Overview:
>
>     The current documentation on the wiki indicates that the uploading
> of a compressed interdiff is the preferred method of requesting
> sponsorship for a new upstream version of a package (where that
> version is not yet in Debian).  This proposal suggests that the
> presentation of the new candidate diff.gz is sufficient for
> reconstructing the candidate new upstream, and should be used in
> preference to interdiff for new upstream versions.  No change is
> proposed to the requested information for package updates of the same
> upstream version or merges from Debian.  This mail is intended to
> provoke discussion about this issue.
>
> History,
>
>     During the feisty cycle (and even late edgy), there was a fair
> amount of discussion regarding sponsoring processes, and the sponsor
> queues became the standard means by which those without upload
> permission to a given component would request upload.  At that time,
> the apparent consensus in #ubuntu-motu was that interdiff was a useful
> tool to present the differences of a new upstream version ahead of a
> version in Debian, the process then consisting of attaching an
> interdiff to a bug report, and uploading the candidate package to
> REVU, or some similar repository from which dget would retrieve the
> package.
>
>     During the gutsy cycle, this was documented in the MOTU
> Contributing wiki page (1), and was used by a growing number of
> contributors, especially when preparing new candidates for universe
> inclusion.  There was also significant use of REVU for this, and a
> number of complaints about tracking the status of candidates in both
> REVU and Malone.  In some cases, the candidates were ignored, and in
> others there were colliding uploads.  After several different models,
> it was agreed in a succession of MOTU meetings, ending in the hardy
> that new candidates for universe would be handled as follows:
>
> NEW packages: all to be uploaded to REVU.  status to be tracked in
> REVU.  Malone placeholder bug in place to handle assignment of the
> packager.  A specification (2) was started with initial review at UDS
> Boston to further reduce confusion in the future.  The REVU sorting
> algorithm was adjusted to better focus on new packages needing review.
>
> new upstream candidates: All methods other than interdiff were to be
> deprecated, in favor of consistency.  Documentation (3) was added to
> the wiki explaining both the processes for generating and testing an
> interdiff to be presented for review, and constructing a candidate
> locally for review and sponsoring.
>
> bugfixes, patches, etc: All such candidates were to be handled as
> debdiffs uploaded to bug reports (with occasional exceptions for Vcs
> branches for Vcs-managed ubuntu-local packages).
>
> Issues with the current state of affairs:
>
> a) Despite the documentation, Interdiff remains confusing to some
> people.  For constructors, this is often related to confusion over
> whether "-p1" should or should not be included as an argument.  For
> reviewers, this is often related to unfamiliarity with the process of
> constructing a new target candidate.
>
> b) As contributors who have been working in universe for some time
> begin to submit candidates to main, these submissions tend to follow
> the processes in place in universe.  As the use of interdiffs was
> never discussed in depth with the main sponsors, this has led to some
> confusion on the part of main sponsors as why they are receiving an
> interdiff that is not human-readable.
>
> c) When constructed according to the documentation, the presented
> interdiff.gz file contains the complete contents of both the new and
> old diff.gz files, and so is both larger and harder to use than the
> new candidate diff.gz alone.
>
> Proposal for discussion:
>     Rather than requesting the submission of an interdiff, sponsors
> should request (and expect) the presentation of a diff.gz file.
>
> Benefits:
>
> a) Easier for contributors: as the diff.gz is a natural result of
> preparing a new candidate, and requires no additional manipulation
> b) Easier for sponsors, as there is likely to be less difficulty
> constructing a new package with a diff.gz than from the interdiff (in
> the simple case, untar upstream, apply the diff, build).
> c) Less space used by attachments in Malone
> d) Less confusion by main sponsors who may not be familiar with the
> processes in use by the universe sponsors
>
> Detriments:
>
> a) Increased process change (especially after it was finally all settled)
> b) Those familiar with the current process have to learn the new
> process, and may need to revise local scripts or aliases
>
> Note:
>
>     This is a call for discussion.  Please share your opinion, and any
> additional thoughts.  My intent in drafting this is to seek a process
> for new upstream candidates that meets the goals of all of
> contributors, universe sponsors, main sponsors, Malone administrators,
> and QA initiatives.  The use of diff.gz may not be the best solution,
> and counter proposals would be as welcome as support.
>
> 1: https://wiki.ubuntu.com/MOTU/Contributing
> 2: https://blueprints.launchpad.net/ubuntu/+spec/review-process-convergence
> 3; https://wiki.ubuntu.com/UbuntuDevelopment/Interdiff

I like diff.gz better than interdiff.  

Perhaps a diff of the debian/dirs would be even easier (Changes outside the 
debian dir that need to be maintained should be patched).  Then the 
sponsorship work flow would be something like:

1.  Get the current source package.
2.  Get the new upstream based on the watch file (uscan)
3.  Review the provided 'patch' to the debian dir
4.  Apply the patch
5.  Build, Test, Etc.

Anythiing in diff.gz outside the debian dir that needs to be preserved from 
release to release should be dealt with as a special case.

Scott K