Getting stuff backported is too hard

[Iain Lane]

Hello,

On Sun, Aug 01, 2010 at 01:28:04AM -0400, Scott Kitterman wrote:
>On Saturday, July 31, 2010 05:30:04 pm Iain Lane wrote:
>> [...]
>> Yeah, I'm aware of this. I'm generally loathe to ping people for
>> requests that aren't urgent though. Backports generally fall into the
>> “not urgent” category in my book. And a process which requires this is
>> a process which needs fixing somehow. Maybe that somehow is more manpower.
>
>I think more manpower is the main problem.  I've had one offlist volunteer and
>I, for one, would welcome you to get involved.
>
>I do think that there are special considerations associated with approving a
>backport that make it not something everyone should do.  They aren't that
>hard, but they do require a bit off additional familiarity.  We went through a
>phase a couple of years ago where there were cases of archive admins bypassing
>the backports process and doing their own backports.  Some of these did not go
>well.  Every archive admin is a very experienced developer, but that doesn't
>guarantee they will consider all aspects of backports requirements without
>some focus on what should be approved and what should not.  I think the same
>is even more true of less experienced developers.

It's good to have input from someone who knows a lot about the
process, and is indeed worrying that there is past experience of
developers not being careful enough when uploading packages. I would
very much like to get to a place where we can trust our developers to
do such.

>> > [...]
>> >
>> >> I'd like some feedback on whether people feel that there is a problem,
>> >> feel that backports should be used more and on my proposed
>> >> process.
>> >
>> >This is almost exactly the current process for sourceful backports that
>> >require changes.
>> >
>> >Unlike SRU, backports are usually no different (except a new
>> >debian/changelog entry) than the current package in the development
>> >release.  Also unlike SRU, ~ubuntu-backporters don't review the diff from
>> >the package in the target release, they review the diff from the
>> >development release.  So, in most cases, there would be nothing to reivew
>> >in the queue.
>>
>> That's a good, and interesting, point. I wonder if ubuntu-backporters
>> review is required at all for no-change backports. Do you think that
>> there would be a big problem were developers allowed to request
>> backports themselves without further review? I don't know if this
>> extends to source-change backports too: would have to think on it some
>> more. Something like this, for the no-change case:
>
>I do think even no change backports need review.  It is not at all rare for
>people to request backports of packages with reverse dependencies without
>understanding the implications for them.  In cases where a backport goes bad
>(they are rare, but they do happen - see the subversion 1.5 backport for Hardy
>- I allowed it to go forward after testing a selection of rdepends, not all of
>them and we selected poorly), I think it's important to have someone backports
>focused involved in the process to get it fixed.
>
>My experience is that people who want backports tend to be focused on a
>relatively narrow case and aren't that interested in expending a lot of effort
>on the broader implications to the archive of what they are requesting.

This is one of the reasons why I wanted to insert developers more
centrally into the process — they should be those who can best judge
the impact of what they propose to upload. Mistakes will indeed
happen, but backports come with a lower quality expectation than SRUs,
so I believe that we can tolerate this so long as they are infrequent
and minor.

>The review that's needed by ubuntu-backporters is not mostly about the code,
>it's about how disruptive the backport will be to users that run with
>backports enabled.  Because of the way backports currently works, we need to
>work under this assumption.
>
>>    1) Developer builds package from development release on target
>>    release and discovers it works without further changes
>>    2) Developer files a request with ubuntu-archive subscribed to have
>>    the package backported. This is very much akin to a sync request
>>    3) AA processes and syncs into $target-backports.
>>
>> This places more trust and responsibility in the hands of
>> developers. I don't know how much review the backporters team does
>> currently. Perhaps this could be codified in the policy. I suspect
>> that most backports requested by developers are waved through anyway,
>> so hopefully no further bad uploads will happen.
>
>That hasn't been my experience.

OK

>> [...]
>> Backporting seems to be located out of the sphere of the rest of
>> Ubuntu development. Was this deliberate when the current procedure was
>> conceived? My proposal will integrate it somewhat. I don't know if a
>> significant proportion of backporting requests are coming from
>> non-developers currently. I think that, if developers are given a more
>> integral role, the process will become more transparent and efficient
>> — and therefore more satisfactory to users.
>
>I didn't go look and count, but my perception is that most are from non-
>developers.

We also get a lot of requests from non-developers for other packaging
tasks, for example new upstream versions. The current process does
require some technical skill at present as the proposed package must
be tested.

>> [...]
>> This whole train of thought was set off by the amount of upstream
>> backport PPAs that I'm seeing springing up. This points to a place
>> where our distribution model isn't meeting the needs of upstream
>> developers. I would like to think that -backports is a nice home for
>> these things instead, but currently I wouldn't advocate it to an
>> upstream as the process is just too clunky. Let's fix this. (I am
>> aware that the proper fix also blocks on selective backports, but
>> let's do what we can now)
>
>I like the idea of encouraging use of backports for this.  I think I've lost
>track on what your current proposal is.  It would be helpful to me if you
>could restate what you would suggest based on the feedback so far.

I would ideally like my most recent proposal of developers uploading
packages directly. But I am happy to defer to your experience of this
not being such a good idea. You raised a good point about most
backports being no-change, and I don't think I have a decent solution
for that, besides one which requires changes in LP:

   1) Developer requests backport or sponsors an existing backport
   request (through ~ubuntu-sponsors) by using a Launchpad interface to
   copy a package from the development release to $target-backports
   2) Package lands in unapproved and is reviewed by backporters from
   there

For source-change backports, packages are uploaded directly to
unapproved and reviewed by backporters as in 2). There is always an
associated bug report as for SRUs. There's an additional 

This is based on the assumption that unapproved is a good place for a
team to review packages. This assumption comes from my observations of
the improvements in Universe SRUs since uploading directly to
unapproved became the norm.

I'll file the LP bug report for this if others think this would be an
improvement. There should already be an internal interface for this as
I believe the AAs have a script which performs this task, albeit
without the unapproved step.

But for now I'm happy to join the team if you'd like me to. At least
then I'll be able to speak with experience when I suggest improvements.

>There are many cases where PPA is the best solution because only users that
>want that particular package will get it.  That makes PPAs great for people
>who are focused on one package and want the latest when such packages might
>not be suitable for general use.
>
>Getting the archive fixed to not automatically install from backports is an
>essential piece of getting this fixed.

It is, but we should always be concerned to not let the perfect be the
enemy of the good.

I'm keen not to repeatedly go over the same ground, but I think that
improving backports could be a good topic of discussion for the next
UDS. A link to the old spec would be useful for now.

Ta,
Iain
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20100801/d9fe90b9/attachment.pgp