Getting stuff backported is too hard

[Scott Kitterman]

On Saturday, July 31, 2010 05:30:04 pm Iain Lane wrote:
> Hiya,
> 
> Thanks for replying.
> 
> On Sat, Jul 31, 2010 at 10:30:58AM -0400, Scott Kitterman wrote:
> >On Saturday, July 31, 2010 06:55:55 am Iain Lane wrote:
> >> Hello all,
> >> 
> >> [...]
> >> 
> >>    1) File a bug report, do testing, prepare a new source package if
> >>    necessary.
> >>    2) Wait for backporting testers to test your package
> >>    3) Eventually a backporter (member of the ~ubuntu-backporters team)
> >>    will come along and bless your request
> >>    4) The package is uploaded
> >> 
> >> I've found that in practice (2) doesn't happen, and the backporters
> >> team often takes a very long time to get around to reviewing
> >> "confirmed" bugs.
> >
> >Backports testers is essentially dead.  If you put in the bug that the
> >package builds, installs, and runs when you file it, you can skip step 2.
> > There is no requirement for it to be someone else doing the testing.
> >
> >#3 can take a long time because there aren't very active
> >~ubunt-backporters at the moment (volunteers welcome).  If you do have
> >one you want done, feel free to ping me on IRC to review it.  That will
> >almost always get you approval within a few hours.
> 
> Yeah, I'm aware of this. I'm generally loathe to ping people for
> requests that aren't urgent though. Backports generally fall into the
> “not urgent” category in my book. And a process which requires this is
> a process which needs fixing somehow. Maybe that somehow is more manpower.

I think more manpower is the main problem.  I've had one offlist volunteer and 
I, for one, would welcome you to get involved.

I do think that there are special considerations associated with approving a 
backport that make it not something everyone should do.  They aren't that 
hard, but they do require a bit off additional familiarity.  We went through a 
phase a couple of years ago where there were cases of archive admins bypassing 
the backports process and doing their own backports.  Some of these did not go 
well.  Every archive admin is a very experienced developer, but that doesn't 
guarantee they will consider all aspects of backports requirements without 
some focus on what should be approved and what should not.  I think the same 
is even more true of less experienced developers.

> >> In contrast, Debian's process[1] is much more lightweight:
> >>    1) Build and upload a package with minimal changes from the version
> >>    you are backporting
> >> 
> >> Now, obviously there are differences between the two distros, but I
> >> wonder if we can make our process more lightweight. I don't propose
> >> changing the requirements for package eligibility[2], but instead the
> >> technical hurdles.
> >
> >One of the biggest differences is that backports.org is an external third
> >party service and not an official part of Debian.  Ubuntu backports
> >started out this way, but was integrated a long time ago.
> 
> bpo is planned to become an official Debian service IIRC. I can't find
> any link to back this up, but I'm fairly confident it's true. Not sure
> how/if the procedure will change though.
> 
There are a number of important differences between Debian and Ubuntu backports 
that make them (in my opinion) not very comparable.

1.  Debian backports are already set up so that you have to explicitly request 
a package from backports, enabling the backports repository doesn't install 
everything from backports.

2.  Debian backports is only for packages that need to be recompiled to work 
on stable.  Users are supposed to get packages from Testing that don't need to 
be recompiled for Stable via partial upgrades.

3.  Debian backports does not require reverse dependencies to be checked or 
updated if a backport breaks them.

I don't think any of this will change when bpo becomes "official".

> > [...]
> > 
> >> I'd like some feedback on whether people feel that there is a problem,
> >> feel that backports should be used more and on my proposed
> >> process.
> >
> >This is almost exactly the current process for sourceful backports that
> >require changes.
> >
> >Unlike SRU, backports are usually no different (except a new
> >debian/changelog entry) than the current package in the development
> >release.  Also unlike SRU, ~ubuntu-backporters don't review the diff from
> >the package in the target release, they review the diff from the
> >development release.  So, in most cases, there would be nothing to reivew
> >in the queue.
> 
> That's a good, and interesting, point. I wonder if ubuntu-backporters
> review is required at all for no-change backports. Do you think that
> there would be a big problem were developers allowed to request
> backports themselves without further review? I don't know if this
> extends to source-change backports too: would have to think on it some
> more. Something like this, for the no-change case:

I do think even no change backports need review.  It is not at all rare for 
people to request backports of packages with reverse dependencies without 
understanding the implications for them.  In cases where a backport goes bad 
(they are rare, but they do happen - see the subversion 1.5 backport for Hardy 
- I allowed it to go forward after testing a selection of rdepends, not all of 
them and we selected poorly), I think it's important to have someone backports 
focused involved in the process to get it fixed.

My experience is that people who want backports tend to be focused on a 
relatively narrow case and aren't that interested in expending a lot of effort 
on the broader implications to the archive of what they are requesting.

The review that's needed by ubuntu-backporters is not mostly about the code, 
it's about how disruptive the backport will be to users that run with 
backports enabled.  Because of the way backports currently works, we need to 
work under this assumption.

>    1) Developer builds package from development release on target
>    release and discovers it works without further changes
>    2) Developer files a request with ubuntu-archive subscribed to have
>    the package backported. This is very much akin to a sync request
>    3) AA processes and syncs into $target-backports.
> 
> This places more trust and responsibility in the hands of
> developers. I don't know how much review the backporters team does
> currently. Perhaps this could be codified in the policy. I suspect
> that most backports requested by developers are waved through anyway,
> so hopefully no further bad uploads will happen.

That hasn't been my experience.

> >As I see it, there are two problems blocking timely backports at the
> >moment:
> >
> >1.  No ~ubuntu- backporters are regularly reviewing the backports requests
> >(I do it now and then and when pinged, I don't have the impression anyone
> >else is more active).  This could be solved by volunteers (and is still
> >an problem if your proposal is adopted).  Any MOTU is eligible for this
> >role if they are interested.  Please contact me if anyone is interested.
> 
> I am interested (and have enquired before, but was rebuffed and then
> ignored so didn't push it very hard) — but I don't think the current
> process is a very good way for me to spend my time. If we can work
> something out then you can count on my help.
> 
> Backporting seems to be located out of the sphere of the rest of
> Ubuntu development. Was this deliberate when the current procedure was
> conceived? My proposal will integrate it somewhat. I don't know if a
> significant proportion of backporting requests are coming from
> non-developers currently. I think that, if developers are given a more
> integral role, the process will become more transparent and efficient
> — and therefore more satisfactory to users.

I didn't go look and count, but my perception is that most are from non-
developers.

I was not involved when it was conceived, so I can't tell you.  It has become 
more integrated and responsive than when I started.  It used to be that only 
core-dev could upload source backports, even for Universe packages.

When there is someone available to regularly review backports requests, I 
think the current process moves pretty quickly and is not very heavy.  It 
breaks down when that doesn't happen.

> >2.  Archive admins take a long time to process no change backports.  As a
> >non- Canonical archive admin, I can accept packages from unapproved, but
> >I can't run the backporting script for no-change backports.  It's my
> >impression that this is not routinely done by all archive admins on their
> >archive day.  I generally see these done by Riddell (who gets bonus
> >points in my book for also regularly processing sync requests) or jamesw
> >(I'd imagine others have been doing it, just not for backports I've
> >approved).  I have not been following details of the archive admin
> >workload, so I don't know what's behind this.
> 
> Right, this is something that could perhaps be raised (at a release
> meeting?) if it's really a problem. I haven't followed this myself so
> have no information to back the claim up.

It could be and I'd be glad to do it.  I'd want to know that we had people 
available to get the earlier stages of the process moving quickly first.

> >Developers uploading no change backports directly would allow me to accept
> >them and work around this problem.  As with sync requests from Debian, I
> >do believe it's generally better to move code directly from one archive
> >to another without having it pass through a developer's system.  There is
> >always some small marginal risk of error.
> 
> I agree: this should be avoided if possible. This is why I don't
> advocate the use of the new syncpackage script (although I have used
> it myself in the past for uploads that cannot wait for whatever
> reason). My modified proposal removes the need for this.
> 
> >I think your proposal of direct uploads would (to the extent I'm
> >available) give us a work around for being blocked at this stage.  If
> >there is a backport that is particularly urgent for some reason, I think
> >this is reasonable to do even under the current rules (it stretches them,
> >but personally, I don't mind).  I don't think we should make this the
> >default.  I think we should figure out why things are taking so long to
> >process and fix that.
> 
> Yep. I am trying to reduce the amount of blocking on other people that
> goes into the process as it stands, in order to make it more
> streamlined. I think this is where the biggest wins can be found.
> 
> This whole train of thought was set off by the amount of upstream
> backport PPAs that I'm seeing springing up. This points to a place
> where our distribution model isn't meeting the needs of upstream
> developers. I would like to think that -backports is a nice home for
> these things instead, but currently I wouldn't advocate it to an
> upstream as the process is just too clunky. Let's fix this. (I am
> aware that the proper fix also blocks on selective backports, but
> let's do what we can now)

I like the idea of encouraging use of backports for this.  I think I've lost 
track on what your current proposal is.  It would be helpful to me if you 
could restate what you would suggest based on the feedback so far.

There are many cases where PPA is the best solution because only users that 
want that particular package will get it.  That makes PPAs great for people 
who are focused on one package and want the latest when such packages might 
not be suitable for general use.

Getting the archive fixed to not automatically install from backports is an 
essential piece of getting this fixed.

Scott K