Staging area for hardy-proposed ?

[Jordan Mantha]

On Wed, Jun 4, 2008 at 6:48 PM, Matt Zimmerman <mdz at ubuntu.com> wrote:

> On Wed, Jun 04, 2008 at 09:43:27PM -0400, Scott Kitterman wrote:
> > On Wednesday 04 June 2008 21:22, Matt Zimmerman wrote:
> > > On Wed, Jun 04, 2008 at 04:32:15PM -0400, Scott Kitterman wrote:
> > > > Is running with -proposed activated all the time a supported
> > > > configuration?  I think -proposed is suitable for package testing,
> but a
> > > > bit risky for everyday use.
> > >
> > > Because it is valuable to the project for packages in -proposed to be
> > > tested as widely as possible, I think it is worthwhile to maintain a
> low
> > > barrier to entry if possible.
> > >
> > Unless one knows that one is testing a particular package, what fix to
> verify,
> > and to be on the lookout for regressions in that package, I think the
> > marginal utility of additional people running with the package is very
> low.
> >
> > In general, I think testers should test (and really need to be aware of
> what
> > they are testing) and users should use the output of the test process
> > (i.e. -updates in this case).
>
> I don't agree.  It's very much the same principle which applies to people
> running our development branches.  In the course of using the system
> normally, things get tested, and regressions are found and reported.
>

Except we normally don't want packages in -proposed for more than a week or
so whereas the development branch is tested by a lot more users for a lot
more time.

I agree that this process doesn't provide for verification of fixes, but it
> does provide regression testing.
>

I seriously question the usefulness of -proposed for regression testing. We
market the development branch to people who are willing and able to test. We
give them lots of time to do so. The -proposed repository appears to general
users as sort of smashup between -updates and -backports. "Get the updates
before they're in -updates!" However, people don't actually do real
regression testing of -updates packages (users assume they're already
tested) so I don't see much of a motivation for them to do so with
-proposed. You're just banking on people stumbling upon regressions which
seems much less efficient than having fewer people doing targeted testing.

I view -proposed as a convenient pool to hold packages that need testing. I
don't feel like we should just recommend users run their production machines
with -proposed enabled, but rather implement a SRU testing tracker (using
the iso testing infrastructure) to make it easy for people to install, find
the test case, confirm fixes and report regressions. Most SRUs don't require
multiple packages to be installed, but for sets like gnome, openoffice,
kernel a PPA for staging could be used from a testing tracker. SRU bugs have
a lot of pre-upload comment and work that can often make reading the bug
report tedious and confusing. Tracking testing separately from the bug
report would help make it easier for people to contribute to the testing
efforts.

-Jordan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20080605/0dbf96fb/attachment-0001.htm