nvidia/fglrx expedited SRUs

[Scott Kitterman]

Bryce Harrington <bryce at canonical.com> wrote:

>On Mon, Sep 03, 2012 at 10:37:55PM +0200, Martin Pitt wrote:
>> Bryce Harrington [2012-08-31 10:47 -0700]:
>> > The mechanics of packaging a driver update are pretty much push
>button
>> > at this point.  Everything is scripted and well documented.  The
>> > ubuntu-x community is able to package new driver updates and get
>them
>> > into the xorg-edgers and x-updates PPAs within days of their
>release.
>> > Due to the level of QA done internally at NVIDIA, issues with
>released
>> > drivers are very unusual.
>> 
>> How much testing on the Ubuntu side do these drivers usually get?
>
>Historically, this is how testing has worked:
>
>1.  The driver gets posted to xorg-edgers PPA right away.  This is a
>    heavily subscribed PPA and users are not at all shy about raising
>    complaints if there are problems.
>
>2.  The update becomes available in x-updates PPA where a larger, more
>    conservative-minded population is exposed to it.
>
>3.  A member of ubuntu-x will boot and smoke-test the driver.  By this
>    point we generally know from the above testing that there aren't
>    major flaws, but this is still a pre-requisite for the SRU process.
>    We check that unity works, check a few effects, possibly run a game
>    or other 3D app.
>
>There do exist tests for the nvidia and fglrx driver, but these are
>proprietary and internal to NVIDIA and AMD, so we cannot run them on
>our
>end.
>
>Other test suites include Piglit and the Phoronix Test Suite.  These
>have been discussed in QA sessions at recent UDS's, but I do not
>believe
>they're set up to run in our internal test frameworks.  Indeed, there's
>a number of complications for running the tests in a headless
>environment.
>
>
>> > However, for fglrx-updates and nvidia-current-updates we're finding
>it
>> > has been taking a considerable amount of time to get these released
>to
>> > the public.  The SRU process tends to slow us down here, and
>> > understandably so; the driver is closed source so the usual SRU
>review
>> > processes can't be used.
>> 
>> Is the main problem that they spend too much time in the unapproved
>> queue (i. e. SRU reviewers are hesitant to review/accept them), or
>too
>> long in -proposed (because testing feedback is difficult to gather),
>> or both?
>
>While we haven't done a huge number of -updates SRU's, history has
>shown
>that they tend to get accepted quickly.  See bug #887630 for instance.
>
>Where they get hung up is in the testing phase.  I think there's three
>reasons for this.
>
>  1.  The SRU bug report is a packaging request.  So there really isn't
>      a defined "test case".  (I guess we could say, "Yes, after
>     uploading x.y.z to -proposed, x.y.z is available in -proposed" but
>      that seems silly.)
>
>  2.  The driver changelog tends to be terse 1-2 sentences per fix.  So
>      reproducing the original issues can be challenging.
>
>  3.  Invariably someone will comment on the SRU bug that they have a
>      regression with the update, that they don't have with stock.  Who
>      knows if they do or not, or even if they're testing the right
>      thing.  But this stops the process cold.
>
>The third issue is particularly annoying because other users who *do*
>know the update brings improvements are forced to manually upgrade (and
>endure the resulting hassles that brings.)  And it's frustrating to the
>uploader - why bother having two different packages with basically the
>same exact driver, when all the work has been done to get a new one in,
>just because of an alleged regression reported by just one guy.

This is supposed to stop things.  The goal of -updates is to be free of regressions even if that means some things can't be fixed.

I'd think for basic infrastructure like video drivers this would be rather more important, not less.

Scott K
(SRU team member))

>> > But I think we can make a slight change which will gain us most of
>the
>> > benefits without incurring regression risks.  I'd like to propose
>the
>> > following for your approval:
>> > 
>> >  A.  Immediately after release we have the same major versions in
>> >      nvidia-current and nvidia-current-updates.  In this situation,
>we
>> >      allow the SRU team to expedite the upload without needing to
>wait
>> >      for testing in -proposed.
>> 
>> One thing we should keep in mind is that we have had n-c-updates for
>> several releases now. So if people installed -updates in e. g.
>> oneiric, upgraded to precise, quantal etc., they will have -updates
>> from day one. I think for this case, as well as making the process
>> have fewer variables, I'd skip this part and only do B.
>
>So that I understand correctly you're suggesting that A be dropped from
>the proposal?
>
>If we move people from the -experimental beta driver back to stock on
>upgrade (which I think we might want to do), maybe it would be worth
>considering moving people from -updates back to stock on upgrade as
>well?  That would resolve this use case, and enable us to do the
>quicker
>release to -updates?
>
>> >  B.  For subsequent updates, the normal SRU procedures are
>followed,
>> >      except that the 1 week testing requirement can be reduced or
>waved
>> >      if a sufficient quantity/breadth of testing has been
>performed.
>> 
>> This seems appropriate to me given that these drivers are not the
>> default and you have to actively look for them. I'm still interested
>> in above questions, though.
>
>Thanks,
>Bryce
>
>
>-- 
>technical-board mailing list
>technical-board at lists.ubuntu.com
>https://lists.ubuntu.com/mailman/listinfo/technical-board