[ubuntu-x] Final Maverick i830, i845g, i855 support

[Bryce Harrington]

On Mon, Sep 13, 2010 at 05:18:27PM -0400, Eric Appleman wrote:
>   On 09/13/2010 03:41 PM, Bryce Harrington wrote:
> > On Mon, Sep 13, 2010 at 12:22:48AM -0400, Eric Appleman wrote:
> >> I'm okay with this. Do you guys want me to organize a hardware testing
> >> effort to flesh out how well this ends up working?
> > Would you mind outlining what you have in mind?
> >
> > Bryce
> With pleasure.
> 
> Remember this? https://wiki.ubuntu.com/X/UxaTesting
> 
> We should duplicate that effort for i8xx testing and call upon the 
> posters of the Maverick testing forum and Freenode to post their 
> experiences.
> 
> While the i8xx issues are documented in one way or another across this 
> mailing list with respect to the current broken situation, there will be 
> a need for a centralized location for such data on the wiki as the 
> driver changes are pushed into the archives. It's not hard for me to 
> imagine that the Maverick forum will get a string of "Just updated pkgs. 
> WTF happened to mah grafix?" topics.
> 
> In my opinion, UxaTesting was extremely successful because you guys made 
> it ridiculously simple for Ubuntu users to share their results in a 
> detailed and organized manner. Editing a wiki and figuring out your VGA 
> string are effortless compared to joining and posting on a mailing list.

I'd actually suggest you'd derive from one of our later wiki test page
formats, e.g.:

 https://wiki.ubuntu.com/X/Testing/GEMLeak
 https://wiki.ubuntu.com/X/Testing/NouveauEvaluation

The Uxa test page was nice in that it gathered a *lot* of feedback, the
problem was it was a bit too freeform and was hard to analyze and draw
conclusions from.

Remember, the purpose of testing is not to collect a mound of anecdotal
evidence, but rather to make a well-informed decision.  One of the
errors we made with the Uxa testing was not tracking what versions the
user tested, so as we uploaded patches and fixes we could distinguish
test results relevant against the current version vs. results against
past versions.  So by the time we needed to make some decisions, the
page had lost much of its usefulness for analysis.

The NouveauEvaluation page is probably my favorite so far, and directly
helped us make some good decisions, and made it really clear where the
remaining problems were.  The only thing I'd add would be to make bug #
links more explicit, so if there's only a few failures we can
investigate those cases individually.  And maybe encourage people to
keep the table sorted by date last tested.

One risk when doing crowd-sourced testing is that the testers tend to be
self-selected for people with bugs.  So it is possible you get 9 out of
10 test responders saying Foo change is GOOD and solves whatever
problem, but when you roll out the change you learn those 9 people
represent like <1% of the userbase, and that 1 responder accounts for
99% who had been fine and are now regressed.  %-}


The QA team also did a test campaign for proprietary drivers for Lucid,
which turned out *really* well from my perspective.  The test campaign
organized testers to re-test once every week, and we could track who
tested what when, so it was really clear when a regression occurred.
The QA team has their own web app for this:

  http://xorg.qa.ubuntu.com/

The UI is a bit funky and has more of a learning curve than a simple
wiki.  Maybe you'd find wiki easier to get up and running with quickly,
given how late it is in the cycle, but worth a look.  If this looks
interesting, talk to Ara to get set up.

> I'm quite optimistic that what we may learn will be critical to 
> evaluating the efficacy of fbdev+kms, bringing any potential bugs or 
> freezes to light, and perhaps steer us in finding a more ideal solution 
> for Natty.

It certainly would be informative to see separate NouveauEvaluation-like
tables of results for each configuration option.  Could help in making a
good decision.

I'm a bit worried both tables are going to show a lot of red.

Bryce