Feedback on the QA cycle

[Simon Steinbeiß]

On Thu, 20 Mar 2014 16:53:00 -0700
Elizabeth Krumbach Joseph <lyz at ubuntu.com> wrote:

> Thanks, I do hope we can keep this one on track as it's an important discussion

+1

> > I have a few doubtful thoughts on exploratory testing. How do we motivate
> > people to run exploratory testing with the development version, while it is
> > not ready for production, or day-to-day environments? If the tests aren't
> > run on/as your main system, how can the testing be natural enough to be of
> > exploratory nature? How do we specify a good balance between feature and
> > exploratory testing?
> 
> I think what we'd struggle with is not people being unwilling to do
> the testing, as we know there are lots of people who do actually run
> development versions since we're always hearing feedback about how
> stable it is :) I think the issue is connecting them with bug
> reporting and other mechanisms for reporting results. I think if we
> even got feedback given via the mailing list it would be helpful. Not
> sure how to make this easier for people.

I've been running development versions on my daily workstation for basically as long as I've contributed to Xubuntu. I can't remember a cycle where things were so badly broken by an upgrade that it was a real problem. Anyhow, this doesn't mean I'd recommend this to anyone who doesn't feel comfortable with either having a second partition with the latest stable release around (and a separate home partition) or who feels comfortable about re-installing/downgrading. That said, I'd motivate anyone who contributes or wants to contribute to Xubuntu to try this.
I agree with Elisabeth that the main issue is not receiving bug reports from people. The infrastructure for this has already been made a lot easier to use and a lot happens automatically via apport. I'm also uncertain how we could improve this.
My guess is that to some extent people who haven't used open source products for a while are simply not familiar with the concept of being able to report a bug to "someone who reads/cares". How many of you guys have reported bugs against Windows (back in the day) via the automated error-reporting popup (wouldn't even know anymore how that's called exactly)? It just felt futile, so I never sent those error-reports (sorry, Windows-devs).

> > CONCLUSION
> >
> > To end the feedback on a positive note (though there weren't so many
> > negative points in total anyway), I think we have been up to the highest
> > possible standard with QA considering the size of our team and the amount of
> > new things landing this cycle.
> >
> > Finally, a big THANK YOU Elfy for running the QA team, doing all the calls,
> > reporting back to us, taking care of bugs being noticed, features landing in
> > time et cetera... Last but not least, thanks for putting up with us all who
> > have sometimes more or less neglected our duties in QA and being
> > unresponsive to questions and calls. It is very much appreciated, and I
> > totally think that 14.04 would be a lesser release without your work and
> > persistence!
> 
> Absolutely, Elfy's really done an exceptional job staying on top of
> all of this even with all his other commitments to Ubuntu and beyond.
> Thank you for your work!

+1000

 - Simon