Feedback on the QA cycle

[Elfy]

On 21/03/14 00:10, Simon Steinbeiß wrote:
> On Thu, 20 Mar 2014 16:53:00 -0700
> Elizabeth Krumbach Joseph <lyz at ubuntu.com> wrote:
>
>> Thanks, I do hope we can keep this one on track as it's an important discussion
> +1
>
>>> I have a few doubtful thoughts on exploratory testing. How do we motivate
>>> people to run exploratory testing with the development version, while it is
>>> not ready for production, or day-to-day environments? If the tests aren't
>>> run on/as your main system, how can the testing be natural enough to be of
>>> exploratory nature? How do we specify a good balance between feature and
>>> exploratory testing?
>> I think what we'd struggle with is not people being unwilling to do
>> the testing, as we know there are lots of people who do actually run
>> development versions since we're always hearing feedback about how
>> stable it is :) I think the issue is connecting them with bug
>> reporting and other mechanisms for reporting results. I think if we
>> even got feedback given via the mailing list it would be helpful. Not
>> sure how to make this easier for people.
> I've been running development versions on my daily workstation for basically as long as I've contributed to Xubuntu. I can't remember a cycle where things were so badly broken by an upgrade that it was a real problem. Anyhow, this doesn't mean I'd recommend this to anyone who doesn't feel comfortable with either having a second partition with the latest stable release around (and a separate home partition) or who feels comfortable about re-installing/downgrading. That said, I'd motivate anyone who contributes or wants to contribute to Xubuntu to try this.
> I agree with Elisabeth that the main issue is not receiving bug reports from people. The infrastructure for this has already been made a lot easier to use and a lot happens automatically via apport. I'm also uncertain how we could improve this.

This echoes my thoughts - I really don't see how we can improve on the 
tools we use, without it causing us to duplicate work.
> My guess is that to some extent people who haven't used open source products for a while are simply not familiar with the concept of being able to report a bug to "someone who reads/cares". How many of you guys have reported bugs against Windows (back in the day) via the automated error-reporting popup (wouldn't even know anymore how that's called exactly)? It just felt futile, so I never sent those error-reports (sorry, Windows-devs).
Also, there are more than 1 stagnant reports on Launchpad - if people 
use the tracker, it get's flagged for others reporting on the same image 
or package and hence reports are Confirmed and marked as having appeared 
on a tracker too.

That's likely to, or I would hope, cut down on dupes - which causes 
people extra work.

>>> CONCLUSION
>>>
>>> To end the feedback on a positive note (though there weren't so many
>>> negative points in total anyway), I think we have been up to the highest
>>> possible standard with QA considering the size of our team and the amount of
>>> new things landing this cycle.
>>>
>>> Finally, a big THANK YOU Elfy for running the QA team, doing all the calls,
>>> reporting back to us, taking care of bugs being noticed, features landing in
>>> time et cetera... Last but not least, thanks for putting up with us all who
>>> have sometimes more or less neglected our duties in QA and being
>>> unresponsive to questions and calls. It is very much appreciated, and I
>>> totally think that 14.04 would be a lesser release without your work and
>>> persistence!
>> Absolutely, Elfy's really done an exceptional job staying on top of
>> all of this even with all his other commitments to Ubuntu and beyond.
>> Thank you for your work!
> +1000
>
>   - Simon
>


-- 
Ubuntu Forum Council Member
Xubuntu QA Lead