Backtracing, Invalidated Bugs and Quality

[Null Ack]

Gday everyone. As part of my work with the QA Team I want to
contribute to fixing the process gaps in this area. Can I summarise
what I see as the problem:

Problem situation: I'm increasingly noticing that certain types of
bugs are being marked invalid or incomplete with boilerplate type
messages instructing the bug reporter to conduct a backtrace. The
engagement of the end user is poor, the user experience is
non-intuitve, the documentation to walk through how a user can do this
is poor and the net effect is that the "hit rate" of users actually
fulfilling the request is very low. The net result is usually that the
bug stagnates and duplicate bugs pile up. It may, or may not, then get
filled upstream if the number of duplicate bugs gets high enough for
somebody to notice it. I have a high regard for all the Ubuntu
developers, and I say this carefully, but I think if were all honest
about the situation like some developers have been to me there is an
element of "gee Im so busy and this bug report looks non trivial...I
might just copy and paste my back trace wording to this, move the
status to get it out of the way and if it's a real problem eventually
a bunch of users will report this too and then I might send it
upstream then because upsteam have the time to look at this."

There was a discussion on IRC which I've summarised here for the list
and some proposed action items:

1. The Tool Chain For Debugging is Not Robust

The point was made that the debugging toolchain is complex and will
not consistently provide the needed debugging information on all
occasions. Sometimes the retracing will fail for some reason. I simply
see this as a longer term challenge for the FOSS community to work on
bugs in the toolchain - obviously having a reliable and repeatable
method for getting right into the guts of the registers and stack is
important for fixing the more curly bugs. The toolchain being
imperfect however is not an excuse for failing to implement best
practices in Ubuntu for debugging in the meantime. We can make
progress.

2. The Volume Of Bugs Coming Through Makes The Hard Ones Too Hard

It was suggested that the number of bugs coming through is so high
that trying to fix the more tricky ones isnt worth the time given
available amounts of person power. I made the point and I'd like to
highlight it again that the complexity of fixing a bug should not be
the criteria for which bugs get developer attention. The best practice
for building quality in Ubuntu in my view is the determinants should
be how seriously it effects the user experience and how common that
user experience is. When I've got stuck in my testing work on Ubuntu
I've appealed for help in the testing section of the Ubuntu forum or
on IRC, and I've been greatly encouraged by good helpful responses
back. Im sure bug squadders and Ubuntu testers would be happy to
respond to developers with unit testing, feedback etcetc. I recently
helped Alexander Sack with performance feedback on a web browsing item
and unit testing - it was fun!

2. The Need For Improving Apport

A developer suggested that there is not a gap with Apport as it exists
now. I disagree and I sighted the example of where a package is
compiled with optimised compiler flags that a debug package will need
to be installed to get a meaningful trace. I know from experience that
automated ways of doing this, or atleast having an easier and
intuitive user workflow for this is better than documentation. I
really like Markus' ideas for improving Apports functionality he
shared earlier.

Action Item 1: I'm not a developer, but I can help any developers with
testing and feedback for enhancements to Apport. I might also be able
to assist with design / blueprints / discussing possible features. Or,
someone come up with compelling reasons why Apport is fine the way it
is and the worflow issues can be resolved another way.

Another thing that came up in the talks was that the backtrace
boilerplate copy and paste wasnt always accurate in the circumstances
its being used. Sometimes the real issue is being able to replicate
the problem not the backtrace. Or, a backtrace on a debug build is
truly needed but the user doesnt know how to help in detail and bug
squadders cant replicate the problem at will on their configurations.
Or, since there is considerable obselete info hanging around there is
confusion with bug squadders about what exactly to do and human error
has occured.

Action Item 2: A review of the documentation on both the user side and
the bug squadder / developer side to more fully explain and walk
people through the situation. I can help here too, but again I'm not a
developer so especially the more technical aspects of the backtrace,
why it sometimes fails, how to do manually, will need other peoples
involvement. Basically to improve the hit rate.

That seems to be what the IRC logs touched on, thanks.