unnecessary QA work (was Re: dapper-live test)

Matt Zimmerman mdz at ubuntu.com
Mon May 8 18:06:03 BST 2006


On Mon, May 08, 2006 at 12:49:20AM +0100, Colin Watson wrote:
> On Thu, May 04, 2006 at 10:30:06AM -0700, Matt Zimmerman wrote:
> > Duplicates are a fact of life, but it is not too much to ask, no.
> > Consider that it takes the bug reporter perhaps a minute to look for
> > their bug, and then a few minutes more to write it up if they don't
> > find it.  Then consider that all of these reports, from thousands of
> > people, are funneled to a tiny development team, who must deal with
> > all of the duplicates.  This means they spend less time on real bugs,
> > which means less value for the users reporting them.
> 
> I used to agree with you, but after dealing with a package with a high
> concentration of duplicate reports, I've changed my mind.
> 
> 
> The assumption here is that the alternative to a duplicate bug report is
> the user going away mollified, perhaps subscribing to a bug, but not
> posting any comments. In my experience, this is not what happens in many
> cases, particularly in cases where users simply do not have the
> experience to tell reliably whether their bug is a duplicate of an
> existing one or not. What happens is that the user either (a) files a
> new bug which may or may not be a duplicate, or (b) comments on an
> existing bug which may or may not be the same as their bug, perhaps
> attaching debugging information. Each case has its failure modes; from
> my sample size, I don't observe that either case is significantly skewed
> towards the success mode or the failure mode.
> 
> In case (a)'s failure mode (a duplicate bug), every bug tracking system
> I've ever worked with, including Malone, can deal with the situation
> easily. You mark the bug as a duplicate, perhaps adding a comment, and
> move on.

It is much faster to find duplicates in a short list of bugs against a
single package; it is much more effort to deal with them on a
distribution-wide scale as we primarily do.  There are very few cases
where we have someone focused on a package to the extent that you are with
ubiquity.

> In case (b)'s failure mode (comment about a bug attached to a different
> existing bug), I've never seen a bug tracking system that deals with the
> situation gracefully. The best I've seen is the 'clone' command in
> debbugs, which at least allows you to split off the bug history to
> another bug and then deal with the two parts independently, but even
> that isn't particularly great. I've never seen a system that allows a
> developer to hide extraneous comments to avoid them causing confusion to
> that developer in the future when they come to work on the bug, so the
> failure mode always leaves permanent noise, unlike case (a)'s failure
> mode which results in a neat list of duplicate bugs in a portlet which
> can easily be ignored.

I think that being able to trim comments would be a sensible and useful
feature in Malone for this reason.  Threading would also help this problem,
but I know that's been avoided for reasons of UI simplicity.

> If you have to ask a commenter to file a new bug, it depends on whether
> you think the bug is important or not. If the bug seems important, you
> have to fix it immediately anyway or re-file it yourself, as commenters
> don't always get round to filing a new bug on request; plus, in Malone,
> you have to remember to make sure that the commenter is subscribed to
> the bug, and I often forget. You can't always assume that somebody else
> will file the bug if it's important; see for example
> https://launchpad.net/distros/ubuntu/+source/ubiquity/+bug/42260, where
> a user reported a crash affecting Vietnamese users that you can only
> recover from by rebooting or by knowing an obscure command-line rune,
> but which no other users would notice at all. I had to add that one to
> my to-do file and make sure to remember to fix it separately, which is
> less efficient than letting a bug tracking system remember about it for
> me.

Users who are mistaken about whether their bug is a duplicate can just as
easily abuse the duplicate marking feature as comment on the wrong bug.
Yes, this is easy to reverse, but only when we notice it, and we don't have
enough eyes to monitor all such activity.

I appreciate this situation on the micro scale, but when considering the
(literally) thousands of untriaged bugs we have right now, we need to do
something to stem the tide of irrelevant reports for the distribution as
awhole.

> > If someone is not willing to spend a minute searching for a bug, will they
> > be willing to answer questions about their problem to help get it fixed?
> > Will they be willing to gather and send stack traces, debug logs, etc.?  If
> > not, then I would rather receive the same report from someone willing to do
> > so.
> 
> Malone's searching interface is not all it could be. For instance:
> 
>   https://launchpad.net/products/malone/+bug/33881
>   https://launchpad.net/products/malone/+bug/38881

This is clearly true.  However, I find that Google generally provides
excellent search results for Launchpad, and has the bonus side effect of
finding other relevant resources outside of Launchpad (such as bug reports
in other systems) at the same time.

> Furthermore, it's not always at all obvious where to look for a bug.
> Perhaps in the case of simple self-contained applications it's not too
> difficult, but complicated subsystems like the installer or hardware
> detection or font management have their bugs spread out over a number of
> packages and it's not even conceptually easy for Malone to guide users
> to the right place; it's increasingly infeasible to search through all
> Ubuntu bugs for an issue unless you happen to be technically savvy
> enough to know the right technical terms to search for (as far as I can
> tell, Malone full-text searches only search the description and similar
> fields, not all comments).

Folks doing bug triage have very much the same problem, but with thousands
of bugs rather than the single bug the reporter needs to consider at the
time.

-- 
 - mdz



More information about the ubuntu-devel mailing list