Bug Triage processes need some improvement and automation...

Thomas Ward teward at trekweb.org
Tue Nov 5 16:53:08 UTC 2013 




*Sent from my iPhone.  Please excuse any typos, as they are likely to happen by accident.*

On Nov 5, 2013, at 11:01 AM, AG Restringere <ag.restringere at gmail.com> wrote:

>> I have to (constructively) disagree with these suggestions:
>> - With the first one because, although it's an improvement, it's just a distraction of what is really important now.
>> - With the second one because of putting barriers in front of collaboration, one key of success.
>> - With the point of view in general because forgetting simplicity, the ultimate goal of any system.
> 
> Got it, understood, you don't want to alter the current processes, adding a "Managed By:" field and using it doesn't mean altering processes, it just provides additional information that reduces some uncertainty.
> 
> Let's look at this recent bug I submitted to Launchpad:
> https://bugs.launchpad.net/ubuntu/+source/unity/+bug/1221304
> 
> Right now there are some bug comments from Stephen M. Webb (Bregma) indicating some sort of triage activity but there's nothing on the bug report letting me know if it's being actively triaged and by who.  

As it should be.  Triaging is *not* managed by a single individual for a bug, it's a community effort with everyone working together to get the bug to where it can be worked on.

Take any individual nginx bug in which I have had sponsored-uploaded fixes for as an example, security bugs included, or is currently marked as Triaged.  Back when Debian maintainers were working on it in Ubuntu, they were OK with it, but were stumbling along on the triage process, and fixing bugs like in Debian, but getting hung up on the 'version is stuck here' stuff and not fixing all the bugs in Precise and other releases.  Then I come around and provide the Bug Squad knowledge needed.  That shifted the active triage to me from the Debian maintainers.  But I by no means am the only one triaging nginx bugs, there are a few people who work on the bugs and add information or request information.  And there are times where I do hand off the bug to Debian for them to fix, which means Debian is triaging it then, and not myself.  And while my name is likely on the debdiff I was likely not the one who developed the fix.

> This information should be clearly displayed on the bug report header. Here's a quick mock-up of that clearly describes what I would like to see on a bug report.
> 
> ---------------------------------------------------------
> Bug #1221304
> 
> "Global app-menu is missing from panel..."
> 
> Reported by: AG Restringere on 2013-09-05                    /* who reported the bug? */
> 
> Affects: (the package information below)
> Assigned to: (Developers, Packagers, etc...to fix...)        /* which developer or packager is fixing the bug? */
> 
> Managed by: Stephen M. Webb, Alberto Salvia Novella   /* who is triaging the bug right now? */

Additional useless cruft because this cannot be auto assigned if, say, twenty people comment on it.  The system won't be able to determine who is actually doing triage and who is just putting "This affects me" and stuff, case in point any of a thousand kernel bugs with such cases.

> 
> ---------------------------------------------------------
> 
> The addition of this field would in no way alter current Bug Triage processes or methods that are currently in place, those would remain the same. This would just make it simple and clear what is going on with the bug without having to analyze comments. Then members could use the Launchpad search filters to find bugs don't have anyone managing them removing unnecessary complexity from the bug search process.  This tool would simply be for informational purposes and everyone would still have access to the bug and be able to help triage it if they wanted to.  

But how would this improve the system?  This just sounds like a way to add more 'Minor feature request' stuff to the Launchpad devs, and they more than likely are already inundated with minor things that are being trumped by more major things.

As well, the comments are there for a reason.  A bug's 'state' of being actively worked on hinges on comments and activity, not a list of triagers.  A sign of a good bug is a bug in which those affected, those helping to triage, and those who can fix it are all collaborating, and in those cases it wouldn't work to have a list of those who are "triaging" the bug because it'd add more work for bug supervisors to figure out who actually *is* triaging while others are just commenting.

This also adds to the complexity of the process of triaging whether you personally think it does or not.  The only way to logically and feasibly apply this is if the 'triaged by' role only picks up on comments by people on bug squad or bug control, and ignores the others, or if the process is manually done and bug controllers (who already have elevated permissions) are the only ones who can set the status of who is triaging the bug, and that too adds additional work to an already-pretty-simplistic system.

> 
> Best regards,
> AG 
> 
> 
>> On Tue, Nov 5, 2013 at 6:45 AM, Alberto Salvia Novella <es20490446e at gmail.com> wrote:
>> I have to (constructively) disagree with these suggestions:
>> 
>> - With the first one because, although it's an improvement, it's just a distraction of what is really important now.
>> - With the second one because of putting barriers in front of collaboration, one key of success.
>> - With the point of view in general because forgetting simplicity, the ultimate goal of any system.
>> 
>> 
>> El 04/11/13 23:32, AG Restringere escribió:
>>>> Bug Squad does not have (i.e. to set the "Triaged" state and the bug
>>>> importance, as well as other bug-control specific tasks), the
>>>> "Assigned To" field on bugs is used to identify who the work on fixing
>>>> the bug is assigned to, not the triager.
>>> 
>>> Sorry, I didn't understand just how specific the usage of the term "Assigned To:" is in Ubuntu.  Given that in Ubuntu the "Assigned To:" term/status is used specifically for Developers and Packagers that will fix the bug then I am not using a correct term.  In that case a new term is needed to avoid confusion.  The better term would be a "Managed By:" status because the Bug Triage person is managing the bug but not fixing it. This would work in a similar way to "Assigned To:" but it would apply to Bug Squad and Bug Control members and not Developers or Packagers.  The new status would make it crystal clear which Bug Squad/Control member is managing the bug and whether it's being actively managed. The Launchpad bug-menu and search filters would have to be modified to accommodate this and only Bug Squad/Control members could have permissions over it is so the public wouldn't use it by mistake.
>>>  
>>>> Assigning one "triage owner" for a bug defeats the general idea of
>>>> that collaboration of which myself and others are so fond of.
>>> 
>>> Collaboration could still be possible but there would be a more systematic approach.  Just because a bug is "Managed By:" a single Bug Triage person and doesn't mean that they can't ask other Bug Triage members for help, advice, to look at a log, agree that a two bugs are duplicates, etc...It just means that in the end of the day one single Bug Triage person is making comments on the bug and that one person is responsible for triaging it. Just take the number of open non-triaged bugs and divide them by the number of Bug Triage "staff" currently available, you'll most likely get a very large quantity.  Given the high level of interest that OEM's and games developers have in Ubuntu you'll probably want to ensure Bug Triage is as fast as possible.  Unfortunately Bug Triage is a limited resource, the team only has so much time to work on a large number of bugs, so you'll have streamline the process to make it faster.
>>> 
>>>> I'm confused here...(...)...In comparison with the
>>>> packages' bugs which I am specifically subscribed to, I've seen very
>>>> few bug-control subscribed bug stuff, so I'm a little confused with
>>>> this modification or concern.
>>> 
>>> Then this is something team specific that I will have to bring to the attention of the Ubuntu-X team and I'll have to see if there's a way they can tone down their bug volume or if I can turn off my bug-mail for that specific team.
>>> 
>>> All in all I don't know how "workable" these suggestions are at this point and it is certainly a long-term process of improvement that will have to be taken in small incremental stage. The most important item in my view is to implement the "Managed By:" tool/status and the "one bug one Triager" system.
>>> 
>>> Best regards,
>>> AG
>>> 
>>> 
>>>> On Mon, Nov 4, 2013 at 4:15 PM, Thomas Ward <teward at ubuntu.com> wrote:
>>>> Hey, AG, thanks for splitting this off into a separate discussion.
>>>> 
>>>> On Mon, Nov 4, 2013 at 3:54 PM, AG Restringere <ag.restringere at gmail.com> wrote:
>>>> > Re-posting my message to this list to create a new thread about Bug Triage
>>>> > procedures. I've outlined some ideas that I think would really improve the
>>>> > efficiency and clarity of Bug Triage operations.
>>>> >
>>>> > --------------------------------
>>>> >
>>>> > In my experience in - when I dabbled in some bug control work as part of the
>>>> > Ubuntu-X team - the Bug Triage process is still very tedious and lacks
>>>> > sufficient automation. Most of the time and effort I spent doing bug control
>>>> > work was spent browsing Launchpad and reading through hundreds of bug-emails
>>>> > in order to find bugs to work on.  Most of my time was spent searching for
>>>> > bugs but very little of my time was spent actively working on bugs and being
>>>> > productive.  Also, many times I saw bugs that had comments from Bug Control
>>>> > members but it was never clear who was working on the bug or what they
>>>> > wanted to do with it.  This often lead me to add comments when they weren't
>>>> > needed or not contribute when a bug actually needed attention and action.
>>>> >
>>>> > Modifications I would make to the Bug Triage process:
>>>> >
>>>> > 1. Assignment and eliminating redundancy:
>>>> >
>>>> > When a Bug Triager begins working on a bug they should assign themselves to
>>>> > the bug on Launchpad if they intend to actively work on it.  Only one Bug
>>>> > Triage member should be assigned and actively working on a bug at any given
>>>> > time and they should effectively "own" that bug and be responsible for it.
>>>> > The only other people who should be working on that specific bug should be
>>>> > Reporters, Testers and Developers.  Assignment would help other Bug Triage
>>>> > people to know "this bug is actively owned by another member" and know to
>>>> > move on to other bugs and leave that one alone.  It would be even better if
>>>> > Launchpad could filter out the bugs that were actively assigned to Bug
>>>> > Control members so people could find those that nobody was working on and
>>>> > needed attention.  Sufficient criteria for finding new bugs could be as
>>>> > simple as "Confirmed"+"Unassigned".
>>>> 
>>>> I am against this suggestion as it stands, maybe because I do not
>>>> understand your reasoning for it or the potential execution of this,
>>>> but also because there needs to be made a distinction, in my opinion,
>>>> between the role of Bug Triager and the role of "Package Fixer" (for
>>>> lack of a better term, this would be someone with the package
>>>> knowledge and development knowledge to fix the bugs once             they're
>>>> "Triaged").
>>>> 
>>>> A triager may help get the bug from New to Triaged or New to
>>>> Confirmed, but ultimately someone with developer knowledge of the
>>>> package, or the knowledge to patch the package, is going to be the one
>>>> the bug is "assigned" to for the work item.  As well, a single
>>>> individiual triager may have to collaborate with other triagers in
>>>> order to get the package to the "Triaged" state.  I myself have
>>>> collaborated with other bug controllers and bug triagers in order to
>>>> get bugs moved along to a point where a developer can work on the
>>>> bugs, and in most cases, I quite like that collaboration.  That
>>>> collaboration would then, in a sense, mean that all the triagers who
>>>> have collaborated on it are "owners" of the bug for a triaging sense.
>>>> Assigning one "triage owner" for a bug defeats the general idea of
>>>> that collaboration of which myself and others are so fond of.
>>>> 
>>>> Also, unless you're proposing changing the bug system to have an
>>>> additional "Triage Owner" role and field on the bug and restricting
>>>> "Triage Owner" to bug controllers who actually have the access that
>>>> Bug Squad does not have (i.e. to set the "Triaged" state and the bug
>>>> importance, as well as other bug-control specific tasks), the
>>>> "Assigned To" field on bugs is used to identify who the work on fixing
>>>> the bug is assigned to, not the triager.  I still stand by this,
>>>> because as one of the people primarily working on the nginx package
>>>> now, I have seen people assign themselves to bugs and fix them, or
>>>> assign themselves, and then hand me the work later, and reassigning it
>>>> to me as the person who will fix it or SRU it or whatever.
>>>> 
>>>> >
>>>> > 2. Email volume reduction:
>>>> >
>>>> > Bug Triage members should only receive emails about bugs they're actively
>>>> > assigned to.  It's really time consuming to sort through hundreds of
>>>> > bug-mails that involve bugs that are not relevant to ones currently being
>>>> > worked on.  This applies to all roles such as Testers, Reporters and others
>>>> > as well.  The only general emails that should be received should be from the
>>>> > discussion or developer mailing lists.
>>>> 
>>>> I'm confused here.  As a Bug Squad member, I have received             exactly 0
>>>> email addresses for subscribed bugs, in that the Bug Squad isn't
>>>> subscribed to any bugs by default.  As a Bug Control member, I see
>>>> some crash bug data for which bugcontrol is subscribed to, or is a
>>>> member of one of the teams subscribed.  In comparison with the
>>>> packages' bugs which I am specifically subscribed to, I've seen very
>>>> few bug-control subscribed bug stuff, so I'm a little confused with
>>>> this modification or concern.
>>>> 
>>>> >
>>>> > 3. Auto-assignment process queue:
>>>> >
>>>> > Similar to a tech-support ticket system the next step in this process would
>>>> > be to introduce a process queue with auto-assignment of bugs to Bug Triage
>>>> > members.  I don't know how this would work but some status change in the bug
>>>> > would have to trigger it's submission it into the process queue such as
>>>> > reaching a Confirmed status or increased Reporter activity at some threshold
>>>> > level. The distribution of the bugs would have to take into account the
>>>> > work-load of the Bug Triage members and distribute them evenly but perhaps
>>>> > that's a bit too complicated to do in code. Maybe random assignment would be
>>>> > better or it could based on the package selection preferences of individual
>>>> > members. Perhaps there could even be some senior Bug Control members who
>>>> > would manually assign the bugs from the queue.  This would eliminate the
>>>> > need for Bug Triage members to even need to go to                 Launchpad to search for
>>>> > bugs unless they're doing some extra research.  Bugs would be sent to them
>>>> > via email automatically when they're ready to be triaged and auto-assigned
>>>> > without any extra steps needed.
>>>> >
>>>> > Conclusion:
>>>> >
>>>> > If the above steps were implemented or some equivalent processes I think the
>>>> > Bug Triage would be streamlined, eliminate redundancy and get faster
>>>> > turn-around times. Bug Triage members would be more                 focused and successful.
>>>> > Newer Bug Triage members would be able to be "plugged in" to a standardized
>>>> > process and this would improve retention because people would see results
>>>> > faster with less effort.
>>>> >
>>>> > --------------------------------
>>>> >
>>>> > Hopefully the feedback and ideas above can be tested in some form and
>>>> > implemented.
>>>> >
>>>> > Best regards,
>>>> > AG
>>>> >
>>>> > --
>>>> > Ubuntu-bugsquad mailing list
>>>> > Ubuntu-bugsquad at lists.ubuntu.com
>>>> > https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugsquad
>>>> >
>>>> 
>>>> ------
>>>> Thomas
>> 
>> 
>> --
>> Ubuntu-bugsquad mailing list
>> Ubuntu-bugsquad at lists.ubuntu.com
>> https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugsquad
> 
> -- 
> Ubuntu-bugsquad mailing list
> Ubuntu-bugsquad at lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugsquad

I'm with Alberto on this, I disagree with all of these suggestions.

Especially since none of these suggestions, from my point of view, add anything to the system and if anything add work for the triagers and also adds more work for the LP devs just to implement this.

-----
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-bugsquad/attachments/20131105/3b490acb/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2326 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-bugsquad/attachments/20131105/3b490acb/attachment.bin>

More information about the Ubuntu-bugsquad mailing list