From mpt at canonical.com Mon Jul 2 00:14:28 2007 From: mpt at canonical.com (Matthew Paul Thomas) Date: Mon, 2 Jul 2007 12:14:28 +1200 Subject: martin's apport test stuff & simple bug triaging In-Reply-To: <511f47f50706291253qfc01a9bjc2f4404b0a35fd9e@mail.gmail.com> References: <511f47f50706291253qfc01a9bjc2f4404b0a35fd9e@mail.gmail.com> Message-ID: <9cca465716868b1ec4eb518c3e549d49@canonical.com> On Jun 30, 2007, at 7:53 AM, shirish wrote: > ... > 1. Have a filter by distribution and/or releases. So let's say I wanna > have an overview of all bugs filed after gutsy tribe 1 onwards . Right > now thats very hard to know, the filter should work while filing the > bug so it makes easier. I've added this use case to "Search based on date a bug was reported" . > 2. perhaps have a gutsy-users or something like that on freenode so > people can do simple bug triaging like I did with jerome or something > close to that so the guys can work on more confirmed bugs. > ... I think that's what #ubuntu+1 is for. Cheers -- Matthew Paul Thomas http://mpt.net.nz/ -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part URL: From henrik at ubuntu.com Tue Jul 3 10:40:30 2007 From: henrik at ubuntu.com (Henrik Nilsen Omma) Date: Tue, 03 Jul 2007 12:40:30 +0200 Subject: Bug workflow - a wider view Message-ID: <468A279E.1090306@ubuntu.com> Hi, Robert Collins linked to a very interesting video today on his blog. It's a Google tech talk by Mary Poppendieck called 'Competing On The Basis Of Speed' [1]. Highly recommended. The main message was about getting the process right, and used both manufacturing, design and software engineering as examples. In each case QA is and integral part of the process, and must be seen as such. In Ubuntu we are quite good at following the principles described in that talk in many areas. We release code early and often so it gets thoroughly tested by a large community; we stress test our CD generating machinery regularly with both daily builds and periodic milestones. We also refactor and improve our procedures constantly. I'd like to bring more focus on the overall process into our bug work as well and the first step is to get a good overview of what we are doing and what we need. I'd like to get feedback from both developers and bug triagers on what procedures work for you and where the current bottlenecks are. How can we make the process simpler and more uniform so that more people will enjoy doing QA? I'd prefer the discussion to not focus too much on the finer details of the tools (ie. feature wishlists of Launchpad or bughelper can be discussed separately), but rather the overall flow of feedback from our testers and users and how that can be used most efficiently to improve Ubuntu. The current guide to bug triage is: https://wiki.ubuntu.com/Bugs/HowToTriage but this needs some updating IMO and not just to reflect recent Launchpad changes. I've started posting some of my thinking here: https://wiki.ubuntu.com/Bugs/WorkFlow Comments or edits are of course welcome! Henrik [1] http://video.google.com/videoplay?docid=-5105910452864283694&q=google+talks From henrik at ubuntu.com Tue Jul 3 19:19:55 2007 From: henrik at ubuntu.com (Henrik Nilsen Omma) Date: Tue, 03 Jul 2007 21:19:55 +0200 Subject: Bug workflow - a wider view In-Reply-To: <20070703184235.GK14223@bryceharrington.org> References: <468A279E.1090306@ubuntu.com> <20070703184235.GK14223@bryceharrington.org> Message-ID: <468AA15B.2030606@ubuntu.com> Bryce Harrington wrote: > One thing I think could help a lot is to give more emphasis (and > recognition) to those who are filling the tester role. I think the > overall workflow process would go smoother if we had more testers, with > better tools, processes, and know-how. > That's a good perspective to bring to the debate. The laptop testing team was quite active for a few cycles (after we sent out laptops). It would be great to see that effort revived. During the Feisty cycle we also grew a great team of volunteer ISO testers. The ISO tracker has the potential for being extended to cover test cases of different kinds, from Mozilla to kernel and xorg updates. In the same we we currently list ISOs and test cases we would list milestones or proposed package updates and test cases for those. > I notice with Ubuntu there are a lot of people filling the testing role > officially and unofficially, but there are still a lot of areas where we > could use more good folk like this, or where the knowledge of how to do > good testing could be shared more deeply. Also, I wonder if the > prestige of being a tester could be enhanced via some mechanism > analogous to what MOTU is for developers. Interesting. So far the idea has been that active testers would join ubuntu-qa in the same way active triagers do. We may want to look at a new team for this purpose or at least make the testing path into ubuntu-qa more explicit (the team name does fit the testing role quite well). Henrik From bryce at bryceharrington.org Tue Jul 3 18:42:35 2007 From: bryce at bryceharrington.org (Bryce Harrington) Date: Tue, 3 Jul 2007 11:42:35 -0700 Subject: Bug workflow - a wider view In-Reply-To: <468A279E.1090306@ubuntu.com> References: <468A279E.1090306@ubuntu.com> Message-ID: <20070703184235.GK14223@bryceharrington.org> On Tue, Jul 03, 2007 at 12:40:30PM +0200, Henrik Nilsen Omma wrote: > I'd like to bring more focus on the overall process into our bug work as > well and the first step is to get a good overview of what we are doing > and what we need. I'd like to get feedback from both developers and bug > triagers on what procedures work for you and where the current > bottlenecks are. How can we make the process simpler and more uniform so > that more people will enjoy doing QA? > > I'd prefer the discussion to not focus too much on the finer details of > the tools (ie. feature wishlists of Launchpad or bughelper can be > discussed separately), but rather the overall flow of feedback from our > testers and users and how that can be used most efficiently to improve > Ubuntu. Hi Henrik, One thing I think could help a lot is to give more emphasis (and recognition) to those who are filling the tester role. I think the overall workflow process would go smoother if we had more testers, with better tools, processes, and know-how. Many bug management workflows don't distinguish between "users" and "testers" very much. I guess this is since in proprietary software development, users don't really get much of a role. One of the great things about open source is that users *do* gain a role, and to great effect. However, a risk with this is that the "tester" role can be lost, or reduced in prestige to the point no one really does it. True testing is hard and frustrating, but if done well it can save the developer time and the user pain. Users approach software testing from the perspective of "can I make this new software I've never tried work?" Bug reports are often a last resort, and not something they do often. Testers approach the problem from the different perspective "can I make this software I know really well break?" They are experienced in creating bug reports and know exactly what information to include. They often have interacted with the developers in the past, and earned trust; the developer knows if their tester tried the new software, found no bugs, and gave a thumbs up, then they're good to go. In an ideal situation, the tester devotes themselves to testing each and every release and update, so the developer can come to rely on them. Good testers also take attack bugs with a stronger toolkit than ordinary users. They master the debugger, learn protocol capture tools, learn about tracing tools, run package-specific tests, and experiment when they find a bug to see if they can narrow it down beyond just "it broke". They may research upstream, validate workarounds, write regression test cases, etc. There are some similarities between testers and triagers, however the roles are distinct. Triagers focus on the reports, and working to ensure they are complete, whereas testers focus on the underlying bugs themselves. I notice with Ubuntu there are a lot of people filling the testing role officially and unofficially, but there are still a lot of areas where we could use more good folk like this, or where the knowledge of how to do good testing could be shared more deeply. Also, I wonder if the prestige of being a tester could be enhanced via some mechanism analogous to what MOTU is for developers. Bryce From bryce at bryceharrington.org Tue Jul 3 19:49:23 2007 From: bryce at bryceharrington.org (Bryce Harrington) Date: Tue, 3 Jul 2007 12:49:23 -0700 Subject: Bug workflow - a wider view In-Reply-To: <468AA15B.2030606@ubuntu.com> References: <468A279E.1090306@ubuntu.com> <20070703184235.GK14223@bryceharrington.org> <468AA15B.2030606@ubuntu.com> Message-ID: <20070703194923.GN14223@bryceharrington.org> On Tue, Jul 03, 2007 at 09:19:55PM +0200, Henrik Nilsen Omma wrote: > Bryce Harrington wrote: > > One thing I think could help a lot is to give more emphasis (and > > recognition) to those who are filling the tester role. I think the > > overall workflow process would go smoother if we had more testers, with > > better tools, processes, and know-how. > > > That's a good perspective to bring to the debate. The laptop testing > team was quite active for a few cycles (after we sent out laptops). It > would be great to see that effort revived. During the Feisty cycle we > also grew a great team of volunteer ISO testers. > > The ISO tracker has the potential for being extended to cover test cases > of different kinds, from Mozilla to kernel and xorg updates. In the same > we we currently list ISOs and test cases we would list milestones or > proposed package updates and test cases for those. There's also two different paths that a tester could take. One is to focus on a specific package (maximize domain knowledge) and find lots of various ways to test it. The other is to master a specific set of tools (e.g. dogtail, or ethereal, or whatever), and then apply them across a breadth of packages sequentially (maximize problem knowledge). An elaboration on the latter approach I've thought could be fun for folks would be to organize a team of testers each specialized in a different kind of analysis, and pick one project/package a week or a month to tackle. Sort of like how Wikipedia does with their Featured Article of the Day. Even more important than finding bugs would be to leave the framework of tests and tools for future developers, users, and testers to use. Maybe there could be some tie-ins with other efforts like the bug days. Bryce From kees at ubuntu.com Wed Jul 4 10:37:17 2007 From: kees at ubuntu.com (Kees Cook) Date: Wed, 4 Jul 2007 03:37:17 -0700 Subject: Bug stat graphs Message-ID: <20070704103717.GU29861@outflux.net> Hi, Since Carthik's original Ubuntu Bug Stats[1] has been offline since the "Unconfirmed" -> "New" change, I've put up an updated version here[2] until his is back up and running. Take care, -Kees [1] http://people.ubuntu-in.org/~carthik/bugstats/ [2] http://outflux.net/ubuntu/stats/ -- Kees Cook -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From carthik at gmail.com Wed Jul 4 13:46:13 2007 From: carthik at gmail.com (Carthik Sharma) Date: Wed, 4 Jul 2007 09:46:13 -0400 Subject: Bug stat graphs In-Reply-To: <20070704103717.GU29861@outflux.net> References: <20070704103717.GU29861@outflux.net> Message-ID: <80f75db0707040646u156a7a8cgd23bd16201c4a7d7@mail.gmail.com> Hi Kees, Thank you for the new setup. I have trouble logging in to the ubuntu-in.org server which allows only for ssh-key auth. I will get that fixed soon, hopefully. Thanks again! Carthik. On 7/4/07, Kees Cook wrote: > Hi, > > Since Carthik's original Ubuntu Bug Stats[1] has been offline since the > "Unconfirmed" -> "New" change, I've put up an updated version here[2] > until his is back up and running. > > Take care, > > -Kees > > [1] http://people.ubuntu-in.org/~carthik/bugstats/ > [2] http://outflux.net/ubuntu/stats/ > > -- > Kees Cook > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.6 (GNU/Linux) > > iD8DBQFGi3hdH/9LqRcGPm0RAl57AJ4gIPmRmLbnEd8LjHQXVtKA1wTxegCfT5nC > 9A1PVcfLHEtoNdTm2Ec/5AY= > =JLV7 > -----END PGP SIGNATURE----- > > -- > Ubuntu-bugsquad mailing list > Ubuntu-bugsquad at lists.ubuntu.com > https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugsquad > > -- Ph.D. Candidate University of Central Florida Homepage: http://carthik.net From kees at ubuntu.com Wed Jul 4 14:16:40 2007 From: kees at ubuntu.com (Kees Cook) Date: Wed, 4 Jul 2007 07:16:40 -0700 Subject: Bug stat graphs In-Reply-To: <80f75db0707040646u156a7a8cgd23bd16201c4a7d7@mail.gmail.com> References: <20070704103717.GU29861@outflux.net> <80f75db0707040646u156a7a8cgd23bd16201c4a7d7@mail.gmail.com> Message-ID: <20070704141640.GV29861@outflux.net> Hi Carthik, On Wed, Jul 04, 2007 at 09:46:13AM -0400, Carthik Sharma wrote: > Thank you for the new setup. I have trouble logging in to the > ubuntu-in.org server which allows only for ssh-key auth. I will get > that fixed soon, hopefully. Excellent! If you have the gnuplot templates you're using, I could update mine to generate the plots better -- there are a few features I see in your plots that look much better than what I managed to figure out. :) Take care, -Kees -- Kees Cook -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From mathiaz at ubuntu.com Fri Jul 6 11:12:15 2007 From: mathiaz at ubuntu.com (Mathias Gug) Date: Fri, 6 Jul 2007 12:12:15 +0100 Subject: Status and Importance for "A non-trivial feature request" response Message-ID: <20070706111214.GB5932@mathias.mathiaz.net> Hi, What should be the status and importance for a bug identified as a "A non-trivial feature request" ? Thank you. -- Mathias From pochu at ubuntu.com Fri Jul 6 11:54:59 2007 From: pochu at ubuntu.com (Emilio Pozuelo Monfort) Date: Fri, 06 Jul 2007 13:54:59 +0200 Subject: Status and Importance for "A non-trivial feature request" response In-Reply-To: <20070706111214.GB5932@mathias.mathiaz.net> References: <20070706111214.GB5932@mathias.mathiaz.net> Message-ID: <468E2D93.2080705@ubuntu.com> Mathias Gug wrote: > Hi, > > What should be the status and importance for a bug identified as a "A > non-trivial feature request" ? Probably Wishlist and Confirmed, and forward it upstream (if appropriate). Cheers, Emilio > > Thank you. > > -- > Mathias > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: OpenPGP digital signature URL: From shirishag75 at gmail.com Sat Jul 7 18:48:49 2007 From: shirishag75 at gmail.com (shirish) Date: Sun, 8 Jul 2007 00:18:49 +0530 Subject: [Bug 124536] GDM login doesn't respect/stick the chosen locale Message-ID: <511f47f50707071148h7aa5b577md590ee446300d7af@mail.gmail.com> Hi all, I reported this bug https://bugs.launchpad.net/bugs/124536 . If somebody needs more info. please lemme know. If its an upstream issue then would file it but gotta know if there is some way to confirm it (some way) . -- Shirish Agarwal This email is licensed under http://creativecommons.org/licenses/by-nc/3.0/ 065C 6D79 A68C E7EA 52B3 8D70 950D 53FB 729A 8B17 From konghao at bsw.net.cn Tue Jul 17 02:19:01 2007 From: konghao at bsw.net.cn (konghao) Date: Tue, 17 Jul 2007 10:19:01 +0800 Subject: All-sides Testing Report of Ubuntu-7.04 from BSTQC Message-ID: --Title-- Release of All-sides Testing Report of Ubuntu-7.04 --Release Notice-- Thanks for your attention. BSQTC( Beijing Software Testing & QA Center ) has released the All-sides Testing Report of Ubuntu-7.04. Our engineers did the reports by their really hard work. Every issue in the report had been checked by at least 2 engineers and 3 computers to ensure the recurrence. Every description of the issues had been carefully edited to make it easily to understand and recur. As a member of Ubuntu community , BSTQC would contribute all the resource we have. --How to read the report-- We put more information in our reports to make it easily for understanding and tracking issues. It's more accessible then the report of Ubuntu-6.10. In the report file "Ubuntu 7.04 Testing Report" there are testing entries one by one which covered user document, function, reliability, usability, maintenance, portability and Chinese characteristics. You can see Page-3 of "Ubuntu 7.04 Testing Report" for details. In report file "Ubuntu 7.04 Testing Report" there are different testing results of "Pass","Pass Partly", "n-a","Fail". "Pass" means the result was obviously correct according to the User Document or experience. "Pass Partly" means the main function was correct, but the subfunction had issues. "n-a" means this entry was NOT tested for reasons. "Fail" means the result was obviously different with the expected result according to User Document or experience. When you found "Pass Partly" or "Fail" entries in file "Ubuntu 7.04 Testing Report", you would also find some words llike "bug: 0010" or "bug:0007,0008,0009". The number is the ID of the bugs we processed in our Bug Tracking System. After you got the ID number of a bug ,you can find the description in another file "Ubuntu 7.04 defect report" which contains the detailed description of every bug. For example: When you found the words "bug:0007,0008,0009", it means there are three bugs in this entry. According to the number 0007,0008,0009,in another file "Ubuntu 7.04 defect report" there are three descriptions entries named by the complete tracking ID of "Ubt-7040000007""Ubt-7040000008""Ubt-7040000009". "Ubt-704" is the ID of testing project, so you can search the "Ubuntu 7.04 defect report" by the key word of "Ubt-7040000007" or just "0000007" to find the description of the bug-0007. Every bug has different Severity Level from 1(Fatal) to 5(Suggestion) , in the file we report: Severity-Level | Number of bugs Severity-1 | 0 Severity-2 | 30 Severity-3 | 119 Severity-4 | 235 Severity-5 | 18 The detail of how we define the level at the part of "Defect Summary" in file "Ubuntu 7.04 defect report". We are still considering to improve our report format to make it easily for tracking bugs between "Testing report" and "Defect report". But unfortunately we could not report every bug one by one to the launchpad because of the resource limitation. We did the report based on the install CD of Ubuntu 7.04. Some of the bugs possible have been fixed in updates, please check them carefully. --Where to download the report-- You can download them from the directly URL : http://groups.google.com/group/bstqc/web/Ubuntu704ReportODT.tar.gz http://groups.google.com/group/bstqc/web/Ubuntu704ReportPDF.tar.gz Or ask me by sending me email with title "Asking for Ubuntu704 report" to konghao at bsw.net.cn We'll send you report with ODT format because PDF format is a little big so that some mailservers would possible filtrate it. By the email way, please make sure your mail account has permission of receiving accessory. Please contact us by sending email to konghao at bsw.net.cn when you have any question. Thanks! & Best regards! Hao Kong - BSTQC ---------------------------------------------- National Application Software Testing Labs Beijing Software Testing & QA Center Add:Building 3A,Zhongguancun Software Park, Shangdi,Haidian District,Beijing,China 100094 TEL:(+86)-010-82825511-726 Fax:(+86)-010-82826408 Email:konghao at bsw.net.cn ----------------------------------------------- From henrik at ubuntu.com Wed Jul 18 11:34:39 2007 From: henrik at ubuntu.com (Henrik Nilsen Omma) Date: Wed, 18 Jul 2007 12:34:39 +0100 Subject: All-sides Testing Report of Ubuntu-7.04 from BSTQC In-Reply-To: References: Message-ID: <469DFACF.9040300@ubuntu.com> konghao wrote: > --Title-- > > Release of All-sides Testing Report of Ubuntu-7.04 > > ... Thank you for you detailed test report. I will have a look to see how we might integrate this with our current bug tracking systems. In theory we should be able to parse the ODF (XML) file to extract information. Henrik From mpt at canonical.com Mon Jul 23 05:35:13 2007 From: mpt at canonical.com (Matthew Paul Thomas) Date: Mon, 23 Jul 2007 17:35:13 +1200 Subject: All-sides Testing Report of Ubuntu-7.04 from BSTQC In-Reply-To: <469F7A7A.9040009@ubuntu.com> References: <469F7A7A.9040009@ubuntu.com> Message-ID: <06eb0a9e27934d289c99e68a0ad6ea93@canonical.com> On Jul 20, 2007, at 2:51 AM, Henrik Nilsen Omma wrote: > > konghao wrote: > ... >> That is the key problem , actually, the report was generated >> automatically by our local tracking system, but we didn't find a easy >> way of shifting bugs to launchpad. > ... > We do not have a shortage of bug reports, but rather of well defined > reports. Bugs should have the right package assigned and duplicates > should be marked. We also have more benefit from bugs filed against the > development version (now Gutsy) > ... As I suggested last time, perhaps the Ubuntu BugSquad could organize a Bi-Annual Bug Blitz -- putting the BSTQC report on the Ubuntu wiki, and annotating it with links to the equivalent Launchpad bug reports. Cheers -- Matthew Paul Thomas http://mpt.net.nz/ -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part URL: From konghao at bsw.net.cn Mon Jul 23 07:40:45 2007 From: konghao at bsw.net.cn (konghao) Date: Mon, 23 Jul 2007 15:40:45 +0800 Subject: All-sides Testing Report of Ubuntu-7.04 from BSTQC In-Reply-To: <06eb0a9e27934d289c99e68a0ad6ea93@canonical.com> Message-ID: On Jul 20, 2007, at 2:51 AM, Henrik Nilsen Omma wrote: > > konghao wrote: > ... >> That is the key problem , actually, the report was generated >> automatically by our local tracking system, but we didn't find a easy >> way of shifting bugs to launchpad. > ... > We do not have a shortage of bug reports, but rather of well defined > reports. Bugs should have the right package assigned and duplicates > should be marked. We also have more benefit from bugs filed against the > development version (now Gutsy) > ... >As I suggested last time, perhaps the Ubuntu BugSquad could organize a >Bi-Annual Bug Blitz -- putting the BSTQC report on the Ubuntu wiki, and >annotating it with links to the equivalent Launchpad bug reports. > Thank you for your reply, if Bi-Annual Bug Blitz is a better way to handle the report, I would have a try. Could you introduce someone who are in charge of it for me? And are there some documents of how to do it? or just paste the report on Ubuntu wiki? Regards Konghao - BSTQC From henrik at ubuntu.com Mon Jul 23 10:46:07 2007 From: henrik at ubuntu.com (Henrik Nilsen Omma) Date: Mon, 23 Jul 2007 11:46:07 +0100 Subject: All-sides Testing Report of Ubuntu-7.04 from BSTQC In-Reply-To: References: Message-ID: <46A486EF.9010909@ubuntu.com> konghao wrote: > >> As I suggested last time, perhaps the Ubuntu BugSquad could organize a >> Bi-Annual Bug Blitz -- putting the BSTQC report on the Ubuntu wiki, and >> annotating it with links to the equivalent Launchpad bug reports. >> >> Perhaps someone from the bugsquad could volunteer to have a look at the S2 bugs in the report (themost serious -- there are no S1 bugs). That would give us an idea of whether serious bugs are being found that do not already appear in LP and whether the provided information is useful. > > Thank you for your reply, if Bi-Annual Bug Blitz is a better way to handle the report, I would have a try. Could you introduce someone who are in charge of it for me? And are there some documents of how to do it? or just paste the report on Ubuntu wiki? A bi-annual bug blitz could be useful if we get enough people to participate and esp. if the bugs are filed against the development version. You can contact me with further questions about this. I lead the Canonical Ubuntu QA team. Henrik From brian at ubuntu.com Thu Jul 26 04:05:13 2007 From: brian at ubuntu.com (Brian Murray) Date: Wed, 25 Jul 2007 21:05:13 -0700 Subject: Team Wiki Pages Message-ID: <20070726040513.GT17409@murraytwins.com> I've recently put some effort into redesigning the Bug Squad wiki pages[1]. My intent was to make the pages more approachable for people new to bug triaging, and making each page a bit shorter. I realize the some content is sparse at the moment, FAQ and Roadmap, but wanted to get feedback and invite you to help make it better! Also keep an eye out for syntax like "[[Include(BugSquad/Header)]]" when editing the wiki pages. That particular statement just includes the content from BugSquad/Header. However, on more than one occasion[2] I have used syntax like: [[Include(Bugs/Responses, , from="^== Incomplete bugs without a response from submitter ==", to="==")]] This just includes part of the content of the Bugs/Responses page, and my intent with that was to make it so readers didn't always have to go to another wiki page to find information. I hope you find it more usable and look forward to your input. [1] https://wiki.ubuntu.com/BugSquad [2] https://wiki.ubuntu.com/Bugs/Status -- Brian Murray @ubuntu.com -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: