[Ubuntu-bugcontrol] Inquiry on Improving https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Reporting_Bugs_Upstream

Sun Jul 29 12:15:29 UTC 2012

* Christopher Penalver (christopher.penalver at gmx.com) wrote:
> > ----- Original Message -----
> > From: Dr. David Alan Gilbert
> > Sent: 07/28/12 05:04 PM
> > To: Christopher Penalver
> > Subject: Re: [Ubuntu-bugcontrol] Inquiry on Improving https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Reporting_Bugs_Upstream
> > 
> > * Christopher Penalver (christopher.penalver at gmx.com) wrote:
> > > Ubuntu Bug Control / Ubuntu Kernel Team,
> > > 
> > > Hello everyone. I wanted to inquire about improving
> > > https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Reporting_Bugs_Upstream
> > > . What I have noticed in my triaging linux package bugs is that the high
> > > majority of original reporters who are steered to this article when
> > > asked to upstream their bug, do not follow the directions at all. In
> > > turn, their bugs are promptly ignored, and by kindness/busy'ness of the
> > > maintainer/submaintainer/community developers, not closed or flagged
> > > REJECTED INSUFFICIENT_DATA.
> > 
> > I don't think we should be steering people to report to bugzilla.kernel.org
> > at all.
> 
> David, thank you for responding.
> 
> > For a large part if you report a bug in a distro packaged kernel
> > to upstream kernel you'll be sent away with a flea in your ear.
> 
> If anyone did not test the mainline kernel and posts to BZKO, it's most likely going to get ignored, or shutdown with impunity.
> As well, nobody from the Ubuntu Kernel Team/Ubuntu Bug Control is steering Original Reporters to BZKO unless the problem is proven to exist in mainline, as per well documented UKT/UBC policies https://wiki.ubuntu.com/KernelTeam/KernelTeamBugPolicies#Reporting_Bugs_Upstream .
> 
> > Even then, there is some question whether bugzilla.kernel.org is intended
> > to take user bug reports these days (e.g. see https://lkml.org/lkml/2012/5/13/121 )
> 
> Now that you mention this, I did notice Greg Kroah-Hartman shutdown a handful of Linux USB bugs forwarded from Launchpad a few months ago, which I personally triaged and referred to BZKO as the bug existed in mainline. However, the ORs did not follow the BZKO format that was specifically noted in my comment, referring them to the current Reporting Bugs Upstream article iteration. Greg advised them to report their issue to the USB maintainer mailing list. Unfortunately, he did not mention the rationale for this action. I mean, come on, BZKO is a publicly viewable and postable _Bugzilla_. ;)

Right, but different orgs use bug trackers in different ways; and even people
within orgs.  The Linux kernel itself is such a mix of different people and
maintainers that they each have their own view on how to do things.  Personally
if I hit a kernel bug I can trace to upstream I'd mail the maintainer of
that part of the kernel (having looked it up in the MAINTAINERS file) cc'ing
lkml and the appropriate mailing list for that section giving all the
appropriate information for that type of problem, and then hope for some pointers.
But I can't provide a list of rules on how to do that, it's more gut feel and experience.

> In spite of this, we can guess with a reasonable amount of certainty that the shutdowns happened because the ORs did not follow the BZKO format. In response to the GKH shutdowns, I stepped up the intensity of my wording in my comments to ORs, which got ~2/40 ORs to do it, increasing my prior batting average from 0%. :D

OK I don't know which specific bugs you are referring to; but I doubt
it's specific format they're worrying about; a missing piece of information
relevant to the particular problem is more likely - and that standard
format seems to include most stuff for common problems.

Greg KH is in general pity nice and good; he has got annoyed by some
stuff referred via Ubuntu in the past though ( https://plus.google.com/111049168280159033135/posts/KQEEdn5X7XT ) -
although note a bunch of other devs followed up and disagreed.

> This is the driving factor for wanting to see the article improved by stepping up the intensity of the article wording and increasing the verbosity of explanation. I do not want to receive angst by proxy when I add myself to the CC.
> Also, my intended improvement proved true, "Failure to follow these directions exactly as shown... may likely result in your bug being ... closed...". The ORs rolled the die and lost.
> Ultimately, if BZKO developers decide mailinglist first only, then I agree falling in line with their request is the best way to go.

> 
> > I certainly don't think we should ever do it unless it's been pretty heavily
> > triaged to not be an ubuntu bug.
> 
> This is current UKT/UBC policy.
> 
> > 
> > I also think it's a bad idea to ask end users to think about
> > providing the information needed;
> 
> I do not agree that asking them to think about providing information is a bad idea. All the UKT/UBC documented responses do is politely ask, not demand, ORs for the developer "need to haves" to work on the bug downstream. Testing mainline is a "nice to have" providing the Ubuntu developer(s) more data points and opening up the OR to having more eyeballs look at their bug by BZKO devs. The current UKT/UBC debugging articles, my past article improvements, and my current improvment suggestions are focused on "copy from the article, paste into a terminal, hit enter, and post results into a comment" transactional debugging, that anyone from a child to a senior citizen could accomplish with little to no familiarity with a computer.
> Some of the articles include more in-depth analysis on the transactional debugging, yet this exists for the technically inclined and curious. None are expecting an OR to whip up a patch because an analysis exists.

I agree; the only thing I'd say is just make sure that any messages aimed at
the end user are very clear about that it is voluntary, and never to
set to incomplete if the user hasn't done it.

> > lots of people who report their machine
> > has oops'd or crashed won't have the technical skills to try the things
> > needed.
> 
> The BZKO format part:
> [5.] Output of Oops.. message (if applicable) with symbolic information resolved (see Documentation/oops-tracing.txt)
> 
> is potentially daunting. More work would need to be done to transactionalize it. None the less, this transactionalization seems not to be a show stopper for the current suggested improvement being implimented, but something the Ubuntu community could work on adding in a near future improvement.
> 
> > I wouldn't want either of:
> > 
> >  1) End users to think Ubuntu wouldn't take a bad kernel bug
> > seriously unless they can jump through all of these type of hoops.
> 
> Currently, by the time an OR is asked to read the UKT upstream article, they have already apport-collect'ed in >= Precise, proved the bug is in mainline, and provided all relevant information from https://wiki.ubuntu.com/DebuggingProcedures . The BZKO format is to prepare any report from any distro to be posted on BZKO so a BZKO dev won't ignore/shutdown the report.
> 
> >  2) Kernel guys to get annoyed with Ubuntu for pointing our
> > users at them.
> 
> One could make the same arguement about any distro being pointed to BZKO if something was demonstrated in mainline. Never the less, nobody does this as per UKT/UBC policy. Launchpad is the only forum for Ubuntu users to get Ubuntu bugs fixed. BZKO is for mainline kernel bugs only. No confusion exists about this.
> 
> > I mean after all, the most likely positive response from the kernel guys is
> > to ask for you to try a patch, 
> 
> Positive outcomes are what happens when a Ubuntu OR follows the BZKO format:
> https://bugzilla.kernel.org/show_bug.cgi?id=42993

OK, that's good.

> Although, I did skip a couple parts. The ooptext part, which luckily was not relevant, and:
> [7.1.] Software (add the output of the ver_linux script here)

But that's kind of what I mean by it's more about the 'information relavant to the bug'
rather than necessarily exactly following the guide that's important.

> which I skipped accidently out of ignorance. Again, luckily it was irrelevant.
> 
> > and unless the reporter can rebuild
> > there own kernel, then I'm not sure there is much of a point.
> 
> I did not consider this. Your saying instead of using the mainline prebuilt binaries, build it from kernel.org source tarballs on the ORs computer? While I am not familiar with all the technical nuances of using the prebuilts versus building from source,
> I do approve of and trust the current UKT/UBC policy in that if you test the prebuilts it's sufficient for BZKO, it's sufficient. So far, it has been when the upstream article is _actually_ followed.

Well I don't think it necessarily matters if someone tests a prebuilt binary (as
long as it is exactly an upstream source without patches, and as long as you can
tell the upstream guys what version of there source it referes to).

But my point is really that when they report it upstream it's pretty likely
they'll get given a patch to try (e.g. in your brightness one, they did manage
to get a lot of useful info from you and give some boot options - so that's good;
but it wouldn't be too unusual for it to end with a 'try this patch'.

> The problem is ORs are not following the article. Not, the article is wrong in sending them to BZKO in the first place.
> With this in mind, if the improved article cannnot better persuade ORs to follow it in very high percentages, the next step could be your first point how ORs should not be sent in the first place. A slightly more labor intensive, and considerably more productive solution would be for Ubuntu triagers/developers to further triage the upstream forwarded BZKO reports (not the downstream report) found not to follow the BZKO format. This would allay any BZKO developer frustration with ORs who do not follow the BZKO format.
> While friviously shutting down bugs because the OR made a mistake for not providing the report in the BZKO format, especially when the maintainer/sub/community dev is subscribed to both the BZKO and the mailing list, is not very cool, let alone Ubuntu, disregarding the BZKO format when posting to BZKO, as well as not reasonably facilitating as many eyeballs on a Ubuntu community member's bug as possible, for whatever reason(s), is certainly not cool/Ubuntu.
> Summing this part up, your point about how current policy does not request of them to build from source seems to not be a show stopper for the current suggested improvement, but something to consider in a future improvement via a new thread.

Yeh I don't think being able to build from source is critical; as long
as any report upstream shows they've tested with a particular upstream
version that they can easily reference (ideally either a GIT version
or a specific release version).

> > Now, having said that, if someone wants to take it further and has
> > tried the latest daily build etc, and has the skills then your page
> > probably sets the right tone to make sure they don't do it without
> > providing the right info.
> > 
> > > What seems best is something to the following:
> > > 
> > > <START>
> > > 
> > > Thank you for following this. Firstly, please do not report
> > > your bug upstream unless you have followed the below mentioned
> > > format created by kernel.org developers. Failure to follow these
> > > directions exactly as shown below, may likely result in your bug
> > > being promptly ignored by the kernel maintainer or submaintainer
> > > who could fix your problem, or the report being be closed as
> > > [[https://bugzilla.kernel.org/page.cgi?id=faq.html|REJECTED
> > > INSUFFICIENT_DATA]].
> > 
> > 
> > > ||<tablestyle="background-color: #eee"> <<BR>> [1.] One line summary of the problem: <<BR>> Paste the downstream bug title. <<BR>> <<BR>> [2.] Full description of the problem/report: <<BR>> Paste the Bug Description. <<BR>> <<BR>> [3.] Keywords (i.e., modules, networking, kernel): <<BR>> [4.] Kernel version (from /proc/version): <<BR>> [5.] Output of Oops.. message (if applicable) with symbolic information resolved (see Documentation/oops-tracing.txt) <<BR>> If you have a kernel oops, one may consult http://www.kernel.org/doc/Documentation/oops-tracing.txt . <<BR>> [6.] A small shell script or example program which triggers the problem (if possible) <<BR>> [7.] Environment <<BR>> lsb_release -rd <<BR>> <<BR>> [7.1.] Software (add the output of the ver_linux script here) <<BR>> For Ubuntu, this is found in the directory: <<BR>> /usr/src/linux-headers-<VERSION>/scripts <<BR>> <<BR>> where <VERSION> is the version of the kernel you are using, found in the directory /usr/src. You may run the script via a ter
> mina
> > > l: <<BR>> sh ver_linux <<BR>> <<BR>> [7.2.] Processor information (from /proc/cpuinfo): <<BR>> [7.3.] Module information (from /proc/modules): <<BR>> [7.4.] Loaded driver and hardware information (/proc/ioports, /proc/iomem) <<BR>> [7.5.] PCI information ('lspci -vvv' as root) <<BR>> [7.6.] SCSI information (from /proc/scsi/scsi) <<BR>> [7.7.] Other information that might be relevant to the problem (please look in /proc and include all information that you think to be relevant): <<BR>> List the output of /proc. <<BR>> <<BR>> [X.] Other notes, patches, fixes, workarounds: <<BR>> Be 100% certain you have included all relevant comment information not included in the Bug Description and a kernel developer should review. They are very busy, and do not need to dive through downstream bug reports to find something that should have been provided when the bug was first filed. If you are unsure about anything asked for, or intend on not following this format, please do not file a report. Instead, continue to ask q
> uest
> > > ions in Launchpad and of the Ubuntu community until all issues are cleared up. ||
> > > 
> > > Note the above format was taken directly from [[http://kernel.org/pub/linux/docs/lkml/reporting-bugs.html]].
> > 
> > It probably also needs something to check whether the kernel you're running is
> > tainted; and that if even vaguely possible you must try without any closed
> > source modules loaded.
> 
> I do not think this is part of current UKT/UBC forwarding policy. Be
> that as it may, this seems not a show stopper, but something to discuss
> for future improvement in a new thread. On this point, one would want
> some upstream documentation on forwarding policies to educate ORs.

I think if people forward bugs (especially crashes) which have binary
modules loaded they'll get kicked back very quickly.

> Thanks again for responding.

No problem!

Dave

> MCSE:Security, MCSA, MCDST, MCP, Security+, Network+, A+, NSSP
> 
> This E-Mail was sent via a laptop using Xubuntu, Linux for human beings.

Now there is an unusual combination of lines.

Dave
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\ gro.gilbert @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/