Kernel Crash Dumps

[Joseph Salisbury]

On 03/03/2011 07:34 PM, Brian Murray wrote:
> On Thu, Mar 03, 2011 at 06:26:18PM -0500, Joseph Salisbury wrote:
>> On 03/03/2011 04:46 PM, Brian Murray wrote:
>>> In reviewing the kernel debugging documentation I came across the
>>> instructions for saving a crash dump for a kernel panic[1].  I tried
>>> following them and I'm unable to generate a crash dump.  Additionally,
>>> that page hadn't been touched in quite some time.  I've also found a bug
>>> report[2] about another person having an issue getting crash dumps.
>>>
>>> So my question is - is there something wrong with the documentation?
>>> the crash dump process? or me?
>>>
>>>
>>> [1] https://wiki.ubuntu.com/Kernel/CrashdumpRecipe
>>> [2] https://bugs.launchpad.net/ubuntu/+source/linux-meta/+bug/710733
>>>
>>> Thanks,
>>>
>>
>> I was able to generate a crash dump on a desktop running Natty.
>
> How?  Your last comment in the bug report mentions intermittent failures
> and maxsize is no longer used by apport so that shouldn't have any
> effect.  I'd really like to get the documentation and or the process
> squared away.
>
>
Hi Brian,

I performed some more testing today on a Natty desktop.  I'm able to 
generate a crash dump.  However, I've been getting intermittent 
failures.  Like you mention, changing maxsize doesn't seem to help.  It 
was just a coincidence that crash dump worked for the first time, after 
I increased maxsize.  In all the failures, the system hangs performing 
the following during the dump file creation:

"Copying data		: [N%] <- The percentage when the hang happens varies.

I had to perform some steps in addition to what's listed on the 
CrashdumpRecipe wiki.  To get crash dump working(Although 
intermittently), I performed the following:


1.  Installed linux-crashdump and kdump-tools.
	- Should it be necessary to install kdump-tools?  Without kdump-tools, 
I see the following in /var/crash/vmcore.log:

"/root/usr/bin/makedumpfile: error while loading shared libraries: 
libdw.so.1: cannot open shared object file: No such file or directory"

	- I noticed makedumpfile lives in /usr/bin/ and not /root/usr/bin.
	- I tried creating a sym link in /root/usr/bin to point to the real 
makedumpfile in /usr/bin, but I still got the same error.
	- I performed an ldd on makedumpfile in /usr/bin, and all the libraries 
where found.
	- Again, I tried these things before I installed kdump-tools.  Once 
kdump-tools is installed, the lib load error goes away.

2.  I manually created the /var/crash directory.

3.  Edited /etc/default/apport; Changed enabled from 0 to 1.

4.  Edited /etc/default/kdump-tools:
	- Changed USE_KDUMP from 0 to 1.
	- Uncommented: #KDUMP_SYSCTL="kernel.panic_on_oops=1"
	- Without kdump-tools installed, this file doesn't exist.

5.  Edited /etc/default/kexec.  Changed LOAD_KEXEC from false to true, 
but this didn't seem to make a difference.

6.  Removed 'quiet splash' from the boot parameters(So I could see where 
it was hanging).


To trigger a panic, I perform:
echo c | sudo tee /proc/sysrq-trigger

I'll keep experimenting, and see if I can provide some more information. 
  I'll also update the bug with this latest info.

Thanks,

Joe