[Bug 932687] Re: nfs4_reclaim_locks: unhandled error crashes applications and creates high load
Bryan Quigley
gQuigs at gMail.com
Tue May 21 18:52:39 UTC 2013
If you can distill it to an easy to rebuild setup that would be very
helpful (that isn't dependent on AWS). Does the issue happen if you
scale up even if there isn't load? Or, in other words, is one of your
scripts or apps important for reproducing this issue? If so, can it be
narrowed down to something you can share?
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to nfs-utils in Ubuntu.
https://bugs.launchpad.net/bugs/932687
Title:
nfs4_reclaim_locks: unhandled error crashes applications and creates
high load
Status in “linux” package in Ubuntu:
Incomplete
Status in “nfs-utils” package in Ubuntu:
Invalid
Bug description:
We tried to move our Natty clients to Oneiric but have a severe show-
stopper bug. Oneiric seems to have a problem with nfs. We use nfs for
our home-folders with strict permissions. nfs4-server is running
solaris.
As said no problem on natty, just with oneiric. Oneiric is running
fine some minutes, then in the dmesg we get such output:
[ 7778.934514] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7778.934521] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7869.899811] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7869.899818] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7869.938180] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7869.938184] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7869.950989] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7869.950993] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7869.977253] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7869.977258] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7870.364422] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7870.364429] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7870.594833] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7870.594839] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7870.652639] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7870.652644] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7870.678166] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7870.678171] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7880.217148] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7880.217155] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7880.277521] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7880.277527] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7880.374106] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7880.374113] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7880.440398] nfs4_reclaim_locks: unhandled error -10024. Zeroing state
[ 7880.440404] nfs4_reclaim_open_state: Lock reclaim failed!
[ 7880.451121] nfs4_reclaim_open_state: unhandled error -10026. Zeroing state
[ 7880.451330] nfs4_reclaim_open_state: unhandled error -10026. Zeroing state
[ 7880.451520] nfs4_reclaim_open_state: unhandled error -10026. Zeroing state
[ 7880.451738] nfs4_reclaim_open_state: unhandled error -10026. Zeroing state
[ 7880.451921] nfs4_reclaim_open_state: unhandled error -10026. Zeroing state
[ 7880.452099] nfs4_reclaim_open_state: unhandled error -10026. Zeroing state
[ 7880.452279] nfs4_reclaim_open_state: unhandled error -10026. Zeroing state
[ 8160.252077] INFO: task claws-mail:23156 blocked for more than 120 seconds.
[ 8160.252085] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8160.252092] claws-mail D 0000000000000000 0 23156 1848 0x00000004
[ 8160.252103] ffff8800363e3a08 0000000000000046 ffff8800363e39a8 ffffffffa03275f0
[ 8160.252109] ffff8800363e3fd8 ffff8800363e3fd8 ffff8800363e3fd8 0000000000012a40
[ 8160.252115] ffff8800191edc80 ffff88003d7cdc80 ffff8800363e39e8 ffff88003fc132c0
[ 8160.252121] Call Trace:
[ 8160.252148] [<ffffffffa03275f0>] ? rpc_put_task+0x10/0x20 [sunrpc]
[ 8160.252158] [<ffffffff8110a180>] ? __lock_page+0x70/0x70
[ 8160.252164] [<ffffffff815eff1f>] schedule+0x3f/0x60
[ 8160.252168] [<ffffffff815effcf>] io_schedule+0x8f/0xd0
[ 8160.252173] [<ffffffff8110a18e>] sleep_on_page+0xe/0x20
[ 8160.252177] [<ffffffff815f07ef>] __wait_on_bit+0x5f/0x90
[ 8160.252182] [<ffffffff8110a378>] wait_on_page_bit+0x78/0x80
[ 8160.252189] [<ffffffff81081c50>] ? autoremove_wake_function+0x40/0x40
[ 8160.252194] [<ffffffff8110a48c>] filemap_fdatawait_range+0x10c/0x1a0
[ 8160.252216] [<ffffffffa03be1d0>] ? nfs_writedata_alloc+0x150/0x150 [nfs]
[ 8160.252233] [<ffffffffa03b89e0>] ? nfs_free_request+0x90/0x90 [nfs]
[ 8160.252243] [<ffffffff81115211>] ? do_writepages+0x21/0x40
[ 8160.252252] [<ffffffff8110bd5b>] ? __filemap_fdatawrite_range+0x5b/0x60
[ 8160.252261] [<ffffffff8110bdc8>] filemap_write_and_wait_range+0x68/0x80
[ 8160.252271] [<ffffffff811940e2>] vfs_fsync_range+0x42/0xa0
[ 8160.252277] [<ffffffff811941ac>] vfs_fsync+0x1c/0x20
[ 8160.252295] [<ffffffffa03ad2e3>] nfs_file_flush+0x53/0x80 [nfs]
[ 8160.252301] [<ffffffff811661ff>] filp_close+0x3f/0x90
[ 8160.252307] [<ffffffff81060f3a>] put_files_struct.part.14+0x7a/0xe0
[ 8160.252312] [<ffffffff81062a08>] put_files_struct+0x18/0x20
[ 8160.252316] [<ffffffff81062ad4>] exit_files+0x54/0x70
[ 8160.252320] [<ffffffff81062fed>] do_exit+0x19d/0x440
[ 8160.252325] [<ffffffff8107186a>] ? __dequeue_signal+0x6a/0xb0
[ 8160.252330] [<ffffffff81063434>] do_group_exit+0x44/0xa0
[ 8160.252334] [<ffffffff8107406d>] get_signal_to_deliver+0x27d/0x3f0
[ 8160.252340] [<ffffffff8100a7e6>] do_signal+0x56/0x180
[ 8160.252348] [<ffffffff8104e94d>] ? set_next_entity+0x9d/0xb0
[ 8160.252352] [<ffffffff8104e5e9>] ? finish_task_switch+0x49/0xf0
[ 8160.252356] [<ffffffff815ef8c4>] ? __schedule+0x3d4/0x700
[ 8160.252361] [<ffffffff8100aad5>] do_notify_resume+0x65/0x80
[ 8160.252368] [<ffffffff815fa490>] int_signal+0x12/0x17
[ 8280.252073] INFO: task firefox:5030 blocked for more than 120 seconds.
[ 8280.252080] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8280.252087] firefox D ffffffff81805120 0 5030 1 0x00000004
[ 8280.252097] ffff88000bfc7a08 0000000000000046 ffff88001156adf0 ffff88000bfc7b98
[ 8280.252105] ffff88000bfc7fd8 ffff88000bfc7fd8 ffff88000bfc7fd8 0000000000012a40
[ 8280.252111] ffffffff81c0b020 ffff88003645c560 ffff88000bfc79e8 ffff88003fc132c0
[ 8280.252117] Call Trace:
[ 8280.252130] [<ffffffff8110a180>] ? __lock_page+0x70/0x70
[ 8280.252137] [<ffffffff815eff1f>] schedule+0x3f/0x60
[ 8280.252141] [<ffffffff815effcf>] io_schedule+0x8f/0xd0
[ 8280.252145] [<ffffffff8110a18e>] sleep_on_page+0xe/0x20
[ 8280.252150] [<ffffffff815f07ef>] __wait_on_bit+0x5f/0x90
[ 8280.252154] [<ffffffff8110a378>] wait_on_page_bit+0x78/0x80
[ 8280.252165] [<ffffffff81081c50>] ? autoremove_wake_function+0x40/0x40
[ 8280.252170] [<ffffffff8110a48c>] filemap_fdatawait_range+0x10c/0x1a0
[ 8280.252177] [<ffffffff81115211>] ? do_writepages+0x21/0x40
[ 8280.252181] [<ffffffff8110bd5b>] ? __filemap_fdatawrite_range+0x5b/0x60
[ 8280.252186] [<ffffffff8110bdc8>] filemap_write_and_wait_range+0x68/0x80
[ 8280.252192] [<ffffffff811940e2>] vfs_fsync_range+0x42/0xa0
[ 8280.252196] [<ffffffff811941ac>] vfs_fsync+0x1c/0x20
[ 8280.252217] [<ffffffffa03ad2e3>] nfs_file_flush+0x53/0x80 [nfs]
[ 8280.252223] [<ffffffff811661ff>] filp_close+0x3f/0x90
[ 8280.252229] [<ffffffff81060f3a>] put_files_struct.part.14+0x7a/0xe0
[ 8280.252233] [<ffffffff81062a08>] put_files_struct+0x18/0x20
[ 8280.252237] [<ffffffff81062ad4>] exit_files+0x54/0x70
[ 8280.252243] [<ffffffff81062fed>] do_exit+0x19d/0x440
[ 8280.252251] [<ffffffff8107186a>] ? __dequeue_signal+0x6a/0xb0
[ 8280.252260] [<ffffffff81063434>] do_group_exit+0x44/0xa0
[ 8280.252268] [<ffffffff8107406d>] get_signal_to_deliver+0x27d/0x3f0
[ 8280.252277] [<ffffffff8100a7e6>] do_signal+0x56/0x180
[ 8280.252285] [<ffffffff811afe27>] ? fcntl_setlk+0x67/0x220
[ 8280.252294] [<ffffffff81178e42>] ? do_fcntl+0x1b2/0x340
[ 8280.252302] [<ffffffff8100aad5>] do_notify_resume+0x65/0x80
[ 8280.252311] [<ffffffff815fa490>] int_signal+0x12/0x17
[ 8280.252322] INFO: task claws-mail:23156 blocked for more than 120 seconds.
[ 8280.252328] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8280.252333] claws-mail D 0000000000000000 0 23156 1848 0x00000004
[ 8280.252342] ffff8800363e3a08 0000000000000046 ffff8800363e39a8 ffffffffa03275f0
[ 8280.252350] ffff8800363e3fd8 ffff8800363e3fd8 ffff8800363e3fd8 0000000000012a40
[ 8280.252360] ffff8800191edc80 ffff88003d7cdc80 ffff8800363e39e8 ffff88003fc132c0
[ 8280.252368] Call Trace:
[ 8280.252395] [<ffffffffa03275f0>] ? rpc_put_task+0x10/0x20 [sunrpc]
[ 8280.252404] [<ffffffff8110a180>] ? __lock_page+0x70/0x70
[ 8280.252412] [<ffffffff815eff1f>] schedule+0x3f/0x60
[ 8280.252419] [<ffffffff815effcf>] io_schedule+0x8f/0xd0
[ 8280.252427] [<ffffffff8110a18e>] sleep_on_page+0xe/0x20
[ 8280.252434] [<ffffffff815f07ef>] __wait_on_bit+0x5f/0x90
[ 8280.252442] [<ffffffff8110a378>] wait_on_page_bit+0x78/0x80
[ 8280.252451] [<ffffffff81081c50>] ? autoremove_wake_function+0x40/0x40
[ 8280.252459] [<ffffffff8110a48c>] filemap_fdatawait_range+0x10c/0x1a0
[ 8280.252476] [<ffffffffa03be1d0>] ? nfs_writedata_alloc+0x150/0x150 [nfs]
[ 8280.252491] [<ffffffffa03b89e0>] ? nfs_free_request+0x90/0x90 [nfs]
[ 8280.252495] [<ffffffff81115211>] ? do_writepages+0x21/0x40
[ 8280.252500] [<ffffffff8110bd5b>] ? __filemap_fdatawrite_range+0x5b/0x60
[ 8280.252505] [<ffffffff8110bdc8>] filemap_write_and_wait_range+0x68/0x80
[ 8280.252509] [<ffffffff811940e2>] vfs_fsync_range+0x42/0xa0
[ 8280.252513] [<ffffffff811941ac>] vfs_fsync+0x1c/0x20
[ 8280.252524] [<ffffffffa03ad2e3>] nfs_file_flush+0x53/0x80 [nfs]
[ 8280.252529] [<ffffffff811661ff>] filp_close+0x3f/0x90
[ 8280.252534] [<ffffffff81060f3a>] put_files_struct.part.14+0x7a/0xe0
[ 8280.252538] [<ffffffff81062a08>] put_files_struct+0x18/0x20
[ 8280.252542] [<ffffffff81062ad4>] exit_files+0x54/0x70
[ 8280.252546] [<ffffffff81062fed>] do_exit+0x19d/0x440
[ 8280.252550] [<ffffffff8107186a>] ? __dequeue_signal+0x6a/0xb0
[ 8280.252555] [<ffffffff81063434>] do_group_exit+0x44/0xa0
[ 8280.252561] [<ffffffff8107406d>] get_signal_to_deliver+0x27d/0x3f0
[ 8280.252570] [<ffffffff8100a7e6>] do_signal+0x56/0x180
[ 8280.252578] [<ffffffff8104e94d>] ? set_next_entity+0x9d/0xb0
[ 8280.252586] [<ffffffff8104e5e9>] ? finish_task_switch+0x49/0xf0
[ 8280.252591] [<ffffffff815ef8c4>] ? __schedule+0x3d4/0x700
[ 8280.252596] [<ffffffff8100aad5>] do_notify_resume+0x65/0x80
[ 8280.252601] [<ffffffff815fa490>] int_signal+0x12/0x17
[ 8280.252608] INFO: task firefox:15992 blocked for more than 120 seconds.
[ 8280.252613] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8280.252618] firefox D ffffffff81805120 0 15992 1728 0x00000004
[ 8280.252627] ffff8800362b9c18 0000000000000086 ffff880011569d70 ffff8800362b9da8
[ 8280.252636] ffff8800362b9fd8 ffff8800362b9fd8 ffff8800362b9fd8 0000000000012a40
[ 8280.252645] ffff88003d698000 ffff88003d662e40 ffff8800362b9bf8 ffff88003fd132c0
[ 8280.252654] Call Trace:
[ 8280.252661] [<ffffffff8110a180>] ? __lock_page+0x70/0x70
[ 8280.252665] [<ffffffff815eff1f>] schedule+0x3f/0x60
[ 8280.252668] [<ffffffff815effcf>] io_schedule+0x8f/0xd0
[ 8280.252673] [<ffffffff8110a18e>] sleep_on_page+0xe/0x20
[ 8280.252676] [<ffffffff815f07ef>] __wait_on_bit+0x5f/0x90
[ 8280.252681] [<ffffffff8110a378>] wait_on_page_bit+0x78/0x80
[ 8280.252685] [<ffffffff81081c50>] ? autoremove_wake_function+0x40/0x40
[ 8280.252690] [<ffffffff8110a48c>] filemap_fdatawait_range+0x10c/0x1a0
[ 8280.252694] [<ffffffff81115211>] ? do_writepages+0x21/0x40
[ 8280.252699] [<ffffffff8110bd5b>] ? __filemap_fdatawrite_range+0x5b/0x60
[ 8280.252703] [<ffffffff8110a54b>] filemap_fdatawait+0x2b/0x30
[ 8280.252707] [<ffffffff8110c7b4>] filemap_write_and_wait+0x44/0x60
[ 8280.252726] [<ffffffffa03b06d5>] nfs_getattr+0x105/0x120 [nfs]
[ 8280.252735] [<ffffffff8116c8fe>] vfs_getattr+0x4e/0x80
[ 8280.252741] [<ffffffff8116c988>] vfs_fstatat+0x58/0x70
[ 8280.252745] [<ffffffff8116c9db>] vfs_stat+0x1b/0x20
[ 8280.252748] [<ffffffff8116cb1a>] sys_newstat+0x1a/0x40
[ 8280.252752] [<ffffffff811695e5>] ? fput+0x25/0x30
[ 8280.252756] [<ffffffff8100b705>] ? math_state_restore+0x45/0x60
[ 8280.252762] [<ffffffff815f312e>] ? do_device_not_available+0xe/0x10
[ 8280.252769] [<ffffffff815fb17b>] ? device_not_available+0x1b/0x20
[ 8280.252778] [<ffffffff815fa1c2>] system_call_fastpath+0x16/0x1b
The related applications like firefox or claws-mail don't react anymore, killing them results in zombie-processes..
---
AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
AplayDevices:
**** List of PLAYBACK Hardware Devices ****
card 0: Intel [HDA Intel], device 0: ALC260 Analog [ALC260 Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
ApportVersion: 1.23-0ubuntu4
Architecture: amd64
ArecordDevices:
**** List of CAPTURE Hardware Devices ****
card 0: Intel [HDA Intel], device 0: ALC260 Analog [ALC260 Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
AudioDevicesInUse:
USER PID ACCESS COMMAND
/dev/snd/controlC0: sebastian 1782 F.... xfce4-volumed
sebastian 1800 F.... pulseaudio
sebastian 1811 F.... xfce4-mixer-plu
CRDA: Error: [Errno 2] No such file or directory
Card0.Amixer.info:
Card hw:0 'Intel'/'HDA Intel at 0xf8000000 irq 41'
Mixer name : 'Realtek ALC260'
Components : 'HDA:10ec0260,17348601,00100400'
Controls : 18
Simple ctrls : 10
CurrentDmesg: Error: command ['sh', '-c', 'dmesg | comm -13 --nocheck-order /var/log/dmesg -'] failed with exit code 1: comm: /var/log/dmesg: Permission denied
DistroRelease: Ubuntu 11.10
IwConfig: Error: [Errno 2] No such file or directory
Lsusb: Error: [Errno 2] No such file or directory
MachineType: FUJITSU SIEMENS D2151-A1
Package: nfs-utils
ProcEnviron:
PATH=(custom, user)
SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.0.0-15-generic root=UUID=e35df589-f3cc-42a9-bff5-0c2cb1a7c0c2 ro quiet
ProcVersionSignature: Ubuntu 3.0.0-15.26-generic 3.0.13
RfKill: Error: [Errno 2] No such file or directory
Tags: oneiric
UdevDb: Error: [Errno 2] No such file or directory
Uname: Linux 3.0.0-15-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: sysadmin www-sg
WifiSyslog:
dmi.bios.date: 11/17/2005
dmi.bios.vendor: FUJITSU SIEMENS // Phoenix Technologies Ltd.
dmi.bios.version: 5.00 R1.07.2151.A1
dmi.board.name: D2151-A1
dmi.board.vendor: FUJITSU SIEMENS
dmi.board.version: S26361-D2151-A1
dmi.chassis.type: 6
dmi.chassis.vendor: FUJITSU SIEMENS
dmi.modalias: dmi:bvnFUJITSUSIEMENS//PhoenixTechnologiesLtd.:bvr5.00R1.07.2151.A1:bd11/17/2005:svnFUJITSUSIEMENS:pnD2151-A1:pvr:rvnFUJITSUSIEMENS:rnD2151-A1:rvrS26361-D2151-A1:cvnFUJITSUSIEMENS:ct6:cvr:
dmi.product.name: D2151-A1
dmi.sys.vendor: FUJITSU SIEMENS
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/932687/+subscriptions
More information about the foundations-bugs
mailing list