[Bug 1906476] Re: PANIC at zfs_znode.c:335:zfs_znode_sa_init() // VERIFY(0 == sa_handle_get_from_db(zfsvfs->z_os, db, zp, SA_HDL_SHARED, &zp->z_sa_hdl)) failed
Jacquin Antoine
1906476 at bugs.launchpad.net
Sat Oct 23 22:15:26 UTC 2021
ubuntu 21.10
Linux i5 5.13.0-20-generic #20-Ubuntu SMP Fri Oct 15 14:21:35 UTC 2021
x86_64 x86_64 x86_64 GNU/Linux
zfs-2.0.6-1ubuntu2
zfs-kmod-2.0.6-1ubuntu2
$ rm -rfv .steam
...
'.steam/steam/config/htmlcache/Cache/5f67979416a221e5_0' supprimé
'.steam/steam/config/htmlcache/Cache/2b3b061ac9d6b292_0' supprimé
'.steam/steam/config/htmlcache/Cache/88ec9cdf0f2e7cf6_0' supprimé
'.steam/steam/config/htmlcache/Cache/446e83c112a55833_0' supprimé
'.steam/steam/config/htmlcache/Cache/25bce87ba6a10af5_0' supprimé
'.steam/steam/config/htmlcache/Cache/a409ef32a0f5a1b3_0' supprimé
'.steam/steam/config/htmlcache/Cache/2e8722be934b8d51_0' supprimé
'.steam/steam/config/htmlcache/Cache/cb27d7e85cfb9396_0' supprimé
'.steam/steam/config/htmlcache/Cache/f7bb287f03ab70bb_0' supprimé
'.steam/steam/config/htmlcache/Cache/18b17be83cac58df_0' supprimé
'.steam/steam/config/htmlcache/Cache/9f7e378b5b8fe6cf_0' supprimé
'.steam/steam/config/htmlcache/Cache/3742398e7e6ac7aa_0' supprimé
stall
[ 549.052760] VERIFY(0 == sa_handle_get_from_db(zfsvfs->z_os, db, zp, SA_HDL_SHARED, &zp->z_sa_hdl)) failed
[ 549.052763] PANIC at zfs_znode.c:339:zfs_znode_sa_init()
[ 549.052765] Showing stack for process 18526
[ 549.052766] CPU: 0 PID: 18526 Comm: rm Tainted: P OE 5.13.0-20-generic #20-Ubuntu
[ 549.052768] Hardware name: System manufacturer System Product Name/PRIME Z270-A, BIOS 0505 11/08/2016
[ 549.052769] Call Trace:
[ 549.052772] show_stack+0x52/0x58
[ 549.052776] dump_stack+0x7d/0x9c
[ 549.052781] spl_dumpstack+0x29/0x2b [spl]
[ 549.052791] spl_panic+0xd4/0xfc [spl]
[ 549.052799] ? queued_spin_unlock+0x9/0x10 [zfs]
[ 549.052889] ? do_raw_spin_unlock+0x9/0x10 [zfs]
[ 549.052944] ? __raw_spin_unlock+0x9/0x10 [zfs]
[ 549.052998] ? dmu_buf_replace_user+0x65/0x80 [zfs]
[ 549.053053] ? dmu_buf_set_user+0x13/0x20 [zfs]
[ 549.053107] ? dmu_buf_set_user_ie+0x15/0x20 [zfs]
[ 549.053160] zfs_znode_sa_init+0xd9/0xe0 [zfs]
[ 549.053242] zfs_znode_alloc+0x101/0x580 [zfs]
[ 549.053325] ? dmu_buf_unlock_parent+0x5d/0x90 [zfs]
[ 549.053380] ? do_raw_spin_unlock+0x9/0x10 [zfs]
[ 549.053436] ? dbuf_read_impl.constprop.0+0x30a/0x3e0 [zfs]
[ 549.053489] ? dbuf_rele_and_unlock+0x13b/0x520 [zfs]
[ 549.053541] ? __cond_resched+0x1a/0x50
[ 549.053544] ? __raw_callee_save___native_queued_spin_unlock+0x15/0x23
[ 549.053547] ? queued_spin_unlock+0x9/0x10 [zfs]
[ 549.053597] ? do_raw_spin_unlock+0x9/0x10 [zfs]
[ 549.053647] ? __cond_resched+0x1a/0x50
[ 549.053648] ? down_read+0x13/0x90
[ 549.053650] ? __raw_callee_save___native_queued_spin_unlock+0x15/0x23
[ 549.053652] ? queued_spin_unlock+0x9/0x10 [zfs]
[ 549.053711] ? do_raw_spin_unlock+0x9/0x10 [zfs]
[ 549.053770] ? __raw_callee_save___native_queued_spin_unlock+0x15/0x23
[ 549.053773] ? dmu_object_info_from_dnode+0x8e/0xa0 [zfs]
[ 549.053829] zfs_zget+0x235/0x280 [zfs]
[ 549.053909] zfs_dirent_lock+0x420/0x560 [zfs]
[ 549.053990] zfs_dirlook+0x91/0x2d0 [zfs]
[ 549.054070] zfs_lookup+0x257/0x400 [zfs]
[ 549.054149] zpl_lookup+0xcb/0x220 [zfs]
[ 549.054227] ? __d_alloc+0x138/0x1f0
[ 549.054229] __lookup_hash+0x70/0xa0
[ 549.054231] ? __cond_resched+0x1a/0x50
[ 549.054232] do_unlinkat+0x135/0x2e0
[ 549.054234] __x64_sys_unlinkat+0x3a/0x60
[ 549.054235] do_syscall_64+0x61/0xb0
[ 549.054237] ? do_syscall_64+0x6e/0xb0
[ 549.054238] ? do_syscall_64+0x6e/0xb0
[ 549.054239] ? __x64_sys_write+0x19/0x20
[ 549.054241] ? do_syscall_64+0x6e/0xb0
[ 549.054242] ? asm_sysvec_apic_timer_interrupt+0xa/0x20
[ 549.054243] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 549.054246] RIP: 0033:0x7ff2c3aaf1fb
[ 549.054248] Code: 73 01 c3 48 8b 0d 35 ec 0f 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 07 01 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 05 ec 0f 00 f7 d8 64 89 01 48
[ 549.054249] RSP: 002b:00007ffc261242b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000107
[ 549.054251] RAX: ffffffffffffffda RBX: 00005607c3b07310 RCX: 00007ff2c3aaf1fb
[ 549.054252] RDX: 0000000000000000 RSI: 00005607c3b07418 RDI: 0000000000000008
[ 549.054253] RBP: 00005607c3aa2440 R08: 0000000000000003 R09: 0000000000000000
[ 549.054253] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 549.054254] R13: 00007ffc26124490 R14: 00007ffc26124490 R15: 0000000000000002
$ zpool scrub rpool
pool: rpool
state: ONLINE
scan: scrub repaired 0B in 00:07:17 with 0 errors on Sun Oct 24 00:12:51 2021
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
0518af4d-aa46-5a45-a87f-3ac806eed4c1 ONLINE 0 0 0
errors: No known data errors
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ubuntu-release-upgrader in
Ubuntu.
https://bugs.launchpad.net/bugs/1906476
Title:
PANIC at zfs_znode.c:335:zfs_znode_sa_init() // VERIFY(0 ==
sa_handle_get_from_db(zfsvfs->z_os, db, zp, SA_HDL_SHARED,
&zp->z_sa_hdl)) failed
Status in Native ZFS for Linux:
New
Status in linux package in Ubuntu:
Invalid
Status in ubuntu-release-upgrader package in Ubuntu:
Confirmed
Status in zfs-linux package in Ubuntu:
Fix Released
Status in linux source package in Impish:
Fix Released
Status in ubuntu-release-upgrader source package in Impish:
Confirmed
Status in zfs-linux source package in Impish:
Fix Released
Bug description:
Since today while running Ubuntu 21.04 Hirsute I started getting a ZFS
panic in the kernel log which was also hanging Disk I/O for all
Chrome/Electron Apps.
I have narrowed down a few important notes:
- It does not happen with module version 0.8.4-1ubuntu11 built and included with 5.8.0-29-generic
- It was happening when using zfs-dkms 0.8.4-1ubuntu16 built with DKMS
on the same kernel and also on 5.8.18-acso (a custom kernel).
- For whatever reason multiple Chrome/Electron apps were affected,
specifically Discord, Chrome and Mattermost. In all cases they seem
(but I was unable to strace the processes so it was a bit hard ot
confirm 100% but by deduction from /proc/PID/fd and the hanging ls)
they seem hung trying to open files in their 'Cache' directory, e.g.
~/.cache/google-chrome/Default/Cache and ~/.config/Mattermost/Cache ..
while the issue was going on I could not list that directory either
"ls" would just hang.
- Once I removed zfs-dkms only to revert to the kernel built-in
version it immediately worked without changing anything, removing
files, etc.
- It happened over multiple reboots and kernels every time, all my
Chrome apps weren't working but for whatever reason nothing else
seemed affected.
- It would log a series of spl_panic dumps into kern.log that look like this:
Dec 2 12:36:42 optane kernel: [ 72.857033] VERIFY(0 == sa_handle_get_from_db(zfsvfs->z_os, db, zp, SA_HDL_SHARED, &zp->z_sa_hdl)) failed
Dec 2 12:36:42 optane kernel: [ 72.857036] PANIC at zfs_znode.c:335:zfs_znode_sa_init()
I could only find one other google reference to this issue, with 2 other users reporting the same error but on 20.04 here:
https://github.com/openzfs/zfs/issues/10971
- I was not experiencing the issue on 0.8.4-1ubuntu14 and fairly sure
it was working on 0.8.4-1ubuntu15 but broken after upgrade to
0.8.4-1ubuntu16. I will reinstall those zfs-dkms versions to verify
that.
There were a few originating call stacks but the first one I hit was
Call Trace:
dump_stack+0x74/0x95
spl_dumpstack+0x29/0x2b [spl]
spl_panic+0xd4/0xfc [spl]
? sa_cache_constructor+0x27/0x50 [zfs]
? _cond_resched+0x19/0x40
? mutex_lock+0x12/0x40
? dmu_buf_set_user_ie+0x54/0x80 [zfs]
zfs_znode_sa_init+0xe0/0xf0 [zfs]
zfs_znode_alloc+0x101/0x700 [zfs]
? arc_buf_fill+0x270/0xd30 [zfs]
? __cv_init+0x42/0x60 [spl]
? dnode_cons+0x28f/0x2a0 [zfs]
? _cond_resched+0x19/0x40
? _cond_resched+0x19/0x40
? mutex_lock+0x12/0x40
? aggsum_add+0x153/0x170 [zfs]
? spl_kmem_alloc_impl+0xd8/0x110 [spl]
? arc_space_consume+0x54/0xe0 [zfs]
? dbuf_read+0x4a0/0xb50 [zfs]
? _cond_resched+0x19/0x40
? mutex_lock+0x12/0x40
? dnode_rele_and_unlock+0x5a/0xc0 [zfs]
? _cond_resched+0x19/0x40
? mutex_lock+0x12/0x40
? dmu_object_info_from_dnode+0x84/0xb0 [zfs]
zfs_zget+0x1c3/0x270 [zfs]
? dmu_buf_rele+0x3a/0x40 [zfs]
zfs_dirent_lock+0x349/0x680 [zfs]
zfs_dirlook+0x90/0x2a0 [zfs]
? zfs_zaccess+0x10c/0x480 [zfs]
zfs_lookup+0x202/0x3b0 [zfs]
zpl_lookup+0xca/0x1e0 [zfs]
path_openat+0x6a2/0xfe0
do_filp_open+0x9b/0x110
? __check_object_size+0xdb/0x1b0
? __alloc_fd+0x46/0x170
do_sys_openat2+0x217/0x2d0
? do_sys_openat2+0x217/0x2d0
do_sys_open+0x59/0x80
__x64_sys_openat+0x20/0x30
To manage notifications about this bug go to:
https://bugs.launchpad.net/zfs/+bug/1906476/+subscriptions
More information about the foundations-bugs
mailing list