[Bug 1842020] Re: ceph patch as of 8/29 segfaults all bluestore osds
Harry Coin
hgcoin at gmail.com
Sat Aug 31 14:10:22 UTC 2019
And with debug symbols:
(gdb) run
Starting program: /usr/bin/ceph-bluestore-tool
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGILL, Illegal instruction.
0x0000555555743984 in eth_dev_init_cb_lists ()
(gdb) backtrace full
#0 0x0000555555743984 in eth_dev_init_cb_lists ()
No symbol table info available.
#1 0x0000555555dc045d in __libc_csu_init ()
No symbol table info available.
#2 0x00007fffee524e2e in __libc_start_main (main=0x5555557346b0 <main(int, char**)>, argc=1, argv=0x7fffffffe328, init=0x555555dc0410 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe318)
at ../csu/libc-start.c:264
result = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737351328816, 5390835150573769728, 140737353589600, 140737353586152, 1, 140737488347944, 140737488347960, 140737354007706}, mask_was_saved = 8}}, priv = {pad = {0x1,
0x7fffffffe328, 0x7fffffffe338, 0x7ffff7ffe190}, data = {prev = 0x1, cleanup = 0x7fffffffe328, canceltype = -7368}}}
not_first_call = <optimized out>
#3 0x000055555581e47e in _start () at /usr/include/c++/9/ostream:108
No symbol table info available.
(gdb) backtrace full
#0 0x0000555555743984 in eth_dev_init_cb_lists () at /usr/include/c++/9/ostream:108
No symbol table info available.
#1 0x0000555555dc045d in __libc_csu_init ()
No symbol table info available.
#2 0x00007fffee524e2e in __libc_start_main (main=0x5555557346b0 <main(int, char**)>, argc=1, argv=0x7fffffffe328, init=0x555555dc0410 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe318)
at ../csu/libc-start.c:264
result = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737351328816, 5390835150573769728, 140737353589600, 140737353586152, 1, 140737488347944, 140737488347960, 140737354007706}, mask_was_saved = 8}}, priv = {pad = {0x1,
0x7fffffffe328, 0x7fffffffe338, 0x7ffff7ffe190}, data = {prev = 0x1, cleanup = 0x7fffffffe328, canceltype = -7368}}}
not_first_call = <optimized out>
#3 0x000055555581e47e in _start () at /usr/include/c++/9/ostream:108
No symbol table info available.
(gdb) info registers
rax 0x555555fe0340 93825003291456
rbx 0x36 54
rcx 0xb 11
rdx 0x5555568921a0 93825012408736
rsi 0x7fffffffe328 140737488347944
rdi 0x1 1
rbp 0xc5 0xc5
rsp 0x7fffffffe208 0x7fffffffe208
r8 0x0 0
r9 0x0 0
r10 0x642e6264626f6c62 7218815436009204834
r11 0x20 32
r12 0x555555f2b510 93825002550544
r13 0x1 1
r14 0x7fffffffe328 140737488347944
r15 0x5555568921a0 93825012408736
rip 0x555555743984 0x555555743984 <eth_dev_init_cb_lists+68>
eflags 0x10212 [ AF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
(gdb) x/16i $pc
=> 0x555555743984 <eth_dev_init_cb_lists+68>: pextrq $0x1,%xmm2,0x40c0(%rax)
0x55555574398f <eth_dev_init_cb_lists+79>: pextrq $0x1,%xmm1,0xc1c0(%rax)
0x55555574399a <eth_dev_init_cb_lists+90>: movdqa 0x6c2eae(%rip),%xmm2 # 0x555555e06850
0x5555557439a2 <eth_dev_init_cb_lists+98>: movq %xmm1,0x8140(%rax)
0x5555557439aa <eth_dev_init_cb_lists+106>: movdqa 0x6c2eae(%rip),%xmm1 # 0x555555e06860
0x5555557439b2 <eth_dev_init_cb_lists+114>: movq $0x0,0x8138(%rax)
0x5555557439bd <eth_dev_init_cb_lists+125>: paddq %xmm0,%xmm2
0x5555557439c1 <eth_dev_init_cb_lists+129>: movq %xmm2,0x10240(%rax)
0x5555557439c9 <eth_dev_init_cb_lists+137>: paddq %xmm0,%xmm1
0x5555557439cd <eth_dev_init_cb_lists+141>: pextrq $0x1,%xmm2,0x142c0(%rax)
0x5555557439d8 <eth_dev_init_cb_lists+152>: movdqa 0x6c2e90(%rip),%xmm2 # 0x555555e06870
0x5555557439e0 <eth_dev_init_cb_lists+160>: movq %xmm1,0x18340(%rax)
0x5555557439e8 <eth_dev_init_cb_lists+168>: pextrq $0x1,%xmm1,0x1c3c0(%rax)
0x5555557439f3 <eth_dev_init_cb_lists+179>: movdqa 0x6c2e85(%rip),%xmm1 # 0x555555e06880
0x5555557439fb <eth_dev_init_cb_lists+187>: movq $0x0,0xc1b8(%rax)
0x555555743a06 <eth_dev_init_cb_lists+198>: paddq %xmm0,%xmm2
(gdb) thread apply all backtrace
Thread 1 (Thread 0x7fffee0e20c0 (LWP 825)):
#0 0x0000555555743984 in eth_dev_init_cb_lists () at /usr/include/c++/9/ostream:108
#1 0x0000555555dc045d in __libc_csu_init ()
#2 0x00007fffee524e2e in __libc_start_main (main=0x5555557346b0 <main(int, char**)>, argc=1, argv=0x7fffffffe328, init=0x555555dc0410 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe318) at ../csu/libc-start.c:264
#3 0x000055555581e47e in _start () at /usr/include/c++/9/ostream:108
(gdb)
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/1842020
Title:
ceph patch as of 8/29 segfaults all bluestore osds
Status in ceph package in Ubuntu:
New
Bug description:
The ceph patch on eoan distributed 8/29 crashes all bluestore OSD's
and is unusable at least on some systems. Failed on an old dual Xeon
E5345 box.
Easy test. Run:
/usr/bin/ceph-bluestore-tool
On working systems it reports a help message. On the latest eoan release distributed by canonical it
turns in to an illegal instruction process kill, just after reading /proc/<...>/auxv
looks like something to do with vsock issues.
strace -k -y /usr/bin/ceph-bluestore-tool
is instructive.
Notice valgrind against that program reports thousands of memory allocation issues.
Reverting the system to the snapshot before the apt upgrade restores full operations.
---
ProblemType: Bug
ApportVersion: 2.20.11-0ubuntu7
Architecture: amd64
DistroRelease: Ubuntu 19.10
InstallationDate: Installed on 2019-07-30 (30 days ago)
InstallationMedia: Ubuntu-MATE 19.10 "Eoan Ermine" - Alpha amd64 (20190726)
Package: ceph 14.2.2-0ubuntu2
PackageArchitecture: amd64
ProcEnviron:
LANGUAGE=en_US
TERM=xterm-256color
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcVersionSignature: Ubuntu 5.2.0-15.16-generic 5.2.9
Tags: eoan
Uname: Linux 5.2.0-15-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: True
mtime.conffile..etc.default.apport: 2019-08-30T11:35:29.463071
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1842020/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list