[Bug 1567353] Re: Multipath core dumps when VMs booted from volume go to ERROR state
Launchpad Bug Tracker
1567353 at bugs.launchpad.net
Mon Mar 27 04:17:44 UTC 2017
[Expired for multipath-tools (Ubuntu) because there has been no activity
for 60 days.]
** Changed in: multipath-tools (Ubuntu)
Status: Incomplete => Expired
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/1567353
Title:
Multipath core dumps when VMs booted from volume go to ERROR state
Status in multipath-tools package in Ubuntu:
Expired
Bug description:
We are using
multipath-tools 0.4.9-3ubuntu7.6
We found a couple of multipath core dumps generated on some compute
nodes in our DC:
/var/log/crash/cores:
total 664
drwxr-x--- 2 root root 68 Apr 4 10:11 ./
drwxr-x--- 4 root root 38 Mar 30 22:09 ../
-rw-r----- 1 root root 679936 Apr 4 10:11 core.compute-0-11.domain.tld.1459757511.multipath.33543
/var/log/crash/cores:
total 640
drwxr-x--- 2 root root 68 Apr 4 10:11 ./
drwxr-x--- 4 root root 38 Mar 30 22:09 ../
-rw-r----- 1 root root 655360 Apr 4 10:11 core.compute-0-12.domain.tld.1459757513.multipath.31922
/var/log/crash/cores:
total 796
drwxr-x--- 2 root root 68 Apr 4 10:12 ./
drwxr-x--- 4 root root 38 Mar 30 22:09 ../
-rw-r----- 1 root root 815104 Apr 4 10:12 core.compute-1-10.domain.tld.1459757567.multipath.38307
/var/log/crash/cores:
total 772
drwxr-x--- 2 root root 67 Apr 4 10:12 ./
drwxr-x--- 4 root root 38 Mar 30 22:09 ../
-rw-r----- 1 root root 790528 Apr 4 10:12 core.compute-2-5.domain.tld.1459757554.multipath.60237
These seem to be related to the issue when a volume was deleted (most
probably these volumes was leftovers from when a VM booted from volume
but the VM went into ERROR state due to expiry of
block_device_allocate_retries).
Example from coredump related to compute-0-11:
cinder-volume.log:
2016-04-04 10:11:51.605 24563 INFO cinder.volume.manager [req-af0bea83-2b63-4118-8b3a-1c3aa1c57c8d 80c2726a44c54702a90265160a4adb5b e9e448cc9c59455f96b8325c694bd60c - - -] volume dd29523a-5a19-4075-ac77-84cddb75edc4: deleting
2016-04-04T10:11:51.605818+02:00 cic-1 cinder-volume 2016-04-04 10:11:51.605 24563 INFO cinder.volume.manager [req-af0bea83-2b63-4118-8b3a-1c3aa1c57c8d 80c2726a44c54702a90265160a4adb5b e9e448cc9c59455f96b8325c694bd60c - - -] volume dd29523a-5a19-4075-ac77-84cddb75edc4: deleting
kern.log:
2016-04-04T10:11:51.237422+02:00 compute-0-11 kernel: [397287.619468] device-mapper: multipath: Failing path 8:64.
2016-04-04T10:11:51.241272+02:00 compute-0-11 kernel: [397287.621247] scsi 2:0:0:244: alua: Detached
2016-04-04T10:11:51.245269+02:00 compute-0-11 kernel: [397287.623588] sd 1:0:0:244: alua: port group 01 state A non-preferred supports tolUsNA
2016-04-04T10:11:51.269243+02:00 compute-0-11 kernel: [397287.648665] bio: create slab <bio-1> at 1
2016-04-04T10:11:51.316343+02:00 compute-0-11 kernel: [397287.666870] device-mapper: multipath: Failing path 8:32.
2016-04-04T10:11:51.316352+02:00 compute-0-11 kernel: [397287.666881] end_request: I/O error, dev dm-3, sector 2872
2016-04-04T10:11:51.316354+02:00 compute-0-11 kernel: [397287.671502] bio: create slab <bio-1> at 1
2016-04-04T10:11:51.316358+02:00 compute-0-11 kernel: [397287.698405] Buffer I/O error on device dm-4, logical block 103
2016-04-04T10:11:51.405279+02:00 compute-0-11 kernel: [397287.731521] end_request: I/O error, dev dm-3, sector 2872
2016-04-04T10:11:51.405285+02:00 compute-0-11 kernel: [397287.733538] multipath[33543]: segfault at 0 ip 00007f308136d444 sp 00007ffcb0f3a5d8 error 4 in libc-2.19.so[7f308122b000+1bb000]
2016-04-04T10:11:51.405287+02:00 compute-0-11 kernel: [397287.745535] bio: create slab <bio-1> at 1
2016-04-04T10:11:51.405287+02:00 compute-0-11 kernel: [397287.762127] Buffer I/O error on device dm-4, logical block 103
2016-04-04T10:11:51.461579+02:00 compute-0-11 kernel: [397287.796824] device-mapper: table: 252:3: multipath: invalid initial priority group
2016-04-04T10:11:51.461588+02:00 compute-0-11 kernel: [397287.843587] device-mapper: ioctl: error adding target to table
2016-04-04T10:11:51.509422+02:00 compute-0-11 kernel: [397287.843660] device-mapper: table: 252:3: multipath: invalid initial priority group
2016-04-04T10:11:51.513240+02:00 compute-0-11 kernel: [397287.891469] device-mapper: ioctl: error adding target to table
2016-04-04T10:11:51.558139+02:00 compute-0-11 kernel: [397287.907562] end_request: I/O error, dev dm-3, sector 2560
2016-04-04T10:11:51.558147+02:00 compute-0-11 kernel: [397287.940151] Buffer I/O error on device dm-4, logical block 64
2016-04-04T10:11:51.626379+02:00 compute-0-11 kernel: [397287.976418] end_request: I/O error, dev dm-3, sector 2560
2016-04-04T10:11:51.626387+02:00 compute-0-11 kernel: [397287.977382] bio: create slab <bio-1> at 1
2016-04-04T10:11:51.626388+02:00 compute-0-11 kernel: [397288.008373] Buffer I/O error on device dm-4, logical block 64
2016-04-04T10:11:51.692068+02:00 compute-0-11 kernel: [397288.041954] end_request: I/O error, dev dm-3, sector 2080
2016-04-04T10:11:51.692075+02:00 compute-0-11 kernel: [397288.074050] Buffer I/O error on device dm-4, logical block 4
2016-04-04T10:11:51.757379+02:00 compute-0-11 kernel: [397288.107268] end_request: I/O error, dev dm-3, sector 2080
2016-04-04T10:11:51.757386+02:00 compute-0-11 kernel: [397288.139348] Buffer I/O error on device dm-4, logical block 4
2016-04-04T10:11:51.822990+02:00 compute-0-11 kernel: [397288.172705] end_request: I/O error, dev dm-3, sector 6144
2016-04-04T10:11:51.823012+02:00 compute-0-11 kernel: [397288.204953] Buffer I/O error on device dm-4, logical block 512
2016-04-04T10:11:51.901760+02:00 compute-0-11 kernel: [397288.249673] end_request: I/O error, dev dm-3, sector 41942912
2016-04-04T10:11:51.901767+02:00 compute-0-11 kernel: [397288.283682] Buffer I/O error on device dm-4, logical block 5242608
2016-04-04T10:11:51.972044+02:00 compute-0-11 kernel: [397288.319783] end_request: I/O error, dev dm-3, sector 41942912
2016-04-04T10:11:51.972050+02:00 compute-0-11 kernel: [397288.353952] Buffer I/O error on device dm-4, logical block 5242608
2016-04-04T10:11:52.042861+02:00 compute-0-11 kernel: [397288.390466] end_request: I/O error, dev dm-3, sector 41943024
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/1567353/+subscriptions
More information about the foundations-bugs
mailing list