[Bug 585027] Re: Race condition with dmsetup causes 'map already present' messages
Alasdair G. Kergon
agk at redhat.com
Mon Jul 9 16:11:15 UTC 2012
On Mon, Jul 09, 2012 at 03:29:11PM -0000, Peter Petrakis wrote:
> In examining the ioctl interface to device mapper (kernel), I have not found any architecture that
> assures that when a device mapper device has been marked for deletion, that this information is
> relayed back to userspace so it can block on this deletion. An example will clarify things.
If this is the same process that issued the deletion that needs to wait
till the device has gone, then the udevcookie mechanism is meant to
provide that already. (Multipathd is supposed to be using that
mechanism, upstream at least.)
If it's a different process, then I consider it's userspace's job to
co-ordinate the sequencing of ioctls that conflict with each other.
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to multipath-tools in Ubuntu.
https://bugs.launchpad.net/bugs/585027
Title:
Race condition with dmsetup causes 'map already present' messages
Status in “multipath-tools” package in Ubuntu:
Invalid
Bug description:
Binary package hint: multipath-tools
Environment:
njones at fantail:~$ lsb_release -rd
Description: Ubuntu 10.04 LTS
Release: 10.04
njones at fantail:~$ apt-cache policy multipath-tools
multipath-tools:
Installed: 0.4.8-14ubuntu4
Candidate: 0.4.8-14ubuntu4
Version table:
*** 0.4.8-14ubuntu4 0
500 http://nz.archive.ubuntu.com/ubuntu/ lucid/main Packages
100 /var/lib/dpkg/status
Issue:
While reproducing
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/571093 both
myself and Dustin Kirkland encountered an apparent race condition in
multipath and dmsetup.
We were attempting variants of 'while true; do sudo multipath -F; sudo
multipath -v4 ; done' to create a set of udev add/remove events and
noticed that shortly after starting, no output appeared within the
'udevadm monitor' command.
We observed that the output from a: multipath -v4 was similar to:
root at fantail:~# multipath -v4
Discover device /sys/block/ram0
ram0: device node name blacklisted
Discover device /sys/block/ram1
ram1: device node name blacklisted
Discover device /sys/block/ram2
ram2: device node name blacklisted
Discover device /sys/block/ram3
ram3: device node name blacklisted
Discover device /sys/block/ram4
ram4: device node name blacklisted
Discover device /sys/block/ram5
ram5: device node name blacklisted
Discover device /sys/block/ram6
ram6: device node name blacklisted
Discover device /sys/block/ram7
ram7: device node name blacklisted
Discover device /sys/block/ram8
ram8: device node name blacklisted
Discover device /sys/block/ram9
ram9: device node name blacklisted
Discover device /sys/block/ram10
ram10: device node name blacklisted
Discover device /sys/block/ram11
ram11: device node name blacklisted
Discover device /sys/block/ram12
ram12: device node name blacklisted
Discover device /sys/block/ram13
ram13: device node name blacklisted
Discover device /sys/block/ram14
ram14: device node name blacklisted
Discover device /sys/block/ram15
ram15: device node name blacklisted
Discover device /sys/block/loop0
loop0: device node name blacklisted
Discover device /sys/block/loop1
loop1: device node name blacklisted
Discover device /sys/block/loop2
loop2: device node name blacklisted
Discover device /sys/block/loop3
loop3: device node name blacklisted
Discover device /sys/block/loop4
loop4: device node name blacklisted
Discover device /sys/block/loop5
loop5: device node name blacklisted
Discover device /sys/block/loop6
loop6: device node name blacklisted
Discover device /sys/block/loop7
loop7: device node name blacklisted
Discover device /sys/block/sda
sda: not found in pathvec
sda: mask = 0x1f
sda: dev_t = 8:0
sda: size = 625142448
sda: subsystem = scsi
sda: vendor = ATA
sda: product = WDC WD3200AAKS-0
sda: rev = 12.0
sda: h:b:t:l = 0:0:0:0
sda: serial = WD-WCAS70000070
sda: getprio = NULL (internal default)
sda: prio = 1
sda: getuid = /lib/udev/scsi_id -g -u -d /dev/%n (config file default)
sda: uid = 1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070 (callout)
Discover device /sys/block/sr0
sr0: device node name blacklisted
Discover device /sys/block/sdb
sdb: not found in pathvec
sdb: mask = 0x1f
sdb: dev_t = 8:16
sdb: size = 15679488
sdb: subsystem = scsi
sdb: vendor = Kingston
sdb: product = DataTraveler G2
sdb: rev = PMAP
sdb: h:b:t:l = 6:0:0:0
sdb: serial =
sdb: getprio = NULL (internal default)
sdb: prio = 1
sdb: getuid = /lib/udev/scsi_id -g -u -d /dev/%n (config file default)
error calling out /lib/udev/scsi_id -g -u -d /dev/sdb
Discover device /sys/block/sdc
sdc: not found in pathvec
sdc: mask = 0x1f
sdc: dev_t = 8:32
sdc: size = 41943040
sdc: subsystem = scsi
sdc: vendor = IET
sdc: product = VIRTUAL-DISK
sdc: rev = 0
sdc: h:b:t:l = 7:0:0:0
sdc: serial =
sdc: getprio = NULL (internal default)
sdc: prio = 1
sdc: getuid = /lib/udev/scsi_id -g -u -d /dev/%n (config file default)
error calling out /lib/udev/scsi_id -g -u -d /dev/sdc
Discover device /sys/block/dm-0
dm-0: device node name blacklisted
Discovery status 0
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070 0:0:0:0 sda 8:0 1 [undef][undef
6:0:0:0 sdb 8:16 1 [undef][undef
7:0:0:0 sdc 8:32 1 [undef][undef
sda: ownership set to 1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070
sda: not found in pathvec
sda: mask = 0xc
sda: path checker = directio (config file default)
directio: called for 800
directio: called in synchronous mode
directio: starting new request
directio: async io getevents returns 1 (errno=No such file or directory)
directio: io finished 4096/0
sda: state = 2
sda: prio = 1
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: pgfailover = -1 (internal default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: pgpolicy = failover (internal default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: selector = round-robin 0 (internal default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: features = 0 (internal default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: hwhandler = 0 (internal default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: rr_weight = 1 (internal default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: minio = 1000 (config file default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: no_path_retry = NONE (internal default)
pg_timeout = NONE (internal default)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: set ACT_CREATE (map does not exist)
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: map already present
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: domap (0) failure for create/reload map
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070: remove multipath map
sda: orphaned
And a device was left behind in /dev/mapper:
njones at fantail:~$ ll /dev/mapper/
total 0
drwxr-xr-x 2 root root 80 2010-05-25 04:25 ./
drwxr-xr-x 18 root root 4080 2010-05-25 04:25 ../
brw-rw---- 1 root disk 252, 0 2010-05-25 04:25 1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070
crw-rw---- 1 root root 10, 59 2010-05-22 10:35 control
$ sudo dmsetup ls
1ATA_WDC_WD3200AAKS-00TMA0_WD-WCAS70000070 (252, 0)
We had to perform 'dmsetup remove <id>' to clear the 'map already
present' message.
While I admit that this is a very specific case, it is still a race
condition and does warrant a bug report.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/multipath-tools/+bug/585027/+subscriptions
More information about the foundations-bugs
mailing list