system won't boot due to corruption of persistent iSCSI database

[Alex Muntada]

We have 4 guests in KVM running iSCSI root filesystems.
After a full shutdown of the host, only 1 of the guests booted
again and we suspect it may be due to the host shutting down
before the guests had time to perform their own shutdown.

We struggled for two days with the booting process (upstart,
apparmor, initramfs, iptables, etc.) and discarded potential
sources for error until we end up suspecting that iscsid was
the culprit (almost a wild guess).

To confirm our suspicion we replaced /sbin/iscsid with /bin/true
and then the guest booted fine. A little later we concluded that
the persistent iSCSI database was corrupted, since we could
reproduce the same "end-request I/O errors" after we set up
a system rescue disk and tried to run iscsid manually.

Then, we removed /etc/iscsi/{nodes,send_targets} completely
and rebuild them with "iscsiadm -m discovery ..." from the
rescued system. Rebooted and the guest finished the boot
process seamlessly.

Has anyone suffered the same problem?
What could be the root cause of the problem?
Do you think that having the system rediscover iSCSI nodes
database before booting would be an overkill solution?
Is this issue worth a bug report?

Thanks!

-- 
Alex Muntada <alexm at alexm.org>
http://alexm.org/