Boot with a degraded raid 5

[Ben Ben]

Hi Philip. Thank you so much for taking time to answer me, I feel less
alone now :)

2007/12/4, Phillip Susi <psusi at cfl.rr.com>:
> ben.div wrote:
> > Woh ! Absolutely nobody can help me on this question ? I've already
> > asked about this on 4-5 lists or forums, and I've cumulated : 0 answer.
> > Where could I find help on this subject ? The kernel team ? Who has
> > developped this part (boot on initramfs and device detection) ?
> >
> > I'm stucked on that problem since 2 weeks. Please, help :)
>
> Known issue... though I can't seem to find the bug # now.
>
What is a known issue ? Running a degraded raid 5 at boot is not possible ?

> > Ben Ben a écrit :
> >> So here, I suspected that the wrong option was passed to mdadm in
> >> initramfs, and tell it to not to run a degraded array.
> >> I've found (with grep on initrams content) that the file
> >> /etc/udev/rules.d/85-mdadm.rules contains this line :
> >>
> >> SUBSYSTEM=="block", ACTION=="add|change", ENV{ID_FS_TYPE}=="linux_raid*", \
> >>         RUN+="watershed /sbin/mdadm --assemble --scan --no-degraded"
> >>
> >> I guess it's the boot parameter for mdadm ! So, I changed it, made a new
> >> initramfs, reboot with only 2 disks and... nothing more, it doesn't
> >> start anymore :/
>
> Not sure what's going wrong without any description other than "it
> doesn't start anymore", but that should allow you to boot in a degraded
> array.
>
It does the same thing I describe before : initramfs loads, md driver
try to run array but can't, so it hangs around 3 minutes and give me
the hand in the initramfs console.
Here's outputs of cat /proc/mdstat at this stade. I gave many try
(reboot), as the output is not always the same :

$ cat /proc/mdstat
Personalities : [raid0] [raid6] [raid5] [raid4] [linear] [multipath]
[raid1] [raid10]
md1 : active raid0 sda5[0] sdb5[1]
      78164032 blocks 64k chunks

md0 : inactive hda3[0]
      116141824 blocks

unused devices: <none>

$ cat /proc/mdstat # with kernel 2.6.22.14 md drivers compiled inside
kernel (not as module)
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4] [multipath] [faulty]
md1 : inactive sdb5[1]
      39102080 blocks

md0 : inactive hda3[0]
      116141824 blocks

unused devices: <none>

$ cat /proc/mdstat
Personalities : [raid0] [raid6] [raid5] [raid4] [linear] [multipath]
[raid1] [raid10]
md1 : active raid0 sda5[0] sdb5[1]
      78164032 blocks 64k chunks

md0 : inactive hda3[0]
      116141824 blocks

unused devices: <none>

$ cat /proc/mdstat
Personalities : [raid0] [raid6] [raid5] [raid4] [linear] [multipath]
[raid1] [raid10]
md1 : inactive sdb5[1]
      39102080 blocks

md0 : inactive hda3[0]
      116141824 blocks

unused devices: <none>

All these test have been done with all 3 disks up, and with --run
option for mdadm in /etc/udev/rules.d/85-mdadm.rules.
You can see that md0 is never ran, and md1 (raid0) is sometime ran,
sometimes not.
Here, if I stop and launch manually the md, it works.

If I give the option --no-degraded to mdadm, it work always (if the
array is not degraded).

> >> So, after this long story (sorry), my questions :
> >>
> >> Do you think I'm totally lost, or editing this file is the good way ?
> >> Is there a good reason why ubuntu's dev chose this "--no-degraded"
> >> option for mdadm by default ?
> >> What can I do more ??
>
> The reason is because we don't want to degrade an array just because one
> of the disks has not been detected yet.  The proper solution is to wait
> for either a timeout or manual intervention to go ahead and mount the
> array degraded.
>

Why does the --run option never work, but --no-degraded work, even if
the raid array is not degraded ?

It seems like you suggest to be a "disk not detected yet" problem. How
could I workaround this ? Maybe a "sleep 10" before launching mdadm ?
But why are the disk detected for md1 (raid1), but not for md0
(raid0), while it use the same devices (sda, sdb for md1, hda, sda,
sdb for md0) ?

Maybe it's a udev syntax problem (I didn't take time to study it) ?

Another question : you said I can run the array manually and launch
back the boot process. How can I do this last point ? running /init ?

Thank you !

Ben