Growing a RAID6 array on 11.10 *** SOLVED ***

[Steven Davies-Morris]

On 03/09/2012 07:02 AM, Dave Lawrence wrote:
> On 09/03/12 00:09, Steven Davies-Morris wrote:
>> On 03/07/2012 02:49 PM, Steven Davies-Morris wrote:
>>> I have two RAID arrays on a system. The first is RAID1 and
>>> supports /boot, swap / and /home. The second is RAID6 and
>>> supports everything else. It currently has 4 3TB drives. I have 3
>>> more sitting in the tower and want to add them to the RAID6
>>> array.
>>>
>>> I was using Disk Utility to try to add these three drives but it
>>> doesn't seem to want to do it with the array up and running. When
>>> I take the array down I'm denied access to the array controls in
>>> "Edit Components" so I can't tell it to add drives.
>>>
>>> I remove the partitions from the 3 drives (initialized as GPT).
>>> Then I go into "Edit Components". I see the three drives. I am
>>> told they each have no partitions and each have 3TB available for
>>> use.
>>>
>>> I go to add one drive. I click the box to add it and the click
>>> "Expand". I authenticate then get this message:
>>>
>>> Error expanding RAID Array. An error occurred while performing
>>> on operation on "hitchcock RAID6" (6.0 TB RAID-6 Array): The
>>> operation failed
>>>
>>> Detail information says:
>>>
>>> Error expanding array: helper script exited with exit code 1:
>>> mdadm: added /dev/sdc1 mdadm: Need to backup 6144K of critical
>>> section..
>>>
>>> I now see the new drive in the RAID6 array as a spare.
>>>
>>> What am I missing here? If I can't figure this out I guess it
>>> will be a reboot into a run-level 1 root shell to douse mdadm.
>>> Any pointers from those of you who are quite familiar with using
>>> Disk Utility and RAID arrays?
>>
>> I finally figured out what was going on. Digging further into the
>> error message output I kept looking at the issue of bitmap removal.
>> Why was this necessary? And removal from where? I had an epiphany
>> 30 mins ago reading this mdadm man page:
>>
>> <http://linux.die.net/man/8/mdadm>
>>
>> Which then led me to search online and read this page which has
>> some good information about speeding up working with Linux software
>> raid:
>>
>> <http://www.coderetard.com/2011/02/01/howto-speed-up-linux-software-raid-building-and-re-syncing/>
>>
>>
>>
>> What I wound up doing was removing the bitmap from the array with
>> this shell command:
>>
>> sudo mdadm --grow --bitmap=none /dev/md127
>>
>> I was then able to expand the array using Disk Utility, though I
>> could have done it from a terminal shell with this command:
>>
>> sudo mdadm --grow /dev/md127 --raid-devices=7
>>
>> Once I'm done with the reshaping I'll put the bitmap back after
>> figuring out what it should optimally be. The RAID6 array is
>> reshaping right now with the three new 3TB drives added. In a few
>> hours I will seal it all back up. In the meantime I'm off to Fry's
>> Electronics to return the extra pieces of hardware that I don't
>> need.
>>
>> The final configuration of the media center server will be a 8gb
>> DDR3 machine with a AMD Phenom II X4 quad-core processor. It will
>> have a RAID1 mirror of 2*2TB drives for boot, swap, / and home. And
>> 7*3TB drives for the RAID6 array. 4 will be on the mobo along with
>> the boot mirror. The other 3 drives will be on a 4port SATA card. 6
>> drives are mounted in the 3.5" drive bays and the other 3 plus the
>> PATA DVD burner are mounted in the 5.25" drive bays. 4 big fans,
>> plus 1 giant fan and a 1200watt power-supply. Hopefully I'm now
>> done tinkering with it and can get back to software and music.
>
> Thanks for sharing the solution.  When I read your first post I had
> thought the answer would be much more complex than that for RAID 6.
> Just one tip to add: before reshaping RAID arrays, repartitioning etc
> etc it's a good idea to have a practice run on a Virtual machine.

That's good advice, Dave. I didn't have a VM practice run, but I have
all the data backed up on the original (smaller) drives from two
machines "just in case". So I was willing to shoot myself in the foot as
part of the learning process if need be. As I did when originally trying
to setup my RFAID1 mirror on a pair of 3TB drives using GPT for the boot
disk. Which I eventually abandoned.

The only think that I am left with is that Disk Utility tells me that
the ext4 partition it created to overlay the underlying disk partitions
is misaligned by just under 1MB. Specifically it says this when I go to
look at the Volumes section of the display:

!WARNING: The partition is misaligned by 10311168 bytes. This may result
in very poor performance. Repartitioning is suggested.

When I was battling with adding the three new drives to expand the RAID
6 array I tried formatting them through Disk Utility and through
GParted. Those done through Disk Utility appear to create only a 17.4KB
header offset rather then the 1MB offset created by GParted before it
would create the partition to be given to Linux Raid. Which meant that I
couldn't manually add those drives until I went back to letting Disk
Utility prepare them. This must be the space it uses to setup its
protection from software that only understands about MBR disks.

The array holds mainly large audio and video files (family media center
plus old and in progress music for Systems Theory+related projects).
Performance of Writes to the 4 disk array seemed quite acceptable and
the Reads were very fast (faster than getting similar files of single
disks). So I'm not too concerned about this warning (though it does
raise a WTF??? in my mind), and may not do anything about now it if I'm
happy with the performance of the 7 drive array. But I am interested to
know what others with experience in RAID5 and RAID6 setups think about
this since I couldn't find much of anything about it poking about online.

There was this:
<http://forum.linuxmint.com/viewtopic.php?f=46&t=89212>

Which led me to this:
<http://ubuntuforums.org/showthread.php?t=1635018>
-- 
SDM a 21st century schizoid man in SoCal
Systems Theory website www.systemstheory.net
"overfulnoisecascade" CD coming soon