Hardware vs software raid

[Preston Hagar]

On Tue, Jan 3, 2012 at 6:23 PM, Mat Cantin <mat at cantinbrothers.ca> wrote:
>> Depending on your needs, these cards (or a lot of them
>> at least, since it is easy to saturate the PCI bus) may not give you
>> the highest performance, but for most applications I find that it is
>> more than sufficient (I have it setup on Zoneminder camera servers,
>> Samba servers, NFS file servers, among other things).  It just might
>> not be best for say putting a database on.
>
>
> I've looked at trying to find a reliable means to monitor whether the PCI
> bus is being saturated, only to come up with what seemed like a hardware
> gimmick. Is there a way to determine if the PCI bus is being saturated
> within the OS? I suppose one could calculate the theoretical PCI bus
> throughput in Mb/s, but this doesn't seem very concrete to me since the
> quality of hardware often determines if the top thresholds can be met.
>
> --
>
> matoc
>

Not really that I have found.  I usually just take the claimed and/or
benchmarked drive rates (including what they should be with a RAID
setup) and then look at the bus I am using and see if it looks like I
am going to max it out.  For example, a 32-bit/66mhz PCI bus (the
older kind), has a theoretical max bandwidth of 266 MB/s.  SATA 2 has
a max bandwidth of 375 MB/s (although I haven't really seen any
non-ssd drives that come very close to that), so one drive alone could
saturate it.  A RAID 0 or RAID 10 setup could have an even bigger
performance hit.  That said, usually with drives, you are waiting on
seek time, not bus throughput, so it isn't always an issue.

If you are using PCI-E, then bus bandwidth becomes less of an issue
since even PCI-E 1x is 8 GB/s, but you have to keep an eye on the
cards though since a lot of them are PCI cards crammed onto a PCI-E
board, so they don't really take advantage of the faster speed.

One other thing I have done in the past where performance is an issue
is to run bonnie++ tests to try to gauge what the speeds were.  If
bonnie reports that my reads or writes are about what the bus maximum
is, then I know that is the bottleneck.  If I'm not close, then I know
either my array should be fine or (if I think it should be faster)
then I need to look at why drive performance is bad.

Sorry it isn't a direct answer, but hopefully it might help out some.

Preston