[sclug] Mirror disks across machines like distributed RAID?

[Tom Carbert-Allen]

On 05/01/2010 3:07, Dickon Hood wrote:
>   It did seem to me to be a bit of a
> solution looking for a problem, but it's a neat enough technology and I'm
> sure someone will find one.
>
>    

Well the other solutions I have personally built and tested to this 
problem is an external storage box (or pair of replicated boxs) which 
connect back with FC or iSCSI over copper. These present a number of 
problems:

Huge cost - we regularly shell out ?10K+ on a single HP FC SAN with only 
256mb cache and RAID5 when you put in the two controllers and two FC 
cards in each of the servers, I know copper iSCSI SAN's are cheaper, but 
maybe only 20%

Extra large cost to have two replicated disks - this again costs us 
another 10k depending on the options

High latency - in my tests the FC SAN had 240% latency of identical 
internal disk and around 300% the latency when comparing and internal 
raid5 to SAN raid5, throughput was only marginally different in both cases.

Large rack space - Sans's normally start at 3u minimum each and the ones 
we use are 5U, fine in your own rack maybe, but when collocating in is 
again a huge cost just for a box of 5~10 disks

Extra power/cooling requirement - according to my power metre, the HP FC 
san we use with 2xcontroller and 2xpsu and 2x4port FC card in and 5X 
10krpm SAS disks is clocking about 2A continuous, measuring the 
difference of a 2u server with no disks in it or the same 5 disks that 
were tested in the SAN only increases the power usage by 0.25A (with 
power now costing what like ?20 an amp per month?)

So I continue to build FC SAN's at work, because that's "what 
enterprises expect", but to me I don't see why they put up with all the 
above problems when there is a perfectly good open source solution which 
gives them the same if not better features for <5% the cost.

Anyone else got any other experiences with high availability disk 
solutions either at home or in work?

TCA