[Scottish] RAID Setup[Scanned]

Wed Apr 9 10:58:48 BST 2008

Tim,

The point of the ATA over Ethernet was to allow more than one machine to 
access the storage at a time and without the hassle of NFS or samba.

I don't want to use SCSI because it's direct connect and can't be used on 
more than one machine at a time.  I figured AoE was fast and reliable enough 
to get the job done.  I'm only building these boxes because it's the same 
price as a comercial AoE box from Coraid!

I think what I'm building IS what you're suggesting.  Basically, I *could* 
buy a Coraid box (ata over ethernet) or I could build one (or in this case 
two!) myself that does exactly the same thing.

Is this what you mean?

Phil.

----- Original Message ----- 
From: "Tim Brocklehurst" <t.brocklehurst at henryabram.co.uk>
To: "Phillip Bennett" <phillip at mve.com>; "SLUG-list" 
<scottish at mailman.lug.org.uk>
Sent: Wednesday, April 09, 2008 10:44 AM
Subject: RE: [Scottish] RAID Setup[Scanned]

Phil, what's the benefit of ATA over Ethernet? Why not an ATX board, with 
two 8port SATA controllers and a few (say 4-port) network cards?

Cheers,

Tim B.

-----Original Message-----
From: scottish-bounces at mailman.lug.org.uk 
[mailto:scottish-bounces at mailman.lug.org.uk] On Behalf Of Phillip Bennett
Sent: Wednesday, April 09, 2008 10:26 AM
To: SLUG-list
Subject: Re: [Scottish] RAID Setup[Scanned]

Thanks Tim,

Based on what you've said and what I've done so far, I'm I'm considdering
the following setup.  If anyone sees any problems with it, let me know!

Disk Host:  (There will be two of these, both identical)

* Running Linux on a small form factor board inside the disk enclosure. This
machine will be doing nothing but hosting the disks.  The VM Host will
access these disks as ATA over Ethernet devices.  There will be four bonded
ethernet ports for speed and reliability. 2 ports onboard, two on a second
card, more resiliancy.

* 2 x Hardware RAID5 across 6 500GB disks with a hot spare , giving a total
of (2 x 2.5TB usable) in two RAID5 arrays.  Given 8 ports per RAID
controller, assume two controllers, one for each array.

VM Host: (Only one for now, but more to come later on)

* ATA over Ethernet to the disk host.  Use Software RAID1 to mirror for
redundancy.

* There will be 2 bonded (Gbit) ethernet ports for speed.  Hopefully this
will be fast enough.

*********

I figure this would give the ultimate reliability.  If a disk goes, the RAID
card will flag it and rebuild it.  If a controller goes, the other enclosure
will take care of it.  If a whole enclosure goes, the other one will be
fine.  If the VM host dies, I can boot the VMs on a seperate machine.

What do you think?  Is there going to ba any huge bottlenecks I'm not
seeing?  Will the software RAID1 be good enough?

IMPORTANT: I'm going to be using GFS2 as the file system so that more than
one server can access the volumes at once (still as software RAID1).  Will
this cause a problem for the RAID1?

Thanks in advance,
Phil.

----- Original Message ----- 
From: "Tim Brocklehurst" <t.brocklehurst at henryabram.co.uk>
To: "Phillip Bennett" <phillip at mve.com>; "SLUG-list"
<scottish at mailman.lug.org.uk>
Sent: Tuesday, April 08, 2008 1:11 PM
Subject: RE: [Scottish] RAID Setup[Scanned]

Hi Phil, good to hear from you again.

I would personally go for RAID5 with some hot spares. Disk size depends on
how much you want to spend, but 500GB SATA drives are about £60 on e-buyer
now, so I see no reason to use any less.

You might want to reserve two disks in RAID1 for system, so that gets you to
13 disks for storage.

Assuming two hot spares and 10 data disks (5:1 ratio active to spare is
pretty safe I would think) that would give you 4.5TB of space which could be
accessed at 315 MB/s (roughly), assuming that you can get 35MB/s per disk
(which is possible).

So it really depends on the speed of your network(s) as to whether you can
use the disk-access speed. Remember that a gigabit network will only give
you 120MB/s at most.

I don't have any experience of ATA over Ethernet, but I would think that
normal network speed rules will apply.

Since you're doing RAID on Linux, you can set the system up so it will
inform you of a disk failure. Using hot-spares as well, I doubt a normal
user would actually notice a problem.

Hope this helps a bit,

Tim B.

-----Original Message-----
From: scottish-bounces at mailman.lug.org.uk
[mailto:scottish-bounces at mailman.lug.org.uk] On Behalf Of Phillip Bennett
Sent: Tuesday, April 08, 2008 12:48 PM
To: SLUG-list
Subject: [Scottish] RAID Setup[Scanned]

Hi everyone,

I'm looking into building a RAID setup for our new Virtual Machine host.
I'm trying to condense around 4 or so servers into one box with Xen, using
ATA over Ethernet as the storage solution.

I've found a great case and setup to use (via http://xenaoe.org) that houses
15 drives.  I can fill it up for very little cost, but then I'm wondering
what kind of RAID to use...  We don't have huge space demands and I'm sure
anything will be faster than our current storage solution.  However, I
*would* like some decent speed and redundancy.

So: How would you set something like this up?  RAID5, or maybe some groups
of RAID0+1 or RAID 1+0?  What would give me the best speed as well as
resiliancy?

I was thinking of starting with 15 disks, using two for hot spares, leaving
13 disks between 2 RAID sets - one of 8 and one of 5 disks.  The smaller set
would be RAID5 and used to host the virtual machine images and some less
used files (PC images, drivers etc..) and the larger set would be for our
company data.  I'm just at a loss for how to do it best.  At the price I can
get the parts, I was actually thinking of mirroring the 15 disk enclosure
entirely (ie Software RAID1) to shut the boss up about a single point of
failure.

Does anyone see any problems with this or have any suggestions?

Thanks in advance,
Phil.

_______________________________________________
Scottish mailing list
Scottish at mailman.lug.org.uk
https://mailman.lug.org.uk/mailman/listinfo/scottish

_______________________________________________
Scottish mailing list
Scottish at mailman.lug.org.uk
https://mailman.lug.org.uk/mailman/listinfo/scottish

_______________________________________________
Scottish mailing list
Scottish at mailman.lug.org.uk
https://mailman.lug.org.uk/mailman/listinfo/scottish