[Gllug] Drive array speed

[Mike Brodbelt]

On Mon, 2004-05-24 at 19:30, A J Chapman wrote:
> We've got a few boxes at work each with a Compaq Smart Array 5i
> controller and 3 10000rpm scsi drives in a raid 5 array. Copying large
> files happens at around 2.2MB/s (thats bytes, not bits) locally, twice
> that over the network between similar machines. Thats on windows, but I
> figure it shouldn't make much difference to my question: does this seem
> slow? 

It seems pretty slow to me for that spec. Here's a test on my machine,
which is running 4 10k SCSI drives as RAID5, using the software md
driver.

$ time bash test.sh 
304792+0 records in
304792+0 records out
156053504 bytes transferred in 10.913310 seconds (14299374 bytes/sec)

real    0m14.338s
user    0m0.320s
sys     0m4.124s

The test.sh shell script was:-
dd if=testfile of=/usr/local/src/testfile; sync

The testfile was a nice large file to give an reasonable idea of
sustained throughput. So, the reported real time represents the time for
the operation to complete and sync to disk. That comes out at 10.38
MiB/s. Both source and destination partitions lie on the same underlying
physical devices.

You can check read speed with hdparm, which gives you an idea of the raw
I/O speed:-

$ su -c 'hdparm -Tt /dev/md0'
Password: 

/dev/md0:
 Timing buffer-cache reads:   520 MB in  2.00 seconds = 259.91 MB/sec
 Timing buffered disk reads:  110 MB in  3.04 seconds =  36.23 MB/sec

For another point of reference, the same test, conducted on one of my
servers (with a 353410646 byte file) gave these results:-

 time bash test.sh 
690255+1 records in
690255+1 records out

real    0m18.055s
user    0m0.330s
sys     0m5.030s

That works out to 18.67 MiB/s. Again, source and destination partitions
were made from the same underlying devices, but this machine is running
a 4 disk RAID 10 array on an ICP Vortex SCSI controller. The hdparm
reading for it are:-

/dev/sda:
 Timing buffer-cache reads:   128 MB in  0.47 seconds =272.34 MB/sec
 Timing buffered disk reads:  64 MB in  1.03 seconds = 62.14 MB/sec


> Does anyone know what sort of performance I should expect from
> this setup?

On a decent hardware controller with 3 disks, I'd expect a fair bit more
than 2.2 MiB/s transfer locally. I'd expect any decent array controller 
these days to be able to drive 100Mb ethernet at wire speed. I'd check
your SCSI chain for cabling/termination problems, and maybe look for
firmware updates for your RAID controller. A decent controller should be
able to tell you how many SCSI retries have happened on each device,
which can be a help in diagnosing problems. I've had ongoing grief on
one machine at work which went through a number of "unreliable" disks,
until I figured out that the hot-swap caddies I was using couldn't
handle U160.

HTH,

Mike.

-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug