[Chester LUG] ISCSI boxes

[Ian Crowther]

> I did actually change from blockio to fileio and that brought the speed
> issue under better control.
>
> Just before you posted I am moving my important vms from the san to local
> storage. At that point I am going to blow away the SAN and reinstall it with
> Nexenta.
>
> To reduce the overhead I am tempted to try NFS also.
>
> It will take about several hours to copy the 500 GB vm image across.
>
> What is interesting is that Nextenta actually mention this box as a soho
> san, but when you have your DC, VCenter and additional VM, the IO seems to
> swamp the processor. I can only think it may be the linux software raid.
>
> I did try the onboard raid, but Linux ignored it completely ;). Tried it on
> basis that hardware raid (or hardware assisted raid) may reduce IO commands
> on processor.
>
> Anyhows, will report back.....

I've got two N36Ls each with two disks and 1GB ram.

I am suddenly very confused.

I've put a file on a tmpfs with some random content in /media/FILE for
a very unscientific test.

I've got /var on lvm on top of an mdadm raid1 of two SATA disks. When
I dd from /media/FILE to /var or when I read back from it I get nearly
75MB/s. Fair enough (for me!)

However, when I remove a partition from the RAID and write directly to
it I get 2.4 MB/s. iotop says there is no other appreciable IO. top
says dd is the only real CPU user:

   #mdadm --fail /dev/md1 /dev/sdb5
   mdadm: set /dev/sdb5 faulty in /dev/md1
   #mdadm --remove /dev/md1 /dev/sdb5
   mdadm: hot removed /dev/sdb5 from /dev/md1
   #dd if=/media/FILE of=/dev/sdb5&
   [2] 2206
   #killall -USR1 dd
   839856+0 records in
   839856+0 records out
   430006272 bytes (430 MB) copied, 180.702 s, 2.4 MB/s

What am I missing here? Why would it copy so slowly to the raw disk
partition? Right now I'm getting over 70MB/s with the RAID resyncing.

Ian