[Gllug] ATA errors

[Tethys]

My backup failed last night, because the device was read only:

> building file list ... done
> rsync: delete_file: unlink "/backup/local/shared/camera/2009-08-24--foq/img_0507.jpg" failed: Read-only file system (30)
> rsync: delete_file: unlink "/backup/local/shared/camera/2009-08-24--foq/.xvpics/img_0507.jpg" failed: Read-only file system (30)

That's never a good sign. Looking through the logs, I see a number of
ATA errors, starting with a timeout, a device error, and a bunch of
HSM violations. A few examples:

> Mar 22 07:56:03 riva kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
> Mar 22 07:56:03 riva kernel: ata2.00: cmd 25/00:00:87:d1:86/00:02:03:00:00/e0 tag 0 dma 262144 in
> Mar 22 07:56:03 riva kernel:          res 40/00:00:00:00:00/ff:ff:ff:ff:ff/00 Emask 0x4 (timeout)
> Mar 22 07:56:03 riva kernel: ata2.00: status: { DRDY }
> Mar 22 07:56:03 riva kernel: ata2: hard resetting link
> Mar 22 07:56:03 riva kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
> Mar 22 07:56:03 riva kernel: ata2.00: configured for UDMA/33
> Mar 22 07:56:03 riva kernel: ata2: EH complete
> Mar 22 07:56:03 riva kernel: SCSI device sdb: 1465149168 512-byte hdwr sectors (750156 MB)
> Mar 22 07:56:03 riva kernel: sdb: Write Protect is off
> Mar 22 07:56:03 riva kernel: SCSI device sdb: drive cache: write back

> Mar 22 07:59:05 riva kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
> Mar 22 07:59:05 riva kernel: ata2.00: BMDMA2 stat 0x6d0009
> Mar 22 07:59:05 riva kernel: ata2.00: cmd 25/00:00:df:ce:87/00:02:03:00:00/e0 tag 0 dma 262144 in
> Mar 22 07:59:05 riva kernel:          res 51/04:80:5f:cf:87/00:01:03:00:00/e0 Emask 0x1 (device error)
> Mar 22 07:59:05 riva kernel: ata2.00: status: { DRDY ERR }
> Mar 22 07:59:05 riva kernel: ata2.00: error: { ABRT }
> Mar 22 07:59:05 riva kernel: ata2.00: configured for UDMA/33
> Mar 22 07:59:05 riva kernel: ata2: EH complete
> Mar 22 07:59:05 riva kernel: SCSI device sdb: 1465149168 512-byte hdwr sectors (750156 MB)
> Mar 22 07:59:05 riva kernel: sdb: Write Protect is off
> Mar 22 07:59:05 riva kernel: SCSI device sdb: drive cache: write back

> Mar 22 08:01:15 riva kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
> Mar 22 08:01:15 riva kernel: ata2.00: cmd 25/00:00:ff:1e:88/00:02:03:00:00/e0 tag 0 dma 262144 in
> Mar 22 08:01:15 riva kernel:          res ff/ff:ff:ff:ff:ff/ff:ff:ff:ff:ff/ff Emask 0x2 (HSM violation)
> Mar 22 08:01:15 riva kernel: ata2.00: status: { Busy }
> Mar 22 08:01:15 riva kernel: ata2.00: error: { ICRC UNC IDNF ABRT }
> Mar 22 08:01:15 riva kernel: ata2: hard resetting link
> Mar 22 08:01:15 riva kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
> Mar 22 08:01:15 riva kernel: ata2.00: configured for UDMA/33
> Mar 22 08:01:15 riva kernel: ata2: EH complete
> Mar 22 08:01:15 riva kernel: SCSI device sdb: 1465149168 512-byte hdwr sectors (750156 MB)
> Mar 22 08:01:15 riva kernel: sdb: Write Protect is off
> Mar 22 08:01:15 riva kernel: SCSI device sdb: drive cache: write back

SMART doesn't seem to be much help here, with the device refusing to
run anything but the mandatory offline test:

> riva:~# smartctl --test=offline /dev/sdb
> smartctl version 5.36 [i686-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
> Home page is http://smartmontools.sourceforge.net/
> 
> Default Self Test Successful
> riva:~# smartctl --test=short /dev/sdb
> smartctl version 5.36 [i686-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
> Home page is http://smartmontools.sourceforge.net/
> 
> Short offline self test failed [unsupported field in scsi command]
> riva:~# smartctl --test=long /dev/sdb
> smartctl version 5.36 [i686-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
> Home page is http://smartmontools.sourceforge.net/
> 
> Long (extended) offline self test failed [unsupported field in scsi command]

It's a Seagate drive:

> Host: scsi1 Channel: 00 Id: 00 Lun: 00
>   Vendor: ATA      Model: ST3750528AS      Rev: CC38
>   Type:   Direct-Access                    ANSI SCSI revision: 05

I'm running 2.6.18-128.4.1.el5 on CentOS 5.

Any ideas? Does anyone know enough about the ATA spec to tell me what
these errors actually mean? Is this a genuine hardware failure? If so,
where's it likely to be? Drive? Controller? Cable? The drive is only a
few weeks old, so while sometimes shit happens, I want to investigate
the (probably more likely) alternatives as well.

Tet
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug