[Gllug] Problems rebuilding a RAID5 array after a failed disc on Centos 5
Oliver Howe
ojhowe at gmail.com
Sat Sep 25 09:17:23 UTC 2010
[root at backup2 ~]# cat /etc/mtab
/dev/sda3 / ext3 rw 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0
devpts /dev/pts devpts rw,gid=5,mode=620 0 0
/dev/sda1 /boot ext3 rw 0 0
tmpfs /dev/shm tmpfs rw 0 0
/dev/sdb2 /disk01 ext3 rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0
[root at backup2 ~]#
here is the output from dmesg
SCSI subsystem initialized
3ware 9000 Storage Controller device driver for Linux v2.26.08.002-2.6.18.
ACPI: PCI Interrupt 0000:06:00.0[A] -> GSI 16 (level, low) -> IRQ 169
PCI: Setting latency timer of device 0000:06:00.0 to 64
scsi0 : 3ware 9000 Storage Controller
3w-9xxx: scsi0: Found a 3ware 9000 Storage Controller at 0xda300000, IRQ:
169.
3w-9xxx: scsi0: Firmware FE9X 3.08.00.016, BIOS BE9X 3.08.00.004, Ports: 24.
Vendor: AMCC Model: 9650SE-24M DISK Rev: 3.08
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sda: 156227584 512-byte hdwr sectors (79989 MB)
sda: Write Protect is off
sda: Mode Sense: 23 00 00 00
SCSI device sda: drive cache: none
SCSI device sda: 156227584 512-byte hdwr sectors (79989 MB)
sda: Write Protect is off
sda: Mode Sense: 23 00 00 00
SCSI device sda: drive cache: none
sda: sda1 sda2 sda3
sd 0:0:0:0: Attached scsi disk sda
Vendor: AMCC Model: 9650SE-24M DISK Rev: 3.08
Type: Direct-Access ANSI SCSI revision: 05
sdb : very big device. try to use READ CAPACITY(16).
SCSI device sdb: 6835791872 512-byte hdwr sectors (3499925 MB)
sdb: Write Protect is off
sdb: Mode Sense: 23 00 00 00
SCSI device sdb: drive cache: none
sdb : very big device. try to use READ CAPACITY(16).
SCSI device sdb: 6835791872 512-byte hdwr sectors (3499925 MB)
sdb: Write Protect is off
sdb: Mode Sense: 23 00 00 00
SCSI device sdb: drive cache: none
sdb: sdb1 sdb2
sd 0:0:1:0: Attached scsi disk sdb
Vendor: AMCC Model: 9650SE-24M DISK Rev: 3.08
Type: Direct-Access ANSI SCSI revision: 05
sdc : very big device. try to use READ CAPACITY(16).
SCSI device sdc: 10741958656 512-byte hdwr sectors (5499883 MB)
sdc: Write Protect is off
sdc: Mode Sense: 23 00 00 00
SCSI device sdc: drive cache: none
sdc : very big device. try to use READ CAPACITY(16).
SCSI device sdc: 10741958656 512-byte hdwr sectors (5499883 MB)
sdc: Write Protect is off
sdc: Mode Sense: 23 00 00 00
SCSI device sdc: drive cache: none
sdc: unknown partition table
sd 0:0:2:0: Attached scsi disk sdc
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
audit(1285090871.535:2): enforcing=1 old_enforcing=0 auid=4294967295
security: 3 users, 6 roles, 1587 types, 172 bools, 1 sens, 1024 cats
security: 59 classes, 49762 rules
SELinux: Completing initialization.
SELinux: Setting up existing superblocks.
SELinux: initialized (dev sda3, type ext3), uses xattr
SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses genfs_contexts
SELinux: initialized (dev devpts, type devpts), uses transition SIDs
SELinux: initialized (dev eventpollfs, type eventpollfs), uses task SIDs
SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev futexfs, type futexfs), uses genfs_contexts
SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
SELinux: initialized (dev cpuset, type cpuset), not configured for labeling
SELinux: initialized (dev proc, type proc), uses genfs_contexts
SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
audit(1285090871.836:3): policy loaded auid=4294967295
input: PC Speaker as /class/input/input1
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:1:0: Attached scsi generic sg1 type 0
sd 0:0:2:0: Attached scsi generic sg2 type 0
Intel(R) PRO/1000 Network Driver - version 7.2.7-k2-NAPI
Copyright (c) 1999-2006 Intel Corporation.
ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI 16 (level, low) -> IRQ 169
PCI: Setting latency timer of device 0000:03:00.0 to 64
e1000: 0000:03:00.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
00:15:17:29:dc:62
ACPI: PCI Interrupt 0000:00:1f.3[B] -> GSI 19 (level, low) -> IRQ 233
hdb: ATAPI 24X CD-ROM drive, 256kB Cache, UDMA(33)
Uniform CD-ROM driver Revision: 3.20
intel_rng: FWH not detected
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:03:00.1[B] -> GSI 17 (level, low) -> IRQ 185
PCI: Setting latency timer of device 0000:03:00.1 to 64
e1000: 0000:03:00.1: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
00:15:17:29:dc:63
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 18 (level, low) -> IRQ 177
PCI: Setting latency timer of device 0000:04:00.0 to 64
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378 (0x778), irq 7 [PCSPP,TRISTATE,EPP]
e1000: 0000:04:00.0: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
00:30:48:63:c7:44
e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
ACPI: PCI Interrupt 0000:04:00.1[B] -> GSI 19 (level, low) -> IRQ 233
PCI: Setting latency timer of device 0000:04:00.1 to 64
e1000: 0000:04:00.1: e1000_probe: (PCI Express:2.5Gb/s:Width x4)
00:30:48:63:c7:45
e1000: eth3: e1000_probe: Intel(R) PRO/1000 Network Connection
lp0: using parport0 (interrupt-driven).
lp0: console ready
ACPI: Power Button (FF) [PWRF]
ACPI: Power Button (CM) [PWRB]
ibm_acpi: ec object not found
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
device-mapper: ioctl: 4.11.0-ioctl (2006-09-14) initialised:
dm-devel at redhat.com
EXT3 FS on sda3, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev sda1, type ext3), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
kjournald starting. Commit interval 5 seconds
EXT3 FS on sdb2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev sdb2, type ext3), uses xattr
Adding 16386292k swap on /dev/sda2. Priority:-1 extents:1 across:16386292k
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses
genfs_contexts
IA-32 Microcode Update Driver: v1.14a <tigran at veritas.com>
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
process `sysctl' is using deprecated sysctl (syscall)
net.ipv6.neigh.lo.retrans_time; Use net.ipv6.neigh.lo.retrans_time_ms
instead.
ADDRCONF(NETDEV_UP): eth0: link is not ready
e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
ADDRCONF(NETDEV_UP): eth1: link is not ready
e1000: eth1: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex
e1000: eth1: e1000_watchdog: 10/100 speed: disabling TSO
ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
audit(1285090889.292:4): audit_pid=2179 old=0 by auid=4294967295
subj=system_u:system_r:auditd_t:s0
SELinux: initialized (dev rpc_pipefs, type rpc_pipefs), uses genfs_contexts
Bluetooth: Core ver 2.10
NET: Registered protocol family 31
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP ver 2.8
Bluetooth: L2CAP socket layer initialized
Bluetooth: RFCOMM socket layer initialized
Bluetooth: RFCOMM TTY layer initialized
Bluetooth: RFCOMM ver 1.8
Bluetooth: HIDP (Human Interface Emulation) ver 1.1
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
eth0: no IPv6 routers present
eth1: no IPv6 routers present
3w-9xxx: scsi0: AEN: INFO (0x04:0x0029): Verify started:unit=2.
3w-9xxx: scsi0: AEN: INFO (0x04:0x002B): Verify completed:unit=2.
[root at backup2 ~]#
after re-reading this I tried to fsck /dev/sdc but got these results
[root at backup2 ~]# fsck /dev/sdc
fsck 1.39 (29-May-2006)
e2fsck 1.39 (29-May-2006)
Group descriptors look bad... trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/sdc
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
[root at backup2 ~]#
On Fri, Sep 24, 2010 at 4:58 PM, tid <td at bloogaloo.co.uk> wrote:
> On 22 September 2010 21:45, John Edwards <john at cornerstonelinux.co.uk>
> wrote:
> > As this is a problem with your 3ware RAID controller (which is
> > propriety and non-open) I think their support department would
> > be the best people to ask.
>
> Having recently been through the rather terrifying experience of
> recovering 5Tb+ of data
> from a failed SAN based on a 3ware card, I personally recommend using
> this as the
> last option if no other alternatives are available. I wasn't impressed
> with the call-center
> drone's level of technical expertise.
>
> What filesystem type were you using on the failed disk?
> --
> Gllug mailing list - Gllug at gllug.org.uk
> http://lists.gllug.org.uk/mailman/listinfo/gllug
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.lug.org.uk/pipermail/gllug/attachments/20100925/99b38d95/attachment.html>
-------------- next part --------------
--
Gllug mailing list - Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug
More information about the GLLUG
mailing list