Hi
Please help.
I was testing 2.6.16-rc4 and seem to have lost my data filesystem :(
I know it's called scratch - but my wife went and stored data on there -
grrr!
Anyway, I'm now in trouble.
I am running XFS over LVM over a 3-disk RAID5.
Something (libata?) caused some bogus read errors (see kernel-ide for
details) which threw 2 drives out of my raid array:
sd 0:0:0:0: SCSI error: return code = 0x8000002
sda: Current: sense key: Medium Error
Additional sense: Unrecovered read error - auto reallocate failed
end_request: I/O error, dev sda, sector 390716735
raid5: Disk failure on sda1, disabling device. Operation continuing on 2
devices
ata2: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata2: status=0x51 { DriveReady SeekComplete Error }
sd 1:0:0:0: SCSI error: return code = 0x8000002
sdb: Current: sense key: Medium Error
Additional sense: Unrecovered read error - auto reallocate failed
end_request: I/O error, dev sdb, sector 390716735
raid5: Disk failure on sdb1, disabling device. Operation continuing on 1
devices
RAID5 conf printout:
--- rd:3 wd:1 fd:2
disk 0, o:1, dev:sdd1
disk 1, o:0, dev:sdb1
disk 2, o:0, dev:sda1
xfs_force_shutdown(dm-0,0x1) called from line 338 of file
fs/xfs/xfs_rw.c. Return address = 0xc020c0e9
Filesystem "dm-0": I/O Error Detected. Shutting down filesystem: dm-0
Please umount the filesystem, and rectify the problem(s)
I/O error in filesystem ("dm-0") meta-data dev dm-0 block
0x640884a ("xlog_bwrite") error 5 buf count 262144
XFS: failed to locate log tail
XFS: log mount/recovery failed: error 5
XFS: log mount failed
After checking with mdadm --examine I was able to reconstruct the array
using the 2 booted devices as they both had the same event count:
raid5: device sdb1 operational as raid disk 1
raid5: device sda1 operational as raid disk 2
raid5: allocated 3155kB for md1
raid5: raid level 5 set md1 active with 2 out of 3 devices, algorithm 2
RAID5 conf printout:
--- rd:3 wd:2 fd:1
disk 1, o:1, dev:sdb1
disk 2, o:1, dev:sda1
I restarted lvm and remounted the drive but it failed like this:
XFS mounting filesystem dm-0
ata2: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata2: status=0x51 { DriveReady SeekComplete Error }
ata2: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata2: status=0x51 { DriveReady SeekComplete Error }
sd 1:0:0:0: SCSI error: return code = 0x8000002
sdb: Current: sense key: Medium Error
Additional sense: Unrecovered read error - auto reallocate failed
end_request: I/O error, dev sdb, sector 390716735
raid5: Disk failure on sdb1, disabling device. Operation continuing on 1
devices
ata1: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata1: status=0x51 { DriveReady SeekComplete Error }
ata1: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata1: status=0x51 { DriveReady SeekComplete Error }
ata1: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata1: status=0x51 { DriveReady SeekComplete Error }
ata1: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata1: status=0x51 { DriveReady SeekComplete Error }
ata1: no sense translation for op=0x2a cmd=0x3d status: 0x51
ata1: status=0x51 { DriveReady SeekComplete Error }
sd 0:0:0:0: SCSI error: return code = 0x8000002
sda: Current: sense key: Medium Error
Additional sense: Unrecovered read error - auto reallocate failed
end_request: I/O error, dev sda, sector 390716735
raid5: Disk failure on sda1, disabling device. Operation continuing on 0
devices
RAID5 conf printout:
--- rd:3 wd:0 fd:3
disk 1, o:0, dev:sdb1
disk 2, o:0, dev:sda1
xfs_force_shutdown(dm-0,0x1) called from line 338 of file
fs/xfs/xfs_rw.c. Return address = 0xc020c0e9
Filesystem "dm-0": I/O Error Detected. Shutting down filesystem: dm-0
Please umount the filesystem, and rectify the problem(s)
I/O error in filesystem ("dm-0") meta-data dev dm-0 block
0x640884a ("xlog_bwrite") error 5 buf count 262144
XFS: failed to locate log tail
XFS: log mount/recovery failed: error 5
XFS: log mount failed
I then then rebooted into 2.6.15 (where it's been happy for weeks and
weeks!)
raid5: device sdb1 operational as raid disk 1
raid5: device sda1 operational as raid disk 2
raid5: allocated 3155kB for md1
raid5: raid level 5 set md1 active with 2 out of 3 devices, algorithm 2
RAID5 conf printout:
--- rd:3 wd:2 fd:1
disk 1, o:1, dev:sdb1
disk 2, o:1, dev:sda1
Setting up LVM Volume Groups...
Reading all physical volumes. This may take a while...
/dev/hda: open failed: No medium found
Found volume group "video_vg" using metadata type lvm2
1 logical volume(s) in volume group "video_vg" now active
haze:~# mount /scratch
mount: /dev/video_vg/video_lv: can't read superblock
haze:~# xfs_repair -n /dev/video_vg/video_lv
Phase 1 - find and verify superblock...
superblock read failed, offset 0, size 524288, ag 0, rval 0
fatal error -- Invalid argument
haze:~# lvdisplay /dev/video_vg/video_lv
--- Logical volume ---
LV Name /dev/video_vg/video_lv
VG Name video_vg
LV UUID cg2yc1-dO6V-3GlY-JnhY-4nXi-V85s-WTIFml
LV Write Access read/write
LV Status available
# open 0
LV Size 372.61 GB
Current LE 95389
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:0
Any sugestions?
David
--
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
