On Mon, 9 Jan 2006, Timothy A. Holmes wrote:
[...]
permission, and change them, and received notification that my
filesystem was read only. This is an FC3 box running an ext3 file
system. I thought this to be extremely suspicious, so I went to the
console to investigate. When I got there, the screen was full of
scrolling notices that there had been a journal exception.
I rebooted the system, and when it came back telling me that the file
system had been uncleanly unmounted, I forced the file system integrity
check. It promptly kicked me out to a shell, and told me to run the
fsck command, which I did, Its now running the command.
My question has four parts:
1. What happened?
As I am *right now* handling a similar problem I'm in a good position to
to answer: Your disk system probably as a bad problem like a bad sector
(in my case it is a bad sector) and the kernel remounted your partition
read-only to keep things from getting worse.
2. What can I do to prevent it from happening again (this is the
second time in 6 months that this has happened on this box)?
Run a e2fsck on the disk with a '-c' option set (read only test) to find
any bad sectors. Secondly, replace the drive. What that entails depends
largely on how your system is configured (number of drives, using/not
using RAID, partitioning, etc). Personally, I would copy all my data
somewhere else, strip the machine down to metal and rebuild and then copy
my data back. Whether that is an option for you depends on how critical
downtime is for you.
3. what else do I need to do to remedy it
See 2. Also, when you rebuild, you should think about using RAID1 to make
you more resistant to drive failures. Consider the cost of the time and
downtime involved in a system recovery and you will probably conclude that
a second harddisk is cheap.
4. is there any way that I could have known about this earlier, I
interacted with the box last on Saturday afternoon, and it appeared
happy then.
Run the smartd monitoring daemon to get advance notification of failing
drives. (I actually knew I had a developing bad drive - I just have been
too lazy to replace it on my own system before today. Later today I am
going to strip the machine down and re-install Linux on RAID1.)
--
Benjamin Franz
The designer of a new kind of system must participate fully in the implementation.
- Donald E. Knuth
