Newbie Borked His RAID Filesystem?

Hi all,

Sorry if this is a repeat post. I tried posting this a few different ways, with little luck.

I was trying to put together a fileserver using a 6-disc RAID-5 array and Fedora 8. I haven't touched Linux before, but I managed to bungle through the install/setup leaning heavily on Google. Disc /dev/sda held the OS, while /dev/sdb-sdg were the RAID discs. During the RAID setup, /dev/sdd failed, at which point I purchased a replacement, and - on a whim - an additional disc to add to the array.

The new discs showed up as /dev/sdd and /dev/sdh. I originally added them straight into the array using mdadm --add /dev/md0 /dev/sdd /dev/sdh. At this point, /dev/sdd replaced the failed disc, and /dev/sdh was added as a spare. BUT ... I remembered that I screwed up and forgot to add a partition to either of the discs. So I --failed and --removed both discs, used fdisk to add a "linux raid autodetect" partition on both. I then added thosepartitions back into the array using mdadm --add /dev/md0 /dev/sdd1 /dev/sdh1 (using the partitions this time.)

>From there, both discs reappeared as before, with /dev/sdd1 taking its rightful place in the array, and /dev/sdh1 as a spare. I then started the the reshape with mdadm --grow /dev/md0 --raid-devices=7. And agonizingly-long week later, the grow finished. I was going to finish off by checking the md0 filesystem with fsck.ext3 and a resize2fs, but I figured I could do that in the morning, and the machine needed a break. I shut the system down, knowing everything was good.

Until I rebooted.

Upon starting up, the kernel now complains about an invalid argument while trying to open /dev/md0. If I  go into "maintenance" mode try to reassemble using mdadm --assemble -v, I get a bunch of messages for /dev/sd[bcefg] stating that they have no superblocks, and they have the wrong uuids. Oddly, mdadm finds superblocks on /dev/sdd and /dev/sdh, which were the "replacement" discs.

I get the feeling I screwed up in two ways:

a) By not creating a partition right off the bat before adding /devs/sdd and sdh to the array. Neither /dev/sdd1 or /dev/sdh1 show up in the /dev directory now. Only /dev/sdd and /dev/sdh are visible. Oddly, fdisk -l *does* show the raid autodetect partitions on both, and fdisk has no problems verifying the partitions. Does the "boot" flag need to be set on these discs? The other five drives in the array have that flag set, but sdd1 sdh1 don't have that flag active.

b) I probably should have updated mdadm.conf, which I neglected to do after the --grow. It currently lists only 6 drives as being in the array, not seven. Can mdadm --scan recover the order of the discs in the array? That said, I couldn't even update mdadm.conf if I knew what to do, as the "maintenace mode" seems to have mounted the rest of my filesystem in read-only mode. :(

Does anyone know a way out of this? I get the feeling if I can get /dev/sdd1 and /dev/sdh1 to show up again, I might be able to force a reassemble, but I'm not sure how. If I can get the array to re-assemble, how can I dump the results of mdadm --scan --verobse to mdadm.conf if the system is in write-only mode?

Thanks in advance!
Tim This is an e-mail from General Dynamics Land Systems. It is for the intended recipient only and may contain confidential and privileged information. No one else may read, print, store, copy, forward or act in reliance on it or its attachments. If you are not the intended recipient, please return this message to the sender and delete the message and any attachments from your computer. Your cooperation is appreciated.