Re: Problem with md: Not rebuilding rai5

[Neil Brown <neilb@suse.de>]

On Tuesday August 29, nico-kernel20060829@schottelius.org wrote:
> Neil Brown [Tue, Aug 29, 2006 at 07:26:52PM +1000]:
> > On Tuesday August 29, nico-kernel20060829@schottelius.org wrote:
> > > Hello!
> > > 
> > > I created a degrated raid5 on top of md1 and hde1. Then moved the data
> > > from /dev/hdk to the mounted raid5, and then added hdk1 (repartitoned)
> > > to the array. The sync began, but after that hde1 was faulty.
> > 
> > So you created a raid5 containing one drive that was already faulty.
> > That is unfortunate!
> 
> And reported in the manpage of mdadm to be usable (simply specify "missing"
> as keyword).

That's not what I meant.
Creating a 3 drive raid5 with 2 working drives and one 'missing' drive
is fine.
Creating a 3 drive raid5 with one good drive, one faulty drive and one
missing drive is unfortunate.  But the doing anything with a faulty
drive is unfortunate.

> 
> > > I removed it, readded it, but now I've a raid5 with only one active
> > > disk (which should not be possible imho, a raid5 always needs 2 disks)
> > > AND what's even stranger for me, I've two spare disks.
> > 
> > If you have a raid5 with 2 working drives and one fails, how many
> > working drives do you expect to be left?  1.  So the raid is no longer
> > fully functional.
> 
> That is what I also thought. But there are some points that make me
> wonder:
>    a) why is hdk1 marked as spare? I added it this morning and the
>    rebuilt began. Though something happened (I do not know what)
>    and made hdk1 not beeing in the array. (dmesg output
>    is now available at
>    http://home.schottelius.org/~nico/linux/debug/raid/raid5.strange.dmesg)

You added hdk1 as a spare.  md noticed that the array was degraded and
a spare was available so it started recovery of the missing drive on
to the spare.  It remains as a 'spare' until recovery is complete.
Then it becomes a full member.

However recovery didn't complete due to a read error on hde.  So hdk1
is still a spare.
Then you removed and re-added hde1, so it was a spare too.  md didn't
try to reconstruct onto either spare as it didn't have enough working
drives to perform a reconstruction.

> 
>    b) what's the reason, linux does not mark md2 as faulty?
> 

This is no sense in which an array is marked faulty.  There is no
place to put the marking.
If you write to md2, it will fail.
If you read from md2, it might succeed or it might not, depending on
whether the data you try to read is stored on the working device or on
a failed device.

> > You might be able to read some data, but you want
> > able to write.
> > What did you expect to happen when hde1 failed?
> 
> I expected hdk1 and md1 to work.

However md hasn't completely the recovery onto hdk1, so there is
nothing that can be done.

> 
> > I suggest you:
> >   - recreate the array over md1 and hde1
> >   - copy the data back to hdk
> >   - stop the array
> >   - replace hde1
> >   - make the array.
> >   - read the entire array (dd > /dev/null) to make sure it is safe
> >   - copy data back from hdk
> 
> Will linux detect, that md1 and hde1 are from the same array
> and will it see which harddisk is xored with which one?

Linux won't detect anything.  But if you create the array in the same
way that you did before, with md1 and hde1, the data will still be
where it was.  And when you then read from the new md2, you will get
the data that was there before hde1 failed (as long as you don't read
from the block on hde1 that is faulty).

Good luck.

NeilBrown
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/