For the record, I was able to recover the volume group. I failed to
copy the text of the shell session to a file, so this is from memory,
but I'm pretty sure it's accurate enough to provide clues to anyone
in a similar situation.
What I did was sacrifice the two base volumes hda2 and hda4 below.
hda4 was converted to a large ext3 partition to hold what I needed
out of hda2, and to hold the results of using photorec to recover
what it could find of my files which had been stored physically in
hde5. hda2 was then sacrificed and converted to an ext3 partition, to
hold a single-partition minimal install of FC5.
While I was using photorec to recover my files, all 70,000+ of them
scattered across 170 or so recup_xxx directories, I also used
Christophe Grenier's testdisk (which photorec is distributed with) to
see if it would find the lvm information in hda3 and hde5. It did,
and I told it to write the results of scanning hda3 to hda3, or at
least I think I did. I don't know if that step helped or not. Neither
of those tools are what I would call verbose.
By refraining from using LVM in the minimal install in hda2, I think
I was able to avoid further conflicting LVM headers, and was able to
boot the install of FC6 in hde2 once or twice more to mount /dev/hda1
and /dev/hda2, and get the name of the kernel and boot files in hda2
and copy them into /boot/grub/grub.conf in hda1 so I could boot the
FC5 in hda2. (I had just chained to grub on hde1 from grub in hda1,
so that simplified things a bit.)
So I booted the new install of FC5 from hda2, re-read the man entries
on lvm and its subcommands, pvdisplay was able to see both hda3 and
hde5, so I took hope.
I mounted /dev/hda1, cd-ed into the old fc5's /etc/lvm and got the
uuids of both physical volumes from backup/fc5 in there. I checked
dates, and checked the uuids in the entries in the archive directory
to make sure they matched, and that I wasn't going to end up trying
to restore the wrong headers.
One point of possible confusion is that the uuids of the logical
volumes which are recorded there won't help if the headers for the
physical volumes have been written over. The original problem (before
I screwed things up worse in my initial attempts at recovery) was
that attempting to use logical volumes in installing ubuntu on hde2
had overwritten the headers of hde5, and then lvm couldn't find all
of the fc5 logical volume group and refused to mount any of it.
Then I did pvcreate --restorefile backup/fc5 --uuid <uuid-of-physical-
hde5> /dev/hde5 . It first complained that it couldn't find the
volume, and then it said volume created.
So then I did pvcreate --restorefile backup/fc5 --uuid <uuid-of-of-
physical-hda3> /dev/hda3 with no complaints.
So I next did the vgcrestorefile --file backup/fc5 (I think that was
the parameters, wish I had saved the session text) and that gave me a
success message. So I checked the file system with the graphical lvm
widget, and it saw all the lost logical volumes, including the one in
hde5.
In order to boot from hda1 again, I had to get rid of the entries I
had in /etc/fstab referencing the old fc4 leftovers in hda2 that were
no longer there. But now it boots fine. I'll see if booting fc6
works, and whether it screws things up, but only after backing up my
important stuff from the old fc5 install.
Things I learned:
(1) Read the lvm howto at http://tldp.org/HOWTO/LVM-HOWTO/ before
trying anything fancy like including physical volumes from more than
one drive in a volume group.
(2) Don't multiboot other distros that use LVM if do mix physical
drives in a volume group. (I guess this is kind of the same issue as,
when multi-booting MSWxx and Linux or BSD distros, let MSWindwoes or
Partition Magic, you only want to do the base partitions once and
then leave them alone.)
(3) I didn't have much luck with using the LVM tools from one distro
on another (Ubuntu live DVD and Knoppix CD). Probably not a good idea.
(4) I might want to consider not having /usr on an LVM volumes, even
though Fedora Core installs all optional packages there instead of /
usr/local. Part of the problem was that, with /usr in an LVM managed
volume, I couldn't access the tools I needed to fix the LVM volumes.
Alternatively, I might want to keep a live CD with the LVM tools on
it. In fact, maybe I should make a feature request to put the LVM
tools and man pages on the FC restore CDs. (The tools are incomplete,
and the man pages not present on the current restore CDs.)
(0) Backup! Even my kids' experiments with the digital camera have
more value than the cost of a few DVD-Rs and/or CD-Rs or CD-R/Ws. (I
wonder if I'll really believe it this time. I thought I had learned
this lesson when resizing the partitions for the sixth time trying to
find an optimal balance for MSW2k and openbsd botched and left me
backing up development work from the MSW2k partition with MSDOS
through a floppy, but I guess the lesson didn't come home with me
from work.)
On 2006/12/16, at 14:55, Joel Rees wrote:
I was going to give Ubuntu a try and started an install. Since I
wanted to leave my FC5 intact and dual-boot, I was using the Ubuntu
LVM tools to set up LVM and walked on the FC5 LVM headers or
whatever they are called.
My partition setup:
hda1: boot FC5 base partition
hda2: LVM volume, leftovers of FC4 that I've been too lazy to take
a backup of, still intact
hda3: the lost LVM volumes of FC5, /usr et. al., except for one
volume on hde5
hda4: unused (freeBSD install currently sitting idle)
hde1: base partition, failed installing Ubuntu boot here, installed
FC6 instead for LVM tools
hde2: FC6 /usr et. al., in LVM (which I am now sure was a mistake)
hde3 or hde4, I don't remember which: DOS extended
hde5: LVM PV which was used as shared volume on FC5
hde6: swap
hde7: FAT32 share
I think that where things went south was when I added another
volume group on hde using the Ubuntu LVM tools. I saw on
http://tldp.org/HOWTO/LVM-HOWTO/
a few warning words to the effect that one shouldn't add physical
partitions (hde5) from a second drive to a volume group on one
drive (in hda3), and then set up a separate volume group (the
Ubuntu volumes) on the second drive. After partitioning, at the
step where Ubuntu would start actually installing, it complained
that LVM could no longer find the PV that had been in hde5 in the
FC5 volume group.
I found this in a blog by one jalal, apparently of gnomedia.com
(now apparently only in google cache):
Here is what I did: First find out the old UID’s of the
partitions, this is in the /etc/lvm/backup/system file. They are
quite long… make sure you get the UID for the physical volumes.
$pvcreate -u sdSD-2343-SD939-adIda2 /dev/hda6
$pvcreate -u dk33kd-929293nd-adfja298a /dev/hdd1
$vgcreate -v system /dev/hda7 /dev/hdd1
$vgcfgrestore -f /etc/lvm/backup/system system
and lo!, all data present and correct!
In fact, I just rebooted the system and was back where I had
started with the additional benefit of an extra 60Gb of disk
space, because now I had the extra partition properly included.
[Note: in the lines using pvcreate... above I could have used:
$ pvcreate –restorefile /etc/lvm/backup/system
to automatically find the ID’s but I hadn’t realized that at
the time. Without the UID’s then the vgcfgrestore will not find
the physical volumes that it needs to recreate the volume group.]
I tried following that but forget the vgcfgrestore step, and
apparently have lost what headers remained. vgscan no longer sees
the FC5 volume group at all. I do still have FC5's /etc/lvm in the
boot partition (which will try to boot and fail, of course). (Kind
of wish the lvm commands had been under /sbin instead of /usr/sbin .)
Any pointers where I can go to find information that might help me
recover the data?
I'm thinking maybe I should install another FC5 in hda4, just
enough to use the command-line pvcreate tools, and all in one
partition, so I can scrub the FC6 install and get that other LVM
volume off of hde .
