I've got a big problem with a server I run. It's a remote machine that I
administer purely through SSH. I was upgrading the OS from FC3 to FC4
using yum.
Yum barfed halfway through the upgrade process. I think this was to do
with lack of free disk space on /var. I did a 'yum clean all' and that
freed up a lot of space, and then I re-ran the 'yum upgrade' command.
However, this time it froze for a long time at the Transaction Test stage,
so in another terminal I checked top and saw that yum was eating up all
the system RAM. Eventually I had to kill the yum process.
I rebooted the machine as I was getting filesystem errors, and it appeared
that the /usr partition was now mounted read-only. So I rebooted the
machine in order to get all the disks checked at boot time (before they
were mounted - I didn't want to unmount /usr and be stuck) and it never
came back up. I contacted the tech support team who have rebooted the
server for me, but now I have the following issues:
[*] No ssh connection possible. I have managed to open one ssh
connection, but now I can't access any more and can't upload any files
via scp etc. If I try to ssh in from another machine I get the following
error (and both my hosts.allow and hosts.deny files are empty):
ssh_exchange_identification: Connection closed by remote host
[*] No access to the internet, but it still pings its gateway IP. I can't
ping external IP's or domain names, but localhost and the hostname
respond ok. Named won't start and gives the following error:
Starting named:
Error in named configuration:
/usr/sbin/named-checkconf: error while loading shared libraries:
libgssapi_krb5.so.2: cannot open shared object file: No such file or
directory
[FAILED]
[*] I also can't use rpm or yum at all any more. It appears that I have
several versions of rpm installed. If I try and use rpm I get output such
as:
rpm: error while loading shared libraries: librpm-4.4.so: cannot open
shared object file: No such file or directory
[*] and if I try and run 'yum upgrade' I get the following error:
rpmdb: Program version 4.2 doesn't match environment version
error: db4 error(22) from dbenv->open: Invalid argument
error: cannot open Packages index using db3 - Invalid argument (22)
error: cannot open Packages database in /var/lib/rpm
Traceback (most recent call last):
File "/usr/bin/yum", line 7, in ?
yummain.main(sys.argv[1:])
File "/usr/share/yum-cli/yummain.py", line 51, in main
base.getOptionsConfig(args)
File "/usr/share/yum-cli/cli.py", line 150, in getOptionsConfig
self.conf = yumconf(configfile = conffile, root = root)
File "config.py", line 295, in __init__
File "config.py", line 360, in _getsysver
TypeError: rpmdb open failed
So basically my system appears to be totally screwed up. I really need to
get this fixed without having to resort to a fresh install. It would seem
sensible to try and fix the network issue first, so I can download
packages to fix other problems, but does that involve getting named
working first?
How do I fix rpm? Which I presume is also the cause of yum's failure etc.
Any help at all would be greatly appreciated.
Ben
