I note that you say "server" above. I really, really recommend against running X on a server, or actually, on any system where you can avoid it. It's a gigantic program which requires special access to hardware and runs as root -- there are undoubtedly multiple serious security problems just waiting to bite you. You can still run X on your management workstation, and run GUI tools remotely if need be.
agree 100% BUT alas if you want to install oracle server then you still need X which to me is madness but thats just the way it is. If i need to install oracle then i make sure the init level is 3 and not 5
although this being FC5 i doubt its an oracle box