You really might want to turn on the swap.
In 2.4 you don't have the 4/4g split, and have a 1g/3g split. That is a very different memory zone balancing situation than 4g/300MB you're running the 2.6 kernel under.
Unless you're happy with not using 300MB on your server...
I *REALLY* don't want the machine to swap. If I have a choice between using 300MB less RAM and not ever having to touch disk unnecessarily, I'll choose 300MB less RAM until the VM system gets the swap-less kinks worked out of it. The way the box is setup, it should *never* have a reason to page something to disk.
