OOM Killer killing whole system

[Anton Titov <a.titov@host.bg>]

Yesterday I accidently noticed few OOM killer messages in the system log
and leaved a console tailing the log for the night. In 6 in the morning
OOM killer got mad generating 500 lines in the log and 5 minutes later
system closed the ssh connection and became inresponsive. The guy in the
datacenter told me that when he attached keyboard even caps lock was not
working. Inspite of this the system still was responsive (only to) ping.

The strange thing is this machine is relatively light loaded - now after
6 hours being up free shows:
             total       used       free     shared    buffers    cached
Mem:       2075468    1148564     926904          0     123472    314516
-/+ buffers/cache:     710576    1364892
Swap:      1004020          0    1004020

Load average stays under 0.5 most of the time. In 6 in the morning it
should be almost no load (there is no crons scheduled at that time).

I'm attaching messages from the log and my .config.

Anton Titov

Attachment: log.gz
Description: GNU Zip compressed data

Attachment: config.gz
Description: GNU Zip compressed data