Patrick O'Callaghan wrote:
On Tue, 2009-01-20 at 20:06 -0800, Gordon Messmer wrote:
It's also important to bear in mind that under the standard
configuration, you must have at least as much "free" memory as the
largest application in your server, or else that application won't be
able to call external programs.
If you don't have enough memory (RAM+Swap) for the largest app you need
to run, then that app won't be able to run.
You don't understand. Of course you need enough total memory to run
your applications. What I was pointing out was that you need to have
enough "free" memory beyond that for a second copy of the largest
application that you run. Even if it won't be filled, it needs to be
present.
Let's imagine that you have a server with 2GB of RAM, and just 512MB of
swap (maybe based on the idea that swapping will cause the system to
behave badly). Let's also imagine that you've tuned your SQL server to
keep as much data in memory as possible, so it's 1.5GB resident. Now,
if you SQL server has helper applications that it wants to call, it has
to fork() and then exec() to start them. When it does a fork(), the
system doesn't actually copy all of its pages for the new process, but
it does require that the memory be available (the extent to which that
is true depends on your overcommit settings).
This was true in older systems (actually the system just allocated space
for data and stack, since the code segment was shared) but Linux uses a
copy-on-write policy so I don't think it's true any more.
The feature that you're referring to is called "overcommit". I had
hoped that by referring to it *by name*, I could avoid inaccurate
corrections, but I guess not.
Overcommit uses a heuristic algorithm to determine whether or not a
request to allocate more memory than is present (either by malloc or
fork) will be allowed. In many cases, fork() will fail if you do not
have enough memory for a second copy of the application, even though
Linux doesn't copy a complete set of pages during fork(). If you want
the system to work *reliably*, you must have enough free memory for a
second copy of your largest application. In most cases you should
achieve that by having at least as much swap as physical memory.
However, since you don't
have 1.5GB of memory available, the fork() will probably fail, and the
SQL server process can't execute its helper script.
I don't think so (see above).
You're wrong. I helped a friend track down exactly this issue just a
couple of months ago.
This situation would be much harder to diagnose if you had 1GB of swap
and your SQL server were something like 1.3 GB. In that case, it might
sometimes work and sometimes fail depending on how many other processes
were using memory.
And on how much the SQL process is using for a specific run.
If it were specifically using 1.3 GB of memory in a total of 3GB, it
might work some of the time and fail some of the time depending on
whether the rest of the system were using 400MB of memory or 800MB.
So, even if you expect to never *use* swap space, you should have at
least as much swap as physical RAM.
There is no "reasonable" amount of swap that will stop you from running
out of memory in *every* conceivable circumstance. You need to know the
behaviour of your system to make an educated guess.
That's exactly what I'm trying to illustrate, because is is frequently
overlooked. In systems which run applications that consume a lot of
memory, you need to make sure that your total amount of phsycal memory
and swap will leave enough free for a second copy of your very large
application. If not, then fork() may fail, even though fork() doesn't
copy pages.
--
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
