On Thu, 2006-08-03 at 18:27 +0200, Maarten Maathuis wrote:
I have a kernel specific problem and this seemed like a suitable place to ask.
I would like responces to be CC'ed to me if possible.
I use a 2.6.17-ck1 kernel on an amd64 system. I have observed this
problem on other/older kernels.
Whenever there is serious hard drive activity (such as doing "slocate
-u") ext3_inode_cache and dentry_cache grow to a combined 400-500 MiB.
The behavior of slocate (updatedb) is pretty well-known, but nobody has
come up with a real solution.
The amount of objects is more than half a million.
This will slowly decrease to normal, but will take many hours. It does
not result in any OOM, because i have 1 GiB of memory.
As far as i understand hard drive cache should not be in the slab. Are
these just the inode's, because the amount of memory consumption seems
large for that?
inodes and directory cache entries (dentries). In general, it's a good
idea to cache inodes and dentries that have recently been read. slocate
is a special case since it will traverse all of the directories and
never look at them again (until the next time it runs). The kernel
doesn't have any idea that it may be a good idea to free those objects.
I have found a way to clear the memory (and unfortunately most of the cache):
echo 100 > /proc/sys/vm/nr_hugepages
echo 0 > /proc/sys/vm/nr_hugepages
A better way to clear just the inodes and dentries (that aren't in use)
is:
echo 2 > /proc/sys/vm/drop_caches
This feature is relatively new.
This suggest the kernel can free this memory. It's not the caching
that bothers me, what bothers me is that it seems to reside in the
slab.
I'm not sure why that bothers you. A more common complaint is that all
the inodes and dentries being cached push out other pages to swap.
Completely free memory doesn't do the system any good. The kernel
attempts to keep as much as possible in cache in case something is
needed again. These objects are easily reclaimable, so when more memory
is needed, they can be freed with very little overhead.
I am not a developer, so please keep that in mind when replying.
I hope someone can be of help.
Sincerely,
Maarten Maathuis.
Shaggy
