Re: [Lhms-devel] [PATCH 0/7] Fragmentation Avoidance V19

[Badari Pulavarty <pbadari@gmail.com>]

On Thu, 2005-11-03 at 11:35 -0500, Jeff Dike wrote:
> On Thu, Nov 03, 2005 at 12:07:33AM -0600, Rob Landley wrote:
> > I want UML to 
> > be able to hand back however much memory it's not using, but handing back 
> > individual pages as we free them and inserting a syscall overhead for every 
> > page freed and allocated is just nuts.  (Plus, at page size, the OS isn't 
> > likely to zero them much faster than we can ourselves even without the 
> > syscall overhead.)  Defragmentation means we can batch this into a 
> > granularity that makes it worth it.
> 
> I don't think that freeing pages back to the host in free_pages is the
> way to go.  The normal behavior for a Linux system, virtual or
> physical, is to use all the memory it has.  So, any memory that's
> freed is pretty likely to be reused for something else, wasting any
> effort that's made to free pages back to the host.
> 
> The one counter-example I can think of is when a large process with a
> lot of data exits.  Then its data pages will be freed and they may
> stay free for a while until the system finds other data to fill them
> with.
> 
> Also, it's not the virtual machine's job to know how to make the host
> perform optimally.  It doesn't have the information to do it.  It's
> perfectly OK for a UML to hang on to memory if the host has plenty
> free.  So, it's the host's job to make sure that its memory pressure
> is reflected to the UMLs.
> 
> My current thinking is that you'll have a daemon on the host keeping
> track of memory pressure on the host and the UMLs, plugging and
> unplugging memory in order to keep the busy machines, including the
> host, supplied with memory, and periodically pushing down the memory
> of idle UMLs in order to force them to GC their page caches.
> 
> With Badari's patch and UML memory hotplug, the infrastructure is
> there to make this work.  The one thing I'm puzzling over right now is
> how to measure memory pressure.

Yep. This is the exactly the issue other product groups normally raise
on Linux. How do we measure memory pressure in linux ? Some of our
software products want to grow or shrink their memory usage depending
on the memory pressure in the system. Since most memory is used for
cache, "free" really doesn't indicate anything -they are monitoring
info in /proc/meminfo and swapping rates to "guess" on the memory
pressure. They want a clear way of finding out "how badly" system
is under memory pressure. (As a starting point, they want to find out
out of "cached" memory - how much is really easily "reclaimable" 
under memory pressure - without swapping). I know this is kind of 
crazy, but interesting to think about :)

Thanks,
Badari

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/