I think we are getting into more and more of a mess with the existing
memory policies. The refcount issue with shmem is just one bad symptom of
it. Memory policies were intended to be process based and
taking them out of that context causes various issues.
I have thought for a long time that we need something to replace memory
policies especially since the requirements on memory policies go far
beyond just being process based. So some requirements and ideas about
memory policies.
1. Memory policies must be attachable to a variety of objects
- Device drivers may need to control memory allocations in
devices either because their DMA engines can only reach a
subsection of the system or because memory transfer
performance is superior in certain nodes of the system.
- Process. This is the classic usage scenario
- File / Socket. One may have particular reasons to place
objects on a set of nodes because of how the threads of
an application are spread out in the system.
- Cpuset / Container. Some simple support is there with
memory spreading today. That could be made more universal.
- System policies. The system policy is currently not
modifiable. It may be useful to be able to set this.
Small NUMA systems may want to run with interleave by default
- Address range. For the virtual memory address range
this is included in todays functionality but one may also
want to control the physical address range to make sure
f.e. that memory is allocated in an area where a device
can reach it.
- Memory policies need to be attachable to types of pages.
F.e. executable pages of a threaded application are best
spread (or replicated) whereas the stack and the data may
best be allocated in a node local way.
Useful categories that I can think of
Stack, Data, Filebacked pages, Anonymous Memory,
Shared memory, Page tables, Slabs, Mlocked pages and
huge pages.
Maybe a set of global policies would be useful for these
categories. Andy hacked subsystem memory policies into
shmem and it seems that we are now trying to do the same
for hugepages. Maybe we could get to a consistent scheme
here?
2. Memory policies need to support additional constraints
- Restriction to a set of nodes. That is what we have today.
- Restriction to a container or cpuset. Maybe restriction
to a set of containers?
- Strict vs no strict allocations. A strict allocation needs
to fail if the constraints cannot be met. A non strict
allocation can fall back.
- Order of allocation. Higher order pages may require
different allocation constraints? This is like a
generalization of huge page policies.
- Locality placement. These are node local, interleave etc.
3. Additional flags
- Automigrate flag so that memory touched by a process
is moved to a memory location that has best performance.
- Page order flag that determines the preferred allocation
order. Maybe useful in connection with the large blocksize
patch to control anonymous memory orders.
- Replicate flags so that memory is replicated.
4. Policy combinations
We need some way to combine policies in a systematic way. The current
hieracy from System->cpuset->proces->memory range does not longer
work if a process can use policies set up in shmem or huge pages.
Some consistent scheme to combine memory policies would also need
to be able to synthesize different policies. I.e. automigrate
can be combined with node local or interleave and a cpuset constraint.
5. Management tools
If we make the policies more versatile then we need the proper
management tools in user space to set and display these policies
in such a way that they can be managed by the end user. The esoteric
nature of memory policy semantics makes them difficult to comprehend.
6. GFP_xx flags may actually be considered as a form of policy
i.e. GFP_THISNODE is essentially a one node cpuset.
GFP_DMA and GFP_DMA32 are physical address range constraints.
GFP_HARDWALL is a strict vs. nonstrict distinction.
7. Allocators must change
Right now the policy is set by the process context which is bad because
one cannot specify a memory policy for an allocation. It must be possible
to pass a memory policy to the allocators and then get the memory
requested.
I wish we could come up with some universal scheme that encompasses all
of the functionality we want and that makes memory more manageable....
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
