Re: memory resource accounting (was Re: [RFC, PATCH 0/5] Going forward with Resource Management - A cpu controller)

[Nick Piggin <nickpiggin@yahoo.com.au>]

Martin Bligh wrote:

> > > It also saves you from maintaining huge lists against each page.
> > >
> > > Worse case, you want to bill everyone who opens that address_space
> > > equally. But the semantics on exit still suck.
> > >
> > > What was Alan's quote again? "unfair, unreliable, inefficient ...
> > > pick at least one out of the three". or something like that.
> >
> >
> > What's the sucking semantics on exit? I haven't looked much at the
> > existing memory controllers going around, but the implementation I
> > imagine looks something like this (I think it is conceptually similar
> > to the basic beancounters idea):
>
>
> You have to increase the other processes allocations, putting them
> over their limits. If you then force them into reclaim, they're going
> to stall, and give bad latency.


Not within a particular container. If the process exits but leaves around
some memory charge, then that just remains within the same container.

If you want to remove a container, then you have a hierarchy of billing
and your charge just gets accounted to the parent.

> > - anyone who allocates a page for anything gets charged for that page.
> >   Except interrupt/softirq context. Could we ignore these for the 
> > moment?
> >
> >   This does give you kernel (slab, pagetable, etc) allocations as 
> > well as
> >   userspace. I don't like the idea of doing controllers for inode cache
> >   and controllers for dentry cache, etc, etc, ad infinitum.
> >
> > - each struct page has a backpointer to its billed container. At the mm
> >   summit Linus said he didn't want back pointers, but I clarified 
> > with him
> >   and he isn't against them if they are easily configured out when 
> > not   using memory controllers.
> >
> > - memory accounting containers are in a hierarchy. If you want to 
> > destroy a
> >   container but it still has billed memory outstanding, that gets 
> > charged
> >   back to the parent. The data structure itself obviously still needs to
> >   stay around, to keep the backpointers from going stale... but that 
> > could
> >   be as little as a word or two in size.
> >
> > The reason I like this way of accounting is that it can be done with 
> > a couple
> > of hooks into page_alloc.c and an ifdef in mm.h, and that is the 
> > extent of
> > the impact on core mm/ so I'd be against anything more intrusive 
> > unless this
> > really doesn't work.
>
> See "inefficent" above (sorry ;-)) What you've chosen is more correct,
> but much higher overhead. The point was that there's tradeoffs either
> way - the conclusion we came to last time was that to make it 100%
> correct, you'd be better off going with a model like Xen.


So if someone says they want it 100% correct, I can tell them to use
Xen and not put accounting into any place in the kernel that allocates
memory? Sweet OK.

If we're happy with doing userspace only memory, then a similar scheme
can be implemented on an object-accounting basis (eg. vmas). I think
there is something that already implements this.

> 1. You're adding a backpointer to struct page.


That's nowhere near the overhead of pte chain rmaps, though. I think it
is perfectly acceptible (assuming you *did* want to account kernel page
allocations) and probably will be difficult to notice on non-crazy-highmem
boxes. Which is just about everyone we care about now.

> 2. Each page is not accounted to one container, but shared across them,
> so the billing changes every time someone forks or exits. And not just
> for that container, but all of them. Think pte chain based rmap ...
> except worse.


In my proposed scheme, it is just the first who allocates. You hope that
statistically, that is good enough. Otherwise you could go into tracking
what process has a reference to which dentry... good luck getting that
past Al and Christoph.

> 3. When a container needs to "shrink" when somebody else exits, how do
> we do reclaim pages from a specific container?


Not the problem of accounting. Any other scheme will have a similar
problem.

However, having the container in the struct page *could* actually help
directed reclaim FWIW.

--

Send instant messages to your online friends http://au.messenger.yahoo.com 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/