Re: [PATCH 01/05] NUMA: Generic code

[Werner Almesberger <werner@almesberger.net>]

Magnus Damm wrote:
> Sorry, but which one did not work very well? CKRM memory controller or
> NUMA emulation + CPUSETS?

We tried to partition our memory using the NUMA emulation, such that
timing-critical processes would allocate from one node, while all
the rest of the system would allocate from the other node.

The idea was that the timing-critical processes, with a fairly
"calm" allocation behaviour (read file data into the page cache,
then evict it again), would never or almost never trigger memory
reclaim this way, and thus have better worst-case latency.

Unfortunately, our benchmarks didn't show any improvements in
latency. In fact, the results were slightly worse, perhaps because
of processes on the "regular" node holding shared resources while
in memory reclaim.

I'm not entirely sure why this didn't work better. At least in
theory, it should have.

We did this in the ABISS project, about one year ago in response
to quite nasty reclaim latency suddenly appearing in an earlier
2.6 kernel. When we asked various MM developers, but none of them
was aware of any change that would make reclaims all of a sudden
very intrusive, and they attributed it to the "butterfly effect".
After a while (i.e., in later kernels), the butterflies must have
chosen a different victim, and the latency got better on its own.

So, in the end, we didn't need that NUMA hack to control reclaims.
But if they should rear their ugly heads again, it may be worth
having a second look.

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina     werner@almesberger.net /
/_http://www.almesberger.net/____________________________________________/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/