* Christoph Lameter <[email protected]> wrote:
> I have done some testing on NUMA with 8p / 4n and 256 p / 128n which
> seems to indicate that this is doing very well. Having a global
> tasklet avoids concurrent load balancing from multiple nodes which
> smoothes out the load balancing load over all nodes in a NUMA system.
> It fixes the issues that we saw with contention during concurrent load
> balancing.
ok, that's what i suspected - what made the difference wasnt the fact
that it was moved out of irqs-off section, but that it was running
globally, instead of in parallel on every cpu. I have no conceptual
problem with single-threading the more invasive load-balancing bits.
(since it has to touch every runqueue anyway there's probably little
parallelism possible) But it's a scary change nevertheless, it
materially affects every SMP system's balancing characteristics.
Also, tasklets are a pretty bad choice for scalable stuff - it's
basically a shared resource between all CPUs and the tasklet-running
cacheline will bounce between all the CPUs. Also, a tasklet can be
'reactivated', which means that a CPU will do more rebalancing albeit
it's /another/ CPU that wanted that.
So could you try another implementation perhaps, which would match the
'single thread of rebalancing' model better? For example, try a global
spinlock that is trylocked upon rebalance. If the trylock succeeds then
do the rebalance and unlock. If the trylock fails, just skip the
rebalance. (this roughly matches the tasklet logic but has lower
overhead and doesnt cause false, repeat rebalancing)
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
