Re: irq load balancing

[Chris Snook <csnook@redhat.com>]

Venkat Subbiah wrote:
> Since most network devices have a single status register for both
> receiver and transmit (and errors and the like), which needs a lock to
> protect access, you will likely end up with serious thrashing of moving
> the lock between cpus.
> > Any ways to measure the trashing of locks?
>
> Since most network devices have a single status register for both
> receiver and transmit (and errors and the like)
> > These register accesses will be mostly within the irq handler which I
> plan on keeping on the same processor. The network driver is actually
> tg3. Will looks closely into the driver.

Why are you trying to do this, anyway?  This is a classic example of fairness 
hurting both performance and efficiency.  Unbalanced distribution of a single 
IRQ gives superior performance.  There are cases when this is a worthwhile 
tradeoff, but the network stack is not one of them.  In the HPC world, people 
generally want to squeeze maximum performance out of CPU/cache/RAM so they just 
accept the imbalance because it performs better than balancing it, and 
irqbalance can keep things fair over longer intervals if that's important.  In 
the realtime world, people generally bind everything they can to one or two 
CPUs, and bind their realtime applications to the remaining ones to minimize 
contention.

Distributing your network interrupts in a round-robin fashion will make your 
computer do exactly one thing faster: heat up the room.

	-- Chris
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/