Erik Mouw wrote:
On Tue, Apr 18, 2006 at 08:19:17PM +0200, Arjan van de Ven wrote:
but spreading IRQ's across all of the CPU's doesn't seem like it's
ever the right answer.
well it is in some cases, imagine having 2 cpus and 2 gige nics that are
very busy doing webserving. That's an obvious case where 1-nic-per-cpu
ends up doing the right thing... the way it ends up is that each nic has
a full cpu for itself and it's own apaches... almost fully independent
of the other one. Now if you moved both irqs to the same cpu, the
apaches would follow, because if they didn't then you'd be bouncing
their data *all the time*. And at that point the other cpu will become
bored ;)
So what happens with a dual amd64 system where each CPU has its "own"
NIC? Something like this:
MEM0 <--> CPU0 <--- HT ---> CPU1 <--> MEM1
^ ^
| |
HT HT
| |
v v
PCI bridge0 PCI1 bridge
^ ^
| |
PCI PCI
| |
v v
GigE NIC0 GigE NIC1
The "best" approach would be to run an Apache on each CPU using local
memory and NIC and having the IRQs handled by the local CPU. Does the
irq balancer allow such a configuraion, or would it be hamperd by the
process scheduler deciding to run both Apaches on a single CPU?
The trouble is that we're not smart enough to redirect receiving traffic
back to the correct CPU, I think. You'd need to configure different IP
addrs on the same subnet to each NIC, and have intelligent bonding for
outbound traffic.
Then you'd need NUMA locality of IRQs, by bonding them to their closest
CPUs, which is something that's easily done inside the kernel, but is
harder in userspace. but I'm not getting into that silly pissing contest
again. Either mechanism *could* do it. It's just about creating sensbile
APIs and defaults, both of which we're not good at doing as a community.
M.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
