Ingo Molnar wrote:
But it was not me who claimed that 'workqueues are slow'.
The claim was: slower than tasklets.
choice. I am just wondering out loud whether this particular tool, in
its current usage pattern, makes much technological sense. My claim is:
it could very well be that it doesnt make _much_ sense, and in that case
we should provide a non-intrusive migration path away in terms of a
compatible API wrapper to a saner (albeit by virtue of trying to emulate
an existing API, slower) mechanism. The examples cited so far had the
tasklet as an intermediary towards a softirq - what's the technological
point in such a splitup?
I already answered that in detail. In sum, a driver cannot define its
own softirq. Softirqs are not modular.
Tasklets are the closest thing to softirqs for a driver.
The most scalable workloads dont involve any (or many) softirq middlemen
at all: you queue work straight from the hardirq context to the target
process context. And that's what you want to do _anyway_, because you
want to create as little locally cached data for the hardirq context, as
the target task could easily be on another CPU. (this is generally true
for things like block IO, but it's also true for things like network
IO.)
the most scalable solution would be _for the network adapter to figure
out the target CPU for the packet_.
I agree completely. Wanna implement this? I will kiss your feet, and
multi-core CPU vendors will worship you as a demi-god.
Until such time, we must deal with the network stack as it exists today,
and the network drivers as they exist and work today.
Not many (if any) such adapters
exist at the moment. (as it would involve allocating NR_CPUs irqs to
that adapter alone.)
Good news: this is becoming the norm for modern NICs, especially 10Gbps.
Plenty of NICs already exist that support multiple RX rings (persumably
one per CPU), and newer NICs will raise individual MSI[-X] interrupts
based on the RX ring into which a packet was received.
In this area, NIC vendors are way ahead of the Linux net stack.
The Linux net stack is unfortunately not threaded enough to sanely deal
with dividing /flows/ up across multiple CPUs, even if the NIC does
support multiple transmit and receive queues. [side note: initial
multi-queue TX is being worked on, on netdev]
Tasklet is single thread by definition and purpose. Those a few places
where people used tasklets to do per-cpu jobs (RCU f.e.) exist just
because they had troubles with allocating new softirq. [...]
no. The following tale is the true and only history of the RCU tasklet
;-) The RCU guys first used a tasklet, then noticed its bad scalability
(a particular VFS-intense benchmark regressed because only a single CPU
would do RCU completion on an 8-way box) so they switched it to a
per-cpu tasklet - without realizing that a per-cpu tasklet is in essence
a softirq. I pointed it out to them (years down the road ...) then the
"convert rcu-tasklet to softirq" patch was born.
You focused on the example rather than the key phrase: tasklet is
single thread by definition and purpose.
Wanting to change that without analysis of the impact illustrates the
apples-to-oranges change being proposed.
outlined above: if you want good scalability, dont use middlemen :-)
Figure out the target task as early as possible and let it do as much of
the remaining work as possible. _Increasing_ the amount of cached
context (by doing delayed processing in tasklets or even softirqs on the
same CPU where the hardirq arrived) only increases the cross-CPU cost.
Keeping stuff in a softirq only makes (some) sense as long as you have
no target task at all (routing, filtering, etc.).
I do not disagree with these theoretical musings :)
I care the most about the "who will do all this work?" question. In
network driver land, these changes impact hot paths. I am lazy, and
don't care to revisit each network driver hot path and carefully re-tune
each based on this proposal. Who is volunteering?
Jeff
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
