Re: init's children list is long and slows reaping children.

[Nick Piggin <nickpiggin@yahoo.com.au>]

Jeff Garzik wrote:
> Linus Torvalds wrote:
>
> > On Fri, 6 Apr 2007, Jeff Garzik wrote:
> >
> > > I would rather change the implementation under the hood to start per-CPU
> > > threads on demand, similar to a thread-pool implementation.
> > >
> > > Boxes with $BigNum CPUs probably won't ever use half of those threads.
> >
> >
> > The counter-argument is that boxes with $BigNum CPU's really don't 
> > hurt from it either, and having per-process data structures is often 
> > simpler and more efficient than trying to have some thread pool.
>
>
> Two points here:
>
> * A lot of the users in the current kernel tree don't rely on the 
> per-CPU qualities.  They just need multiple threads running.
>
> * Even with per-CPU data structures and code, you don't necessarily have 
> to keep a thread alive and running for each CPU.  Reap the ones that 
> haven't been used in $TimeFrame, and add thread creation to the slow 
> path that already exists in the bowels of schedule_work().
>
> Or if some kernel hacker is really motivated, all workqueue users in the 
> kernel would benefit from a "thread audit", looking at working 
> conditions to decide if the new kthread APIs are more appropriate.

spawn on demand would require heuristics and complexity though. And
I think there is barely any positive tradeoff to weigh it against.

> > IOW, once we get the processes off the global list, there just isn't 
> > any downside from them. Sure, they use some memory, but people who buy 
> > 1024-cpu machines won't care about a few kB per CPU..
> >
> > So the *only* downside is literally the process list, and one 
> > suggested patch already just removes kernel threads entirely from the 
> > parenthood lists.
> >
> > The other potential downside could be "ps is slow", but on the other 
> > hand, having the things stick around and have things like CPU-time 
> > accumulate is probably worth it - if there are some issues, they'd 
> > show up properly accounted for in a way that process pools would have 
> > a hard time doing.
>
>
> Regardless of how things are shuffled about internally, there will 
> always be annoying overhead /somewhere/ when you have a metric ton of 
> kernel threads.  I think that people should also be working on ways to 
> make the kernel threads a bit more manageable for the average human.

There are a few per CPU, but they should need no human management to
speak of.

Presumably if you have a 1024 CPU system, you'd generally want to be
running at least 1024 of your own processes there, so you already need
some tools to handle that magnitude of processes anyway.

> > So I really don't think this is worth changing things over, apart from 
> > literally removing them from process lists, which I think everybody 
> > agrees we should just do - it just never even came up before!
>
>
> I think there is a human downside.  For an admin you have to wade 
> through a ton of processes on your machine, if you are attempting to 
> evaluate the overall state of the machine.  Just google around for all 
> the admins complaining about the explosion of kernel threads on 
> production machines :)

User tools should be improved. It shouldn't be too hard to be able to
aggregate kernel thread stats into a single top entry, for example.

I'm not saying the number of threads couldn't be cut down, but there
is still be an order of magnitude problem there...

--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/