Re: init's children list is long and slows reaping children.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Linus Torvalds wrote:

On Fri, 6 Apr 2007, Jeff Garzik wrote:
I would rather change the implementation under the hood to start per-CPU
threads on demand, similar to a thread-pool implementation.

Boxes with $BigNum CPUs probably won't ever use half of those threads.

The counter-argument is that boxes with $BigNum CPU's really don't hurt from it either, and having per-process data structures is often simpler and more efficient than trying to have some thread pool.

Two points here:

* A lot of the users in the current kernel tree don't rely on the per-CPU qualities. They just need multiple threads running.

* Even with per-CPU data structures and code, you don't necessarily have to keep a thread alive and running for each CPU. Reap the ones that haven't been used in $TimeFrame, and add thread creation to the slow path that already exists in the bowels of schedule_work().

Or if some kernel hacker is really motivated, all workqueue users in the kernel would benefit from a "thread audit", looking at working conditions to decide if the new kthread APIs are more appropriate.


IOW, once we get the processes off the global list, there just isn't any downside from them. Sure, they use some memory, but people who buy 1024-cpu machines won't care about a few kB per CPU..

So the *only* downside is literally the process list, and one suggested patch already just removes kernel threads entirely from the parenthood lists.

The other potential downside could be "ps is slow", but on the other hand, having the things stick around and have things like CPU-time accumulate is probably worth it - if there are some issues, they'd show up properly accounted for in a way that process pools would have a hard time doing.

Regardless of how things are shuffled about internally, there will always be annoying overhead /somewhere/ when you have a metric ton of kernel threads. I think that people should also be working on ways to make the kernel threads a bit more manageable for the average human.


So I really don't think this is worth changing things over, apart from literally removing them from process lists, which I think everybody agrees we should just do - it just never even came up before!

I think there is a human downside. For an admin you have to wade through a ton of processes on your machine, if you are attempting to evaluate the overall state of the machine. Just google around for all the admins complaining about the explosion of kernel threads on production machines :)

	Jeff



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Stuff]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]     [Linux Resources]
  Powered by Linux