Re: high system cpu load during intense disk i/o

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Rafał Bilski wrote:
Hello and thanks for your reply.
Hello again,
The cron job that is running every 10 min on my system is mpop (a fetchmail-like program) and another running every 5 min is mrtg. Both normally finish within 1-2 seconds. The fact that these simple cron jobs don't finish ever is certainly because of the high system CPU load. If you see the two_discs_bad.txt which I attached on my original message, you'll see that *vmlinux*, and specifically the *scheduler*, take up most time. And the fact that this happens only when running two i/o processes but when running only one everything is absolutely snappy (not at all slow, see one_disc.txt), makes me sure that this is a kernel bug. I'd be happy to help but I need some guidance to pinpoint the problem.
In Your oprofile output I find "acpi_pm_read" particulary interesting. Unlike other VIA chipsets, which I know, Your doesn't use VLink to connect northbridge to southbridge. Instead PCI bus connects these two. As You probably know maximal PCI throughtput is 133MiB/s. In theory. In practice probably less. ACPI registers are located on southbridge. This probably means that processor needs access to PCI bus in order to read ACPI timer register. Now some math. 20GiB disk probably can send data at 20MiB/s rate. 200GiB disk probably about 40MiB/s. So 20+2*40=100MiB/s. I think that this could explain why simple inl() call takes so much time and why Your system isn't very responsive.
Thanks, Dimitris
Let me know if You find my theory amazing or amusing.

Hello Rafal,

I find your theory very nice, but unfortunately I don't think it applies here. As you can see from the vmstat outputs the write throughput is about 15MB/s for both disks. When reading I get about 30MB/s again from both disks. The other disk, the small one, is mostly idle, except for writing little bits and bytes now and then. Since the problem occurs when writing, 15MB/s is just too little I think for the PCI bus.

However I find it quite possible to have reached the throughput limit because of software (driver) problems. I have done various testing (mostly "hdparm -tT" with exactly the same PC and disks since about kernel 2.6.8 (maybe even earlier). I remember with certainty that read throughput the early days was about 50MB/s for each of the big disks, and combined with RAID 0 I got ~75MB/s. Those figures have been dropping gradually with each new kernel release and the situation today, with 2.6.22, is that hdparm gives maximum throughput 20MB/s for each disk, and for RAID 0 too!

I have been ignoring these performance regressions because of no stability problems until now. So could it be that I'm reaching the 20MB/s driver limit and some requests take too long to be served?


Dimitris
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Stuff]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]     [Linux Resources]
  Powered by Linux