On Thu, 3 May 2007, David Schwartz wrote:
I needed to recompress some files from .bz2 to .gz so I setup a script to
do
bunzip2 -c $file.bz2 |gzip -9 >$file.gz
I expected that the two CPU heavy processes would end up on different
cpu's and spend a little time shuffling data between the two cpu's on a
system (dual core opteron)
however, instead what I find is that each process is getting 50% of one
cpu while the other cpu is 97% idle.
That would only be possible if the compression/decompression block size is
small compared to the maximum pipe buffer size. I suspect the reverse is the
case.
I'm still running into this problem in various forms
is there an easy way to change the maximum pipe buffer size? (including a
simple change to the kernel source, I do compile my own kernels)
It would be interesting to write an intermediate process that basically
enlarged the pipe buffers and see if that changed anything. Basically, the
intermediate process would allocate a large buffer (16MB or so) and fill it
from 'bunzip2' while draining it to 'gzip' in a non-blocking way (unless the
buffer was full/empty, of course).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
