Re: kernel 2.6.13 buffer strangeness

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 

Hi Nate and Roger,

First off, thanks fo taking some time to answer my call for help :-)

I've already spent some time fiddling with dirty_ratio before I posted my question - yes, I can make different things happen by increasing or decreasing it, but overall I still have the same problem - my video streaming runs out of steam after at most 50 seconds instead of the 3 minutes that I think it should take.

Setting dirty_ratio and dirty_background_ratio to 15/10 respectively makes things worse, I hit the wall after only about 25 seconds.

Setting dirty_ratio and dirty_background_ratio to 85/80 makes things worse - disk writes start after about 15 seconds and I hit the wall after about 35 seconds.

Setting dirty_ratio and dirty_background_ratio to 90/10 puts me back at around 50 seconds, i.e. where I started.

So as far as I can see there is *no way* to get 3 minutes worth of buffering by adjusting these parameters.

Just to remind everyone - I have video data coming in at 25MBytes/sec and I am writing it out to a usb2 hard disk that can sustain 17MBytes/sec. I want my video capture to run at full speed as long as possible by having the 7MBytes/sec deficit slowly eat up the available RAM in the machine. I have 1.5Gb of RAM, 1.3Gb available for buffers, so this should take 3 minutes to consume at 7MBytes/sec.

So, I've tried all the combinations on dirty_ratio and dirty_background_ratio and they *do not help*.
Can anyone suggest something else that I might try? The goal is to have 25MBytes/sec streaming video for about 3 minutes. 
Or is this simply not possible with the current kernel I/O setup? Do I have to do something elaborate myself, like build a big RAM buffer, mount the disk synchronous, do the buffering myself in userland...??

regards, Anthony

Nate Diller wrote:


On 9/9/05, Roger Heflin <[email protected]> wrote:


I saw it mentioned before that the kernel only allows a certain
percentage of total memory to be dirty, I thought the number was
around 40%, and I have seen machines with large amounts of ram,
hit the 40% and then put the writing application into disk wait
until certain amounts of things are written out, and then take
it out of disk wait, and repeat when it again hits 40%, given your
rate different it would be close to 40% in 50seconds.
yes, on 2.6 there are two tunables which are important here. dirty_background_ratio is the threshold where the kernel will begin 
flushing dirty buffers, so it should change how soon the disk becomes
active.  dirty_ratio changes when the write-throttling code kicks in,
which is what Anthony is seeing.  The purpose of the write throttling
code is to limit the dirtying process to disk bandwidth, so that is a
Feature.  Anthony, try *increasing* dirty_ratio, you can go up to 100,
but you could trigger an OOM if you let it get too high, so maybe try
setting it at 85 or so.  This should effectively disable the write
throttling and give you the bandwidth you want.

NATE



And I think that you mean MB(yte) not Mb(it).

                          Roger



-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Anthony Wesley 
Sent: Friday, September 09, 2005 4:11 AM
To: [email protected]
Subject: Re: kernel 2.6.13 buffer strangeness
Thanks David, but if you read my original post in full you'll see that I've tried that, and while I can start the write out sooner by lowering /proc/sys/vm/dirty_ratio , it makes no difference to the results that I am getting. I still seem to run out of steam after only 50 seconds where it should take about 3 minutes. 

regards, Anthony

--
Anthony Wesley
Director and IT/Network Consultant
Smart Networks Pty Ltd
Acquerra Pty Ltd

[email protected]
Phone: (02) 62595404 or 0419409836

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html 
Please read the FAQ at  http://www.tux.org/lkml/



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/






--
Anthony Wesley
Director and IT/Network Consultant
Smart Networks Pty Ltd
Acquerra Pty Ltd

[email protected]
Phone: (02) 62595404 or 0419409836
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]
  Powered by Linux