On 9/9/05, Anthony Wesley <[email protected]> wrote:
> Hi Nate and Roger,
>
> First off, thanks fo taking some time to answer my call for help :-)
>
> I've already spent some time fiddling with dirty_ratio before I posted my
> question - yes, I can make different things happen by increasing or
> decreasing it, but overall I still have the same problem - my video
> streaming runs out of steam after at most 50 seconds instead of the 3
> minutes that I think it should take.
>
> Setting dirty_ratio and dirty_background_ratio to 15/10 respectively makes
> things worse, I hit the wall after only about 25 seconds.
>
not surprising
> Setting dirty_ratio and dirty_background_ratio to 85/80 makes things worse -
> disk writes start after about 15 seconds and I hit the wall after about 35
> seconds.
>
so dirty_background_ratio is working, that's why it took a long time
to start writing anything out
> Setting dirty_ratio and dirty_background_ratio to 90/10 puts me back at
> around 50 seconds, i.e. where I started.
>
this setting should do the trick, so there's something going on here
that isn't expected.
> So as far as I can see there is *no way* to get 3 minutes worth of buffering
> by adjusting these parameters.
>
> Just to remind everyone - I have video data coming in at 25MBytes/sec and I
> am writing it out to a usb2 hard disk that can sustain 17MBytes/sec. I want
> my video capture to run at full speed as long as possible by having the
> 7MBytes/sec deficit slowly eat up the available RAM in the machine. I have
> 1.5Gb of RAM, 1.3Gb available for buffers, so this should take 3 minutes to
> consume at 7MBytes/sec.
>
> So, I've tried all the combinations on dirty_ratio and
> dirty_background_ratio and they *do not help*.
>
dirty_ratio is the tubable you want, if it's not working correctly,
either there's a problem with your setup, or a bug
> Can anyone suggest something else that I might try? The goal is to have
> 25MBytes/sec streaming video for about 3 minutes.
>
how sure are you that you're getting 17MB/s during this test? can you
run "vmstat 1" while this is running to verify? which FS and
scheduler?
just for interest, what's the raw disk bandwidth (use hdparm, or run a
dd, or something)? it would obviously be much better to sustain
25MB/s to disk
> Or is this simply not possible with the current kernel I/O setup? Do I have
> to do something elaborate myself, like build a big RAM buffer, mount the
> disk synchronous, do the buffering myself in userland...??
>
this should be possible, although it could be considered a bit risky WRT OOM.
NATE
> regards, Anthony
>
> Nate Diller wrote:
>
> > On 9/9/05, Roger Heflin <[email protected]> wrote:
> >
> >>I saw it mentioned before that the kernel only allows a certain
> >>percentage of total memory to be dirty, I thought the number was
> >>around 40%, and I have seen machines with large amounts of ram,
> >>hit the 40% and then put the writing application into disk wait
> >>until certain amounts of things are written out, and then take
> >>it out of disk wait, and repeat when it again hits 40%, given your
> >>rate different it would be close to 40% in 50seconds.
> >>
> >
> > yes, on 2.6 there are two tunables which are important here.
> > dirty_background_ratio is the threshold where the kernel will begin
> > flushing dirty buffers, so it should change how soon the disk becomes
> > active. dirty_ratio changes when the write-throttling code kicks in,
> > which is what Anthony is seeing. The purpose of the write throttling
> > code is to limit the dirtying process to disk bandwidth, so that is a
> > Feature. Anthony, try *increasing* dirty_ratio, you can go up to 100,
> > but you could trigger an OOM if you let it get too high, so maybe try
> > setting it at 85 or so. This should effectively disable the write
> > throttling and give you the bandwidth you want.
> >
> > NATE
> >
> >
> >>And I think that you mean MB(yte) not Mb(it).
> >>
> >> Roger
> >>
> >>
> >>>-----Original Message-----
> >>>From: [email protected]
> >>>[mailto:[email protected]] On Behalf Of
> >>>Anthony Wesley
> >>>Sent: Friday, September 09, 2005 4:11 AM
> >>>To: [email protected]
> >>>Subject: Re: kernel 2.6.13 buffer strangeness
> >>>
> >>>Thanks David, but if you read my original post in full you'll
> >>>see that I've tried that, and while I can start the write out
> >>>sooner by lowering /proc/sys/vm/dirty_ratio , it makes no
> >>>difference to the results that I am getting. I still seem to
> >>>run out of steam after only 50 seconds where it should take
> >>>about 3 minutes.
> >>>
> >>>regards, Anthony
> >>>
> >>>--
> >>>Anthony Wesley
> >>>Director and IT/Network Consultant
> >>>Smart Networks Pty Ltd
> >>>Acquerra Pty Ltd
> >>>
> >>>[email protected]
> >>>Phone: (02) 62595404 or 0419409836
> >>>
> >>>-
> >>>To unsubscribe from this list: send the line "unsubscribe
> >>>linux-kernel" in the body of a message to
> >>>[email protected] More majordomo info at
> >>>http://vger.kernel.org/majordomo-info.html
> >>>Please read the FAQ at http://www.tux.org/lkml/
> >>>
> >>
> >>-
> >>To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> >>the body of a message to [email protected]
> >>More majordomo info at http://vger.kernel.org/majordomo-info.html
> >>Please read the FAQ at http://www.tux.org/lkml/
> >>
> >
> >
>
> --
> Anthony Wesley
> Director and IT/Network Consultant
> Smart Networks Pty Ltd
> Acquerra Pty Ltd
>
> [email protected]
> Phone: (02) 62595404 or 0419409836
>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
| |
 |