Okay, just tested a couple of things, here's what I see...
I tested the write speed to the usb2 hard disk and got 21MBytes/sec. It's a laptop
hard drive, only 5400rpm so this is not surprising.
I did this test with my video capture app running, but just displaying data and not writing,
I have another laptop usb2 hard drive here which I just tested and got 17MBytes/sec - it's
a 4200rpm drive so again not surprising numbers.
The video data is written as individual frames, so the efficiency is a bit below the raw throughput,
but my tests were transferring 1.5Gb of data as raw frames - exactly the same way that Coriander would
write its data.
I'd bet a large sum of money that these hard disk figures are correct to within a few percent.
Also, when I am actually capturing I have timed by stopwatch how long the disk activity light
is on, and reached about the same conclusion.
Working the problem from the other direction, the only way to explain the early throttling that
I see would be if *almost no data* is being written to the disk, and this is plainly not the case. Even if
the disk were running at a greatly reduced rate of (say) 10MBytes/sec I would still see 86 seconds
of buffering before the throttle kicks in, and so far I have managed only to get to about 65 or 70.
regards, Anthony
Nate Diller wrote:
Setting dirty_ratio and dirty_background_ratio to 90/10 puts me back at
around 50 seconds, i.e. where I started.
this setting should do the trick, so there's something going on here
that isn't expected.
So as far as I can see there is *no way* to get 3 minutes worth of buffering
by adjusting these parameters.
Just to remind everyone - I have video data coming in at 25MBytes/sec and I
am writing it out to a usb2 hard disk that can sustain 17MBytes/sec. I want
my video capture to run at full speed as long as possible by having the
7MBytes/sec deficit slowly eat up the available RAM in the machine. I have
1.5Gb of RAM, 1.3Gb available for buffers, so this should take 3 minutes to
consume at 7MBytes/sec.
So, I've tried all the combinations on dirty_ratio and
dirty_background_ratio and they *do not help*.
dirty_ratio is the tubable you want, if it's not working correctly,
either there's a problem with your setup, or a bug
Can anyone suggest something else that I might try? The goal is to have
25MBytes/sec streaming video for about 3 minutes.
how sure are you that you're getting 17MB/s during this test? can you
run "vmstat 1" while this is running to verify? which FS and
scheduler?
just for interest, what's the raw disk bandwidth (use hdparm, or run a
dd, or something)? it would obviously be much better to sustain
25MB/s to disk
Or is this simply not possible with the current kernel I/O setup? Do I have
to do something elaborate myself, like build a big RAM buffer, mount the
disk synchronous, do the buffering myself in userland...??
this should be possible, although it could be considered a bit risky WRT OOM.
NATE
--
Anthony Wesley
Director and IT/Network Consultant
Smart Networks Pty Ltd
Acquerra Pty Ltd
[email protected]
Phone: (02) 62595404 or 0419409836
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
| |
 |