On 9/9/05, Anthony Wesley <[email protected]> wrote:
> Okay, just tested a couple of things, here's what I see...
>
> I tested the write speed to the usb2 hard disk and got 21MBytes/sec. It's a
> laptop
> hard drive, only 5400rpm so this is not surprising.
>
> I did this test with my video capture app running, but just displaying data
> and not writing,
>
> I have another laptop usb2 hard drive here which I just tested and got
> 17MBytes/sec - it's
> a 4200rpm drive so again not surprising numbers.
>
> The video data is written as individual frames, so the efficiency is a bit
> below the raw throughput,
> but my tests were transferring 1.5Gb of data as raw frames - exactly the
> same way that Coriander would
> write its data.
>
> I'd bet a large sum of money that these hard disk figures are correct to
> within a few percent.
>
> Also, when I am actually capturing I have timed by stopwatch how long the
> disk activity light
> is on, and reached about the same conclusion.
>
> Working the problem from the other direction, the only way to explain the
> early throttling that
> I see would be if *almost no data* is being written to the disk, and this is
> plainly not the case. Even if
> the disk were running at a greatly reduced rate of (say) 10MBytes/sec I
> would still see 86 seconds
> of buffering before the throttle kicks in, and so far I have managed only to
> get to about 65 or 70.
>
> regards, Anthony
>
i need a vmstat trace ('vmstat 1 > logfile', then run the test, then
^c) before i can comment further. also, your filesystem and
scheduler would be interesting to know. you can find out the
scheduler with 'cat /sys/block/[drive]/queue/scheduler'.
NATE
> Nate Diller wrote:
>
>
> >>Setting dirty_ratio and dirty_background_ratio to 90/10 puts me back at
> >>around 50 seconds, i.e. where I started.
> >>
> >
> > this setting should do the trick, so there's something going on here
> > that isn't expected.
> >
> >
> >>So as far as I can see there is *no way* to get 3 minutes worth of
> buffering
> >>by adjusting these parameters.
> >>
> >>Just to remind everyone - I have video data coming in at 25MBytes/sec and
> I
> >>am writing it out to a usb2 hard disk that can sustain 17MBytes/sec. I
> want
> >>my video capture to run at full speed as long as possible by having the
> >>7MBytes/sec deficit slowly eat up the available RAM in the machine. I
> have
> >>1.5Gb of RAM, 1.3Gb available for buffers, so this should take 3 minutes
> to
> >>consume at 7MBytes/sec.
> >>
> >>So, I've tried all the combinations on dirty_ratio and
> >>dirty_background_ratio and they *do not help*.
> >>
> >
> > dirty_ratio is the tubable you want, if it's not working correctly,
> > either there's a problem with your setup, or a bug
> >
> >
> >>Can anyone suggest something else that I might try? The goal is to have
> >>25MBytes/sec streaming video for about 3 minutes.
> >>
> >
> > how sure are you that you're getting 17MB/s during this test? can you
> > run "vmstat 1" while this is running to verify? which FS and
> > scheduler?
> >
> > just for interest, what's the raw disk bandwidth (use hdparm, or run a
> > dd, or something)? it would obviously be much better to sustain
> > 25MB/s to disk
> >
> >
> >>Or is this simply not possible with the current kernel I/O setup? Do I
> have
> >>to do something elaborate myself, like build a big RAM buffer, mount the
> >>disk synchronous, do the buffering myself in userland...??
> >>
> >
> > this should be possible, although it could be considered a bit risky WRT
> OOM.
> >
> > NATE
>
> --
> Anthony Wesley
> Director and IT/Network Consultant
> Smart Networks Pty Ltd
> Acquerra Pty Ltd
>
> [email protected]
> Phone: (02) 62595404 or 0419409836
>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
| |
 |