Re: O_DIRECT question

[Michael Tokarev <mjt@tls.msk.ru>]

Michael Tokarev wrote:
> Michael Tokarev wrote:
> By the way.  I just ran - for fun - a read test of a raid array.
> 
> Reading blocks of size 512kbytes, starting at random places on a 400Gb
> array, doing 64threads.
> 
>  O_DIRECT: 336.73 MB/sec.
> !O_DIRECT: 146.00 MB/sec.

And when turning off read-ahead, the speed dropped to 30 MB/sec.  Read-ahead
should not help here, I think... But after analyzing the "randomness" a bit,
it turned out alot of requests are coming to places "near" the ones which has
been read recently.  After switching to another random number generator,
speed in a case WITH readahead enabled dropped to almost 5Mb/sec ;)

And sure thing, withOUT O_DIRECT, the whole system is almost dead under this
load - because everything is thrown away from the cache, even caches of /bin
/usr/bin etc... ;)  (For that, fadvise() seems to help a bit, but not alot).

(No, really - this load isn't entirely synthetic.  It's a typical database
workload - random I/O all over, on a large file.  If it can, it combines
several I/Os into one, by requesting more than a single block at a time,
but overall it is random.)

/mjt

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/