Alan Cox wrote:
this is exactly my problem I believe. I have a 4TB RAID SAN and the
problem directory holds well over a million small files. Even a
directory under that directory, du takes almost 40 minutes to total up 74GB.
The directory code in ext2 and ext3 isn't designed for a million files in
a single directory. Its a bizzare corner case. If you split the files
into subdirectories (eg by a hash) you'll get far saner numbers
Unfortunately, in this case, the directory structure is what it is. I
can't change it without a massive amount of pain.
However, on further review, my '6 hour' time has dropped significantly
over the course of the last month or so. I believe this is due to a
re-organizing when data is accessed on that array. Not to mention I did
have a failed drive in there. I don't think that had any bearing on it
since I had a hot spare that took up the load when that drive went bad.
Granted the rebuild time was long, but I know the stats I've pulled
recently have been better. Even now though, it still takes 37 minutes
to du a directory of 74GB. Maybe I"m just missing something.
--
Ita erat quando hic adveni.
Mark Haney
Sr. Systems Administrator
ERC Broadband
(828) 350-2415
