Re: "find" too slow on FC1's reiserfs file system

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I highly recommend xfs over both ext3 and reiserfs.

ext3 happened to cache everything in memory. reiserfs isn't so good about that.

the real problem with ext2 and 3 is that any time the directory inode
is modified i.e.: touch a new file or mkdir a new directory, the
entire thing needs to get written out to disk.  On a single ide disk,
we tested 100,000 entries in a single directory, and at that size, we
could only create like 10 new files or directories per second.  Pretty
pathetic.  ext2 and 3 also suffer from a maximum of 60,000
sub-directories (yes, I've hit it before).

>From the numbers you give, 3.4M/2500dirs averages 1361 entries per
directory.  That assumes an even distribution.

Additionally, you may want to consider moving to larger files. 
filesystems are not terribly efficient with lots of small files.  Your
overhead of managing the filesystem would be greater than the actual
reading of the data.  You may also want to consider moving the data
into a database. 3.4M entries in a single table is not that big. 
It'll be faster and more space efficient.  At one of my pervious jobs,
we did just that, a direct filesystem replacement with a database.  It
was extremely simple.  It had one table with two columns. filename and
data.  It was a quick 1-to-1 mapping implemented in about a week.

Also, turning off atime on the filesystem will  vastly improve your
performance.  It will remove the need to write to the filesystem just
because you did a 'find' or 'cat' a file.

   - Kevin

On Mon, 09 Aug 2004 21:44:28 -0400, jludwig <wralphie@xxxxxxxxxxx> wrote:
> 
> 
> On Sat, 2004-08-07 at 16:59, Guolin Cheng wrote:
> > Hi,
> >
> >  "find " runs far too slow on my reiserfs file system, on the file
> > system I have only 3.4 Million small files under 2500 directories. An
> > ext3 file system with the same contents tooks about 5 minutes for the
> > command "find /demoFileSystem âtype f " to finish, while the same
> > command took forever on a reiserfs file system.
> >
> >  It is against our general ideas of reiserfs file system.. I don't
> > know what's going on.
> >
> >  The reiserfs file system is created with default parameters.
> > "mkreiserfs âlabel demoFS /dev/hdd1 ",  the entry in fstab is:
> >
> > "/dev/hdc1               /demoFS                reiserfs
> > defaults,nosuid 1 2", everything seems normal. When the machines
> > reboots, it mounts reiserfs with no problems.
> >
> >  My base Linux system is FC1, my kernel is a vanilla 2.4.26 kernel,
> > with reiserfs built as a module.
> >
> >  Any ideas?
> >
> >  Thanks a lot.
> >
> >  --Guolin Cheng
> >
> >
> > 
> > ______________________________________________________________________
> > --
> > fedora-list mailing list
> > fedora-list@xxxxxxxxxx
> > To unsubscribe: http://www.redhat.com/mailman/listinfo/fedora-list
> Try slocate (if this finds nothing run updatedb as SU).
> (Slocate uses a database to locate files insted of actually scanning the
> directories (hard drive).
> --
> jludwig <wralphie@xxxxxxxxxxx>
> 
> --
> fedora-list mailing list
> fedora-list@xxxxxxxxxx
> To unsubscribe: http://www.redhat.com/mailman/listinfo/fedora-list
>



[Index of Archives]     [Current Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [Yosemite Photos]     [KDE Users]     [Fedora Tools]     [Fedora Docs]

  Powered by Linux