Re: [patch 2/2] fix file counting

[Eric Dumazet <dada1@cosmosbay.com>]

Dipankar Sarma a écrit :
> On Fri, Jan 27, 2006 at 03:28:57PM -0800, Andrew Morton wrote:
> > "Paul E. McKenney" <paulmck@us.ibm.com> wrote:
> > > > > I am using a patch that seems sligthly better : It removes the filp_count_lock 
> > > > > as yours but introduces a percpu variable, and a lazy nr_files . (Its value 
> > > > > can be off with a delta of +/- 16*num_possible_cpus()
> > > > Yes, I think that is better.
> > > I agree that Eric's approach likely improves performance on large systems
> > > due to decreased cache thrashing.  However, the real problem is getting
> > > both good throughput and good latency in RCU callback processing, given
> > > Lee Revell's latency testing results.  Once we get that in hand, then
> > > we should consider Eric's approach.
> > Dipankar's patch risks worsening large-SMP scalability, doesn't it? 
> > Putting an atomic op into the file_free path?
>
> Here are some numbers from a 32-way (HT) P4 xeon box with kernbench -

Hi Dipankar, thank you very much for doing these tests.

To have an idea of the number of files that are allocated/freed during a 
kernel build, I added 4 fields in struct files_stat.

4th field is the central atomic_t nr_files.
5th field is the counter of allocated files.
6th field is the counter of freed files.
7th field is the counter of changes done on central atomic_t.

(Test machine is a dual Xeon HT, 2 GHz)
# make clean
# cat /proc/sys/fs/file-nr
119     0       206071  119     39169   39022   1131
[root@localhost linux-2.6.16-rc1-mm4]# time make -j4 bzImage
798.49user 82.36system 4:56.56elapsed 297%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (62major+7223263minor)pagefaults 0swaps

# cat /proc/sys/fs/file-nr
153     0       206071  153     755767  755620  5119

So thats (755767-39169)/(4*60+56) = 2420 files opened per second.

Number of changes on central atomic_t was :
(5119-1131)/(4*60+56) = 13 per second.


13 atomic ops per second (instead of 2420*2 per second if I had one atomic_t 
as in your patch), this is certainly not something we can notice in a typical 
SMP machine... maybe on a big NUMA machine it could be different ?

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/