Re: [RFC][PATCH 1/6] Storing ipcs into IDRs

[Nadia Derbey <Nadia.Derbey@bull.net>]

Nadia Derbey wrote:
> Andi Kleen wrote:
>
> > Nadia.Derbey@bull.net writes:
> >
> >
> > > This patch introduces ipcs storage into IDRs. The main changes are:
> > >  . This ipc_ids structure is changed: the entries array is changed 
> > > into a
> > >    root idr structure.
> > >  . The grow_ary() routine is removed: it is not needed anymore when 
> > > adding
> > >    an ipc structure, since we are now using the IDR facility.
> > >  . The ipc_rmid() routine interface is changed:
> > >       . there is no need for this routine to return the pointer 
> > > passed in as
> > >         argument: it is now declared as a void
> > >       . since the id is now part of the kern_ipc_perm structure, no 
> > > need to
> > >         have it as an argument to the routine
> >
> >
> > Thanks for doing this work. It was long overdue.
> >
> > Do you have any data how this changes memory consumption with many 
> > objects?
> > -Andi
>
> Andi,
>
> Here are the results I got when creating 32768 (IPCMNI) msg queues with 
> the patched kernel:
>
> http://akt.sourceforge.net/results/2.6.23-rc2-idr/msg11/output.new
>
> It's the output from msg11.c. This script does what follows:
>     . gets sysinfo(2) results
>     . captures /proc/meminfo
>     . captures /proc/slabinfo
>     . creates XX msg queues (XX given as parameter)
>     . captures /proc/meminfo
>     . captures /proc/slabinfo
>     . gets sysinfo results
>     . outputs all the results
>     . removes the created ipcs
>
> sysinfo results: a BEFORE and an AFTER column are output where necessary.
> BEFORE means "before creating the objects"
> AFTER means "after the objects have been created"
>
> meminfo results: the BEFORE and AFTER files are pasted
>
> slabinfo results: only the differences between the BEFORE and the AFTER 
> are output.
>
>
> Here are also the sizes for the ref and the patched kernel:
>
> size linux-2.6.23-rc2.ref/vmlinux linux-2.6.23-rc2/vmlinux
>    text    data     bss     dec     hex filename
> 4432697  496450  602112 5531259  54667b linux-2.6.23-rc2.ref/vmlinux
> 4430747  496450  602112 5529309  545edd linux-2.6.23-rc2/vmlinux
>
>
>
> The http://akt.sourceforge.net/results/2.6.23-rc2-idr/msg11 directory is 
> structured as follows:
>
> msg11.c: the script I used to generate the results
> output.ref: the output from msg11 with the ref kernel
> output.new: the output from msg11 with the patched kernel
> size: the output from the size command
> ref: directory with the results files for the ref kernel
> new: directory with the results files for the patched kernel
> In these 2 directories:
> *_mem_*before: /proc/meminfo before creating the msg queues
> *_mem_*after: /proc/meminfo after creating the msg queues
> *_slab_*before: /proc/slabinfo before creating the msg queues
> *_slab_*after: /proc/slabinfo after creating the msg queues

Andi,

Here is an annalysis of the results I sent you yesterday (I guess you 
don't have enoguh time to look at everything):

ref code:

1) since /proc/sys/kernel/msgmni has been set to 32768, vmalloc(0x20014) 
is called to allocate the entries[] array (see grow_ary() --> 
ipc_rcu_alloc()).

==>

Before msg queues allocation:  VmallocUsed = 2860 kB
After msg queues allocation:   VmallocUsed = 2996 KB

Once allocated this array is never freed.

Unfortunately, in the result I sent you yesterday, you can't see the 
evolution since it was not the 1st time I was running the test, so the 
vmalloc() has not been called.


2) Since 32768 msg queues have been created and a msg_queue structure <
PAGE_SIZE, kmalloc(0x6C) is called 32768 times

==> size-128 in slabinfo: (I slighty simplified the output to make it 
fit in the mail):

                              objs          slabs
         # name   <active>  <num> <size> : <active> <num>
before : size-128     1070   1320    128 :       44    44
after  : size-128    33840  33840    128 :     1128  1128


patched code:
1) since 32768 msg queues are created, idr_pre_get() is called 32768 times
==> idr_layer_cache in slabinfo:

                                        objs          slabs
         # name           <active>  <num> <size> : <active> <num>
before : idr_layer_cache       112    116    136 :        4     4
after  : idr_layer_cache      1189   1189    136 :       41    41

2) This point remains unchanged compared to the ref code.



Regards,
Nadia
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/