Re: [RFC 0/4] Object reclaim via the slab allocator V1

[Christoph Lameter <clameter@sgi.com>]

On Sun, 2 Jul 2006, Marcelo Tosatti wrote:

> > > I was, of course, referring to the unpleasant requirement that the object
> > > layout start with an atomic_t.
> > 
> > Is that such a problem? It reduces the amount of indirect function calls 
> > needed.
> 
> Need to benchmark I guess.

I think is pretty obvious. With atomic refcounters you can simply scan
a slab for unused objects without any callbacks. Needing a callback for 
every single object is a waste of resources and will limit reclaim 
efficiency. You would have to do 120 callbacks on some slabs just to 
figure out that it is worth trying to free objects in that 
particular slab block.
 
> I agree with Andrew, an atomic counter to indicate usage of the objects
> is too simplistic (other than being unpleasant).

Cannot see a valid reason so far to draw that conclusion. With the right 
convention the atomic refcounter can be used as an indicator that the 
object is being freed (refcnt = 0), not in use (refcnt = 1) or in active 
use (refcnt=2). The easy and efficient access to this kind of knowledge 
about an object is essential for reclaim.

> > > How do you propose handling the locking?  dcache is passed a bare pointer
> > > and no locks are held, but it is guaranteed that the object won't be freed
> > > while it's playing with it?
> > 
> > The reference counter can be used to insure that the object is not freed.
> 
> Locking is pretty tricky...

Not at all. You do an atomic_inc_if_not_zero from the destructor and then 
either will hold the object to good or you were unable to increment the 
refcount and therefore you can give up and return because the object 
is already being freed.

The tricky locking part comes later when the destructor has to establish 
the locks to get all links to the object released.

> What it does is to create a small set callbacks to invoke higher-level
> code:
> 
> +struct slab_reclaim_ops {
> +	int (*objp_is_freeable)(void *objp);
> +	int (*objp_release)(void *objp);
> +	int (*objp_lock)(void *objp);
> +	int (*objp_unlock)(void *objp);
> +};
> 
> Which are used by generic SLAB code to check for, and release, fully
> freeable pages (you've seen it before, from last year). It contains a
> dcache implementation.

Ok. I will have a look at that. But these callbacks are too heavy for my 
taste. A refcounter could avoid all of that.

> You really need cache specific information, such as the tree nature of
> dentry references.

Only the cache knows that. The cache provides a destructo. In that 
destructor the tree nature of the dentries can be considered. The 
destructor knows about the tree nature of the slab and that can navigate 
through the dependency structures that the object may be a part of.

> Christoph, I'm afraid that not using the LRU information can be harmful
> to certain workloads... 

It can be very beneficial too since removing the LRU will avoid locking in 
certain situations and it will shrink the objects. We already have lists 
of all the objects in the slab. The LRU is redundant in some way.

Of course there is the challenge of preserving the LRU like behavior using 
the slab lists. But I think it may be sufficiently approximated by the 
LRU ness of the used slab list and the push back to the partial lists 
whenever we touch a slab during reclaim (we free some objects so the slab 
has to move).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/