Re: [RFC 0/4] Object reclaim via the slab allocator V1

[Christoph Lameter <clameter@sgi.com>]

On Mon, 3 Jul 2006, Marcelo Tosatti wrote:

> > I think is pretty obvious. With atomic refcounters you can simply scan
> > a slab for unused objects without any callbacks. Needing a callback for 
> > every single object is a waste of resources and will limit reclaim 
> > efficiency. You would have to do 120 callbacks on some slabs just to 
> > figure out that it is worth trying to free objects in that 
> > particular slab block.
> 
> Inline the callbacks into a per-cache kmem_cache_reclaim ?

You mean the user writes the check functions? Can you give an example how 
inlining is supposed to work here?

> > Cannot see a valid reason so far to draw that conclusion. With the right 
> > convention the atomic refcounter can be used as an indicator that the 
> > object is being freed (refcnt = 0), not in use (refcnt = 1) or in active 
> > use (refcnt=2). The easy and efficient access to this kind of knowledge 
> > about an object is essential for reclaim.
> 
> But the assumption that "refcnt = 1 implies unused object" is too weak.
> 
> For example, 
> 
> struct dentry {
>         atomic_t d_count;
>         unsigned int d_flags;           /* protected by d_lock */
> 
> d_count can be higher than one _and_ the object freeable. Think of
> workloads operating on a large number of directories.

The scheme that I proposed implies that the refcount handling is changed.
It must be uniform for all object types that use reclaim.

If used for the dcache then dentry handling must be changed so that the 
refcount at the beginnDing of the slab is 1 if the object is reclaimable 
and the higher refcount needs to be an indicator that the object is in 
use. I am not saying that existing use gets us there. Maybe we need to 
call this a reclaim flag instead of a refcount?

> Andrew mentioned:
> 
> "That seems like quite a drawback. A single refcount=2 object on the
> page means that nothing gets freed from that page at all. It'd be easy
> (especially with dcache) to do tons of work without achieving anything."

We can check for a single high count object in a slab and then call
the destructor if we feel that is warranted. The refcount is an 
indicator to the slab of the reclaim status of the object.

> > Ok. I will have a look at that. But these callbacks are too heavy for my 
> > taste. A refcounter could avoid all of that.
> 
> Inline them.

"Inline" seem to be way to say that the user has to provide the function.

> > Of course there is the challenge of preserving the LRU like behavior using 
> > the slab lists. But I think it may be sufficiently approximated by the 
> > LRU ness of the used slab list and the push back to the partial lists 
> > whenever we touch a slab during reclaim (we free some objects so the slab 
> > has to move).
> 
> Well, individual object usage is not reflected at all in the slab lists,
> is it?

Correct. We would have to treat objects in a slab all the same. We could 
just kick out some if we find the slab at the end of the list and see how 
things develop. Pretty bare hacky LRU but it may be better than going 
through huge lists of small objects on a LRU lists.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/