Re: [patch 2/2] cpusets: add interleave_over_allowed option

[Lee Schermerhorn <Lee.Schermerhorn@hp.com>]

On Fri, 2007-10-26 at 14:39 -0700, David Rientjes wrote:
> On Fri, 26 Oct 2007, Lee Schermerhorn wrote:
> 
> > So, you pass the subset, you don't set the flag to indicate you want
> > interleaving over all available.  You must be thinking of some other use
> > for saving the subset mask that I'm not seeing here.  Maybe restoring to
> > the exact nodes requested if they're taken away and then re-added to the
> > cpuset?
> > 
> 
> Paul's motivation for saving the passed nodemask to set_mempolicy() is so 
> that the _intent_ of the application is never lost.  That's the biggest 
> advantage that this method has and that I totally agree with.  So whenever 
> the mems_allowed of a cpuset changes, the MPOL_INTERLEAVE nodemask of all 
> attached tasks becomes their intent (pol->passed_nodemask) AND'd with the 
> new mems_allowed.  That can be done on mpol_rebind_policy() and shouldn't 
> be an extensive change.
> 
> So MPOL_INTERLEAVE, and possibly other, mempolicies will always try to 
> accomodate the intent of the application but only as far as the task's 
> cpuset restriction allows them.
> 
> 		David

Maybe it's just me, but I think it's pretty presumptuous to think we can
infer the intent of the application from the nodemask w/o additional
flags such as Christoph proposed [cpuset relative]--especially for
subsets of the cpuset.  E.g., the application could intend the nodemask
to specify memories within a certain distance of a physical resource,
such as where a particular IO adapter or set thereof attach to the
platform.  

And even when the intent is to preserve the cpuset relative positions of
the nodes in the nodemask, this really only makes sense if the original
and modified cpusets have the same physical topology w/rt multi-level
NUMA interconnects.  This is something that has bothered me about
dynamic cpusets and current policy remapping.  We don't do a good job of
explaining the implications of changing cpuset topology on applications,
nor do we handle it very well in the code.  Paul addresses one of my
concerns in a later message in this thread, so I'll comment there.

Later,
Lee

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/