sched: incorrect argument used in task_hot()

["Chen, Kenneth W" <kenneth.w.chen@intel.com>]

The argument used for task_hot in can_migrate_task() looks wrong:

int can_migrate_task()
{ ...
       if (task_hot(p, rq->timestamp_last_tick, sd))
                return 0;
        return 1;
}

It is not using current time to estimate whether a task is cache-hot
or not on a remote CPU, instead it is using timestamp that the remote
cpu took last timer interrupt.  With timer interrupt staggered, this
under estimate how long a task has been off CPU by a wide margin
compare to its actual value.  The ramification is that tasks should
be considered as cache cold is now being evaluated as cache hot.

We've seen that the effect is especially annoying at HT sched domain
where l-b is not able to freely migrate tasks between sibling CPUs
and leave idle time on the system.

One route to defend that misbehave is to override sd->cache_hot_time
at boot time.  Intuitively, sys admin will set that parameter to zero.
But wait, that is not correct.  It should be set to -1 jiffy because
(rq->timestamp_last_tick - p->last_run) can be negative.  On top of
that one has to convert jiffy to ns when set the parameter. All very
unintuitive and undesirable.

I was very tempted to do:
     now - this_rq->timestamp_last_tick + rq->timestamp_last_tick;

But it is equally flawed that timestamp_last_tick is not synchronized
between this_rq and target_rq. The adjustment is simply inaccurate and
not suitable for load balance decision at HT (or even core) domain.

There are a number of other usages of above adjustment, I think they
are all inaccurate.  Though, most of them are for interactiveness and
can withstand the inaccuracy because it makes decision based on much
larger scale.

So back to the first observation on not enough l-b at HT domain because
of inaccurate time calculation, what would be the best solution to fix
this?

- Ken
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/