On Tue, 12 Jun 2007, Hidetoshi Seto wrote:
>
> [arch/ia64/kernel/time.c]
> > #ifdef CONFIG_SMP
> > /* On IA64 in an SMP configuration ITCs are never accurately synchronized.
> > * Jitter compensation requires a cmpxchg which may limit
> > * the scalability of the syscalls for retrieving time.
> > * The ITC synchronization is usually successful to within a few
> > * ITC ticks but this is not a sure thing. If you need to improve
> > * timer performance in SMP situations then boot the kernel with the
> > * "nojitter" option. However, doing so may result in time fluctuating (maybe
> > * even going backward) if the ITC offsets between the individual CPUs
> > * are too large.
> > */
> > if (!nojitter) itc_interpolator.jitter = 1;
> > #endif
>
> ia64 uses jitter compensation to prevent time from going backward.
>
> This jitter compensation logic which keep track of cycle value
> recently returned is provided as generic code (and copied to
> arch/ia64/kernel/fsys.S).
> It seems that there is no user (setting jitter = 1) other than ia64.
Yes there are only two users of time interpolators. The generic
gettimeofday work by John Stultz will make time interpolators obsolete
soon. I already saw a patch to remove them.
> The cmpxchg is known to take long time in an SMP environment but
> it is easy way to guarantee atomic operation.
> I think this is acceptable while there are no better alternatives.
>
> OTOH, the do-while forces retry if cmpxchg fails (no exchanges).
> This means that if there are N threads trying to do cmpxchg at
> same time, only 1 can out from this loop and N-1 others will be
> trapped in the loop. This also means that a thread could loop
> N times in worst case.
Right.
> Obviously this is a scalability issue.
> To avoid this retry loop, I'd like to propose new logic that
> removes do-while here.
Booting with nojitter is the easiest way to solve this if one is willing
to accept slightly inaccurate clocks.
> The basic idea is "use winner's cycle instead of retrying."
> Assuming that there are N threads trying to do cmpxchg, it also
> be assumed that they are trying to update last_cycle by its own
> new value while all values are almost same.
> Therefore, it will work that treating threads as a group and
> deciding a group's return value by picking up one from the group.
>
> Fortunately, cmpxchg mechanism can help this logic. Only first
> one in group can exchange the last_cycle successfully, so this
> "winner" gets previous last_cycle as the return value of cmpxchg.
> The rests in group will fail to exchange since last_cycle is
> already updated by winner, so these "loser" gets current
> last_cycle on cmpxchg's return. This means that all thread in
> the group can know the winner's cycle.
>
> ret = cmpxchg(&last_cycle, last, new);
> if (ret == last)
> return new; /* you win! */
> else
> return ret; /* you lose. ret is winner's new */
Interesting solution. But there may be multiple updates of last happening.
Which of the winners is the real winner?
>
> I had a test running gettimeofday() processes at 1.5GHz*4way box.
> It shows followings:
>
> - x1 process:
> 0.15us / 1 gettimeofday() call
> 0.15us / 1 gettimeofday() call with patch
> - x2 process:
> 0.31us / 1 gettimeofday() call
> 0.24us / 1 gettimeofday() call with patch
> - x3 process:
> 1.59us / 1 gettimeofday() call
> 1.11us / 1 gettimeofday() call with patch
> - x4 process:
> 2.34us / 1 gettimeofday() call
> 1.29us / 1 gettimeofday() call with patch
>
> I know that this patch could not help quite huge system since
> such system like having 1024CPUs should have better clocksource
> instead of doing cmpxchg. Even though this patch will work good
> on middle-sized box (4~8way, possibly 16~64way?).
Our SGI machines have their own clock that is hardware replicated to all
nodes.
This would certainly a nice improvement for SMP machines. It certainly
works nicely for 2 way systems.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
