On Wed, 2006-02-08 at 04:41 -0500, Steven Rostedt wrote: > On Tue, 7 Feb 2006, Sébastien Dugué wrote: > > > Hi, > > > > I've been experimenting with 2.6.15-rt16 on a dual 2.8GHz Xeon box > > with quite good results and decided to make a run on a NUMA dual node > > IBM x440 (8 1.4GHz Xeon, 28GB ram). > > > > However, the kernel crashes early when creating the slabs. Does the > > current preempt-rt patchset supports NUMA machines or has support > > been disabled until things settle down? > > Yeah, currently the -rt patch doesn't work well with NUMA. OK, I'll have to wait for that I guess. > > > > > Going on, I compiled a non NUMA RT kernel which booted just fine, > > but when examining the latency traces, I came upon strange jumps > > in the latencies such as: > > > > > > <...>-6459 2D.h1 42us : rcu_pending (update_process_times) > > <...>-6459 2D.h1 42us : scheduler_tick (update_process_times) > > <...>-6459 2D.h1 43us : sched_clock (scheduler_tick) > > <...>-6459 2D.h1 44us!: _raw_spin_lock (scheduler_tick) > > <...>-6459 2D.h2 28806us : _raw_spin_unlock (scheduler_tick) > > <...>-6459 2D.h1 28806us : rebalance_tick (scheduler_tick) > > <...>-6459 2D.h1 28807us : irq_exit (smp_apic_timer_interrupt) > > <...>-6459 2D..1 28808us < (608) > > <...>-6459 2D..1 28809us : smp_apic_timer_interrupt (c03e2a02 0 0) > > <...>-6459 2D.h1 28810us : handle_nextevent_update (smp_apic_timer_interrupt) > > <...>-6459 2D.h1 28810us : hrtimer_interrupt (handle_nextevent_update) > > Hmm, are the TSC of the CPUS in sync? If not, you will get bad > messurements of the latency tracer. That is currently why we can't use > tracing with x86_64 x2 CPUS. > > > > > There does not seem to be a precise code path leading to those jumps, > > it seems > > they can appear anywhere. Furthermore the jump seems to always be of ~ 27 ms > > > > I tried running on only 1 CPU, tried using the TSC instead of the cyclone > > timer but to no avail, the phenomenon is still there. > > OK, this scares me. You get these with only one CPU? Is it still > compiled for SMP? If not, see if the latencies go away if you turn off > SMP. Yes the kernel is still compiled for SMP. An UP compiled kernel did not boot. I will try to fix it and go for it again. > > > > > My test program only consists in a thread running at max RT priority doing > > a nanosleep(). > > > > What could be going on here? > > Good question. Could you send your .config The more I think about it, the more I tend to believe it's hardware related. It seems as if the CPU just hangs for ~27 ms before resuming processing. Anyway, here is my .config. Maybe I've got something misconfigured somewhere after all. Thanks for replying.
Attachment:
config-2.6.15-rt16-no-numa.gz
Description: GNU Zip compressed data
- Follow-Ups:
- Re: preempt-rt, NUMA and strange latency traces
- From: Lee Revell <[email protected]>
- Re: preempt-rt, NUMA and strange latency traces
- From: john stultz <[email protected]>
- Re: preempt-rt, NUMA and strange latency traces
- References:
- preempt-rt, NUMA and strange latency traces
- From: Sébastien Dugué <[email protected]>
- Re: preempt-rt, NUMA and strange latency traces
- From: Steven Rostedt <[email protected]>
- preempt-rt, NUMA and strange latency traces
- Prev by Date: Re: [patch 15/44] generic ext2_{set,clear,test,find_first_zero,find_next_zero}_bit()
- Next by Date: Re: preempt-rt, NUMA and strange latency traces
- Previous by thread: Re: preempt-rt, NUMA and strange latency traces
- Next by thread: Re: preempt-rt, NUMA and strange latency traces
- Index(es):
 |