On Thu, 2006-08-03 at 08:48 -0700, Robert Crocombe wrote:
> I am sad.
Cheer up... It's OK ;)
>
> Okay, I couldn't use that same machine, which had to be swapped with
> something else, but I have a very similar machine (32GB vs 8GB of
> RAM, 1 additional video card and some extra 1394b cards) which hit the
> same problem. The trace from it is:
Are you also getting any warnings or BUG reports before this. In your
other dmesg, it should a bug being reported. This could cause problems
later on.
That said, this is starting to look like a NUMA problem. So I would like
to know where that lock is being taken exactly.
>
> md0_raid1/1118[CPU#3]: BUG in debug_rt_mutex_unlock at
> kernel/rtmutex-debug.c:471
>
> Call Trace:
> <ffffffff80487453>{_raw_spin_lock_irqsave+34}
> <ffffffff8022cc03>{__WARN_ON+105}
> <ffffffff8022cbbe>{__WARN_ON+36}
> <ffffffff8024880b>{debug_rt_mutex_unlock+204}
> <ffffffff80486621>{rt_lock_slowunlock+30}
> <ffffffff80487196>{__lock_text_start+14}
> <ffffffff802792f9>{kmem_cache_alloc+207}
> <ffffffff8025b394>{mempool_alloc_slab+22}
> <ffffffff8025b783>{mempool_alloc+80}
> <ffffffff80248e35>{constant_test_bit+9}
> <ffffffff80487960>{_raw_spin_unlock+51}
> <ffffffff80486649>{rt_lock_slowunlock+70}
> <ffffffff802fde74>{get_request+375}
> <ffffffff80209a6d>{mcount+45}
> <ffffffff802fe04c>{get_request_wait+45}
> <ffffffff80209a6d>{mcount+45}
> <ffffffff803023c5>{as_merge+0}
> <ffffffff803023db>{as_merge+22}
> <ffffffff802fe425>{__make_request+750}
> <ffffffff803fc2b7>{md_thread+0}
> <ffffffff802fba78>{generic_make_request+380}
> <ffffffff80248e35>{constant_test_bit+9}
> <ffffffff80487960>{_raw_spin_unlock+51}
> <ffffffff803fc2b7>{md_thread+0}
> <ffffffff803ea43c>{raid1d+246}
> <ffffffff80209a6d>{mcount+45}
> <ffffffff80209a6d>{mcount+45}
> <ffffffff80486649>{rt_lock_slowunlock+70}
> <ffffffff80485568>{thread_return+230}
> <ffffffff80485568>{thread_return+230}
> <ffffffff80248e35>{constant_test_bit+9}
> <ffffffff80487960>{_raw_spin_unlock+51}
> <ffffffff80486649>{rt_lock_slowunlock+70}
> <ffffffff80485568>{thread_return+230}
> <ffffffff80487196>{__lock_text_start+14}
> <ffffffff803fc2b7>{md_thread+0}
> <ffffffff8023fb55>{keventd_create_kthread+0}
> <ffffffff803fc3cf>{md_thread+280}
> <ffffffff8023ff97>{autoremove_wake_function+0}
> <ffffffff8023fe5a>{kthread+224}
> <ffffffff802273bf>{schedule_tail+198}
> <ffffffff8020ae12>{child_rip+8}
> <ffffffff8023fb55>{keventd_create_kthread+0}
> <ffffffff80485568>{thread_return+230}
> <ffffffff80485568>{thread_return+230}
> <ffffffff80485568>{thread_return+230}
> <ffffffff8023fd7a>{kthread+0}
> <ffffffff8020ae0a>{child_rip+0}
> ---------------------------
> | preempt count: 00000002 ]
> | 2-level deep critical section nesting:
> ----------------------------------------
> .. [<ffffffff8048736f>] .... _raw_spin_lock+0x1b/0x28
> .....[<ffffffff80486619>] .. ( <= rt_lock_slowunlock+0x16/0x70)
> .. [<ffffffff80487453>] .... _raw_spin_lock_irqsave+0x22/0x33
> .....[<ffffffff8022cbbe>] .. ( <= __WARN_ON+0x24/0x8a)
>
> which makes:
>
> <ffffffff802792f9>{kmem_cache_alloc+207}
>
> the interesting part?
Yeah, I would say that. I'm sure it's also doing a NUMA alloc in there
too.
>
> However:
>
> rcrocomb@spanky:2.6.17-rt8$ grep DEBUG_INFO .config
> CONFIG_DEBUG_INFO=y
> rcrocomb@spanky:2.6.17-rt8$ cd arch/x86_64/boot/compressed/
> rcrocomb@spanky:compressed$ gdb vmlinux
> GNU gdb Red Hat Linux (6.3.0.0-1.122rh)
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu"...
> (no debugging symbols found)
> Using host libthread_db library "/lib64/libthread_db.so.1".
>
> (gdb) li *0xffffffff802792f9
> No symbol table is loaded. Use the "file" command.
>
> Huh.
??
Are you sure that vmlinux is the one created with the given config file?
There's been times when I added some configs and either forgot to
compile, or the compile failed, and I didn't notice, so the old binary
was being used.
-- Steve
>
> --
> Robert Crocombe
> [email protected]
>
> In case this might be useful:
>
> rcrocomb@spanky:2.6.17-rt8$ grep -i debug .config
> .
> .
> .
> CONFIG_DEBUG_KERNEL=y
> # CONFIG_DEBUG_SLAB is not set
> CONFIG_DEBUG_PREEMPT=y
> CONFIG_DEBUG_RT_MUTEXES=y
> CONFIG_DEBUG_PI_LIST=y
> CONFIG_DEBUG_TRACE_IRQFLAGS=y
> # CONFIG_DEBUG_KOBJECT is not set
> CONFIG_DEBUG_INFO=y
> # CONFIG_DEBUG_FS is not set
> # CONFIG_DEBUG_VM is not set
> CONFIG_DEBUG_RODATA=y
> CONFIG_IOMMU_DEBUG=y
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
