2.6.17-1.2145_FC5 mmap-related soft lockup

["Gary Funck" <gary@intrepid.com>]

A test program which allocates about 256M of MAP_ANONYMOUS mmap memory,
and then spawns 4 processess, where each process i writes to 1/4 of the
mapped memory, and then reads the memory written by
the process (i + 1)%4, triggers a soft lockup, when exiting.
Hardware:
dual core dual Opteron 275 (Tyan motherboard, 4G physical memory)
has been rock solid reliable.

BUG: soft lockup detected on CPU#3!

Call Trace: <IRQ> <ffffffff802b2fb5>{softlockup_tick+219}
       <ffffffff8029708e>{update_process_times+66}
<ffffffff8027a3ed>{smp_local_timer_interrupt+35}
       <ffffffff8027aa95>{smp_apic_timer_interrupt+65}
<ffffffff80263acb>{apic_timer_interrupt+135} <EOI>
       <ffffffff8020e578>{__set_page_dirty_nobuffers+0}
<ffffffff8026a128>{_write_unlock_irq+11}
       <ffffffff8020e62d>{__set_page_dirty_nobuffers+181}
<ffffffff80207af6>{unmap_vmas+1037}
       <ffffffff8023c7d9>{exit_mmap+120} <ffffffff8023eda8>{mmput+44}
       <ffffffff80215ece>{do_exit+599}
<ffffffff8024cacd>{debug_mutex_init+0}
       <ffffffff80262f01>{tracesys+209}
BUG: soft lockup detected on CPU#0!

Call Trace: <IRQ> <ffffffff802b2fb5>{softlockup_tick+219}
       <ffffffff8029708e>{update_process_times+66}
<ffffffff8027a3ed>{smp_local_timer_interrupt+35}
       <ffffffff8027aa95>{smp_apic_timer_interrupt+65}
<ffffffff80263acb>{apic_timer_interrupt+135} <EOI>
       <ffffffff8020e578>{__set_page_dirty_nobuffers+0}
<ffffffff8026a128>{_write_unlock_irq+11}
       <ffffffff8020e62d>{__set_page_dirty_nobuffers+181}
<ffffffff80207af6>{unmap_vmas+1037}
       <ffffffff8023c7d9>{exit_mmap+120} <ffffffff8023eda8>{mmput+44}
       <ffffffff80215ece>{do_exit+599}
<ffffffff8024cacd>{debug_mutex_init+0}
       <ffffffff80262f01>{tracesys+209}
BUG: soft lockup detected on CPU#2!

Call Trace: <IRQ> <ffffffff802b2fb5>{softlockup_tick+219}
       <ffffffff8029708e>{update_process_times+66}
<ffffffff8027a3ed>{smp_local_timer_interrupt+35}
       <ffffffff8027aa95>{smp_apic_timer_interrupt+65}
<ffffffff80263acb>{apic_timer_interrupt+135} <EOI>
       <ffffffff8020e578>{__set_page_dirty_nobuffers+0}
<ffffffff8026a128>{_write_unlock_irq+11}
       <ffffffff8020e62d>{__set_page_dirty_nobuffers+181}
<ffffffff80207af6>{unmap_vmas+1037}
       <ffffffff8023c7d9>{exit_mmap+120} <ffffffff8023eda8>{mmput+44}
       <ffffffff80215ece>{do_exit+599}
<ffffffff8024cacd>{debug_mutex_init+0}
       <ffffffff80262f01>{tracesys+209}
BUG: soft lockup detected on CPU#1!

Call Trace: <IRQ> <ffffffff802b2fb5>{softlockup_tick+219}
       <ffffffff8029708e>{update_process_times+66}
<ffffffff8027a3ed>{smp_local_timer_interrupt+35}
       <ffffffff8027aa95>{smp_apic_timer_interrupt+65}
<ffffffff80263acb>{apic_timer_interrupt+135} <EOI>
       <ffffffff8020e578>{__set_page_dirty_nobuffers+0}
<ffffffff8026a128>{_write_unlock_irq+11}
       <ffffffff8020e62d>{__set_page_dirty_nobuffers+181}
<ffffffff80207af6>{unmap_vmas+1037}
       <ffffffff8023c7d9>{exit_mmap+120} <ffffffff8023eda8>{mmput+44}
       <ffffffff80215ece>{do_exit+599}
<ffffffff8024cacd>{debug_mutex_init+0}
       <ffffffff80262f01>{tracesys+209}

The test program runs successfully, but hangs several seconds upon exit.

The hardware and software configuration has been solid for several months,
but
we have seen timer-related synchronization issues with recent kernels (where
ntp has to force a re-sync for example, and an occasional lost ticks
message).

The test program mentioned above is more complicated than described, and
can't easily be reproduced in source form, but the binary could be
made available.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/