For the last few months we have had "hangs" where the machine
will respond to pings , but nothing else. After upgrading the kernel
we now get a useful message when it hangs, so I thought that
with the message and an example program perhaps someone can track this down.
The nature of the issue is with a program that execs many processes
quickly in succession on an SMP machine and these processes read the
some of the same
files over a CIFS share. At random times the machine hangs under this load.
I created a shell script (below) that does a similar thing and can
reliably cause the
machine to hang.
We are running:
Linux prodeng2 2.6.17-1.2142_FC4smp #1 SMP Tue Jul 11 22:57:02 EDT
2006 i686 i686 i386 GNU/Linux
we get the following error:
Sep 16 00:04:30 prodeng2 kernel: BUG: soft lockup detected on CPU#2!
Sep 16 00:04:30 prodeng2 kernel: <c0447823> softlockup_tick+0x9f/0xb2 <c042b497> update_process_times+0x39/0x5c
Sep 16 00:04:30 prodeng2 kernel: <c041737d> smp_apic_timer_interrupt+0x54/0x5a <c0404800> apic_timer_interrupt+0x1c/0x24
Sep 16 00:04:30 prodeng2 kernel: <c046eed9> generic_fillattr+0x71/0x9f <f8ff18db> cifs_getattr+0x1e/0x27 [cifs]
Sep 16 00:04:30 prodeng2 kernel: <f8ff18bd> cifs_getattr+0x0/0x27 [cifs] <c046ef45> vfs_getattr+0x3e/0x97
Sep 16 00:04:30 prodeng2 kernel: <c046f051> vfs_fstat+0x22/0x31 <c046f724> sys_fstat64+0xf/0x23
Sep 16 00:04:30 prodeng2 kernel: <c0403d2f> syscall_call+0x7/0xb
Sep 16 00:04:30 prodeng2 kernel: BUG: soft lockup detected on CPU#1!
Sep 16 00:04:30 prodeng2 kernel: <c0447823> softlockup_tick+0x9f/0xb2 <c042b497> update_process_times+0x39/0x5c
Sep 16 00:04:30 prodeng2 kernel: <c041737d> smp_apic_timer_interrupt+0x54/0x5a <c0404800> apic_timer_interrupt+0x1c/0x24
Sep 16 00:04:30 prodeng2 kernel: <c046eecd> generic_fillattr+0x65/0x9f <f8ff18db> cifs_getattr+0x1e/0x27 [cifs]
Sep 16 00:04:30 prodeng2 kernel: <f8ff18bd> cifs_getattr+0x0/0x27 [cifs] <c046ef45> vfs_getattr+0x3e/0x97
Sep 16 00:04:30 prodeng2 kernel: <c046f051> vfs_fstat+0x22/0x31 <c046f724> sys_fstat64+0xf/0x23
Sep 16 00:04:30 prodeng2 kernel: <c0403d2f> syscall_call+0x7/0xb
...after googling a little, the closest similar report I find is:
http://blog.gmane.org/gmane.comp.emulators.xen.devel/day=20060916
To tickle this bug do:
1. Create a CIFS share, ideally on a real Windows server since that
is the current context of the issue I am reporting
2. Run this shell script using a path to a file on the share:
file_on_cifs_mnt=$1
while true
do
# limited fork bomb...repeat until hang
c=1;
while [ $c -lt 10000 ]
do
# cif mounted file will cause hang
(cat $file_on_cifs_mnt > /dev/null ; ) &
# local file won't hang me
# (cat /etc/passwd > /dev/null ; ) &
let c=c+1
done
wait
done
NOTE: sometimes it would hang immediately, sometimes after a time,
sometimes I had to run 2 instances of the script concurrently to get the
BUG.
If anyone has a fix OR a work around I would very much appreciate the help.
Thx.
Russ
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
