Hi there, My company's (ISP) bussines model requires dynamic resizing of the client queues. It's achieved by regenerating shaping rules and loading then using batch mode of a tc binary. On production systems it's done once every 1 or 2 minutes. Unfortunately this causes smp kernels to panic. Non-smp kernels don't have such problems. Bug is around a long time. I first noticed it after migrating to shaping configs that use IFB, it might have been 2.6.18 "era". Test scenario: I've put together a test machine with configuration copied from production router. I'm feeding the machine with production traffic by means of port mirroring. Test machine has the same config as production one (including mac addresses), so it tries to route the incoming traffic. Tested kernels were 2.6.31.1 and 2.6.20.6 (config from 2.6.20.6 is in attachment). Both panicked if compiled with SMP support and work stable otherwise. Problem occurs only with cyclic "shaping restarts". For the test, reload operation using "tc -b ..." was executed in an infinite loop. Box's CPU usage was approximately 15%. Panics occur with few hours - one day intervals. Below I attach the panic message captured via serial console: ---------------------------------------------------------------------- printk: 63 messages suppressed. dst cache overflow SMP Modules linked in: ipt_LOG xt_hashlimit ipt_MASQUERADE ip_set_macipmap ip_set_ipmap xt_state w83627hf hwmon_vid eeprom ifb ipt_SET ipt_set ip_set ipip tunnel4 ip_gre e1000 i2c_i801 i2c_core CPU: 1 EIP: 0060:[<f255c08f>] Not tainted VLI EFLAGS: 00010202 (2.6.23.1-smp #2) EIP is at 0xf255c08f eax: c196a000 ebx: 00000100 ecx: ef2f408c edx: f3630000 esi: f0332029 edi: f255c08c ebp: 00000001 esp: f3631ea8 ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0068 Process tc (pid: 27695, ti=f3630000 task=f26a9560 task.ti=f3630000) Stack: c01280ad f26a9560 00000001 f3631eb4 c0106f57 ef2f408c f0e8c08c 00000031 c0495308 0000000a c0124eb6 00000046 00000000 f7657740 f3630000 c0124f4c c180f120 c0114e62 00000000 00000000 f7bfd224 f74ed95c c01047e0 f7bfd224 Call Trace: [<c01280ad>] run_timer_softirq+0xf5/0x154 [<c0106f57>] profile_pc+0x21/0x4a [<c0124eb6>] __do_softirq+0x5d/0xc1 [<c0124f4c>] do_softirq+0x32/0x36 [<c0114e62>] smp_apic_timer_interrupt+0x74/0x80 [<c01047e0>] apic_timer_interrupt+0x28/0x30 [<c014c82e>] remove_vma+0x1c/0x36 [<c014c912>] exit_mmap+0xca/0xe1 [<c011eedc>] mmput+0x1d/0x75 [<c0123688>] do_exit+0x1be/0x68a [<c0123bc0>] sys_exit_group+0x0/0xd [<c0103d12>] sysenter_past_esp+0x5f/0x85 ======================= Code: 00 52 41 f7 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fa 00 00 00 ea 05 00 00 7f 00 00 00 34 a5 96 <c1> 8c 00 fd f3 a5 2b 09 00 d1 d8 32 c0 00 c0 55 f2 00 a0 96 c1 EIP: [<f255c08f>] 0xf255c08f SS:ESP 0068:f3631ea8 Kernel panic - not syncing: Fatal exception in interrupt ---------------------------------------------------------------------- -- Marek Kierdelewicz Kierownik Działu Systemów Sieciowych, KoBa Manager of Network Systems Department, KoBa tel. (85) 7406466; fax. (85) 7406467 e-mail: [email protected]
Attachment:
kernel_config
Description: Binary data
- Follow-Ups:
- Re: 2.6.23.1-smp kernel panic (network-related)
- From: Andrew Morton <[email protected]>
- Re: 2.6.23.1-smp kernel panic (network-related)
- Prev by Date: Re: [to-be-posted-soon] Multiple handlers per marker
- Next by Date: Re: [RFC/PATCH] AVR32: Oprofile support
- Previous by thread: [PATCH] frv: Remove bogus NO_IRQ = -1 define
- Next by thread: Re: 2.6.23.1-smp kernel panic (network-related)
- Index(es):
 |