2.6.19-rc2mm2 as well as 2.6.19-rc3 panics with the following OOPS when I
run kernbench continuously and execute cpu offline and online in parallel.
This is a 2 socket 4 core tyan opteron system with 8Gs of RAM.
The kernel has lockdep, and other debug flags enabled. Also, we have a
lockdep warning with XFS. Attaching the dmesg and the .config used as well.
Here's the snapshot of lockdep warning + oops.
[ 469.336719]
[ 469.336721] =============================================
[ 469.336728] [ INFO: possible recursive locking detected ]
[ 469.336731] 2.6.19-rc3 #2
[ 469.336737] ---------------------------------------------
[ 469.336745] rm/4538 is trying to acquire lock:
[ 469.336753] (&(&ip->i_lock)->mr_lock){----}, at: [<ffffffff80320050>] xfs_ilock+0x52/0x77
[ 469.336782]
[ 469.336784] but task is already holding lock:
[ 469.336791] (&(&ip->i_lock)->mr_lock){----}, at: [<ffffffff80320050>] xfs_ilock+0x52/0x77
[ 469.336816]
[ 469.336817] other info that might help us debug this:
[ 469.336825] 3 locks held by rm/4538:
[ 469.336832] #0: (&inode->i_mutex/1){--..}, at: [<ffffffff80286055>] do_unlinkat+0x75/0x136
[ 469.336866] #1: (&inode->i_mutex){--..}, at: [<ffffffff80285f98>] vfs_unlink+0x45/0x8d
[ 469.336897] #2: (&(&ip->i_lock)->mr_lock){----}, at: [<ffffffff80320050>] xfs_ilock+0x52/0x77
[ 469.336928]
[ 469.336929] stack backtrace:
[ 469.336936]
[ 469.336938] Call Trace:
[ 469.336947] [<ffffffff8024b4cc>] __lock_acquire+0x381/0xb8c
[ 469.336957] [<ffffffff8024bf80>] lock_acquire+0x4b/0x66
[ 469.336966] [<ffffffff80320050>] xfs_ilock+0x52/0x77
[ 469.336976] [<ffffffff80247c97>] down_write+0x1e/0x27
[ 469.336985] [<ffffffff80320050>] xfs_ilock+0x52/0x77
[ 469.336993] [<ffffffff8033cad1>] xfs_lock_dir_and_entry+0x92/0xce
[ 469.337002] [<ffffffff8033cd23>] xfs_remove+0x216/0x44a
[ 469.337013] [<ffffffff80346c96>] xfs_vn_unlink+0x21/0x4f
[ 469.337024] [<ffffffff804a72b9>] __mutex_lock_slowpath+0x249/0x279
[ 469.337034] [<ffffffff8024aaaf>] mark_held_locks+0x65/0x84
[ 469.337043] [<ffffffff804a72b9>] __mutex_lock_slowpath+0x249/0x279
[ 469.337053] [<ffffffff8024ac7c>] trace_hardirqs_on+0x101/0x128
[ 469.337062] [<ffffffff804a72dc>] __mutex_lock_slowpath+0x26c/0x279
[ 469.337072] [<ffffffff802832d9>] permission+0xa0/0xa9
[ 469.337081] [<ffffffff80285fb4>] vfs_unlink+0x61/0x8d
[ 469.337090] [<ffffffff80286098>] do_unlinkat+0xb8/0x136
[ 469.337100] [<ffffffff804a8649>] trace_hardirqs_on_thunk+0x35/0x37
[ 469.337109] [<ffffffff8024ac7c>] trace_hardirqs_on+0x101/0x128
[ 469.337118] [<ffffffff804a8649>] trace_hardirqs_on_thunk+0x35/0x37
[ 469.337129] [<ffffffff80209a4e>] system_call+0x7e/0x83
[ 469.337137]
[ 488.264267] Unable to handle kernel NULL pointer dereference at 0000000000000000 RIP:
[ 488.264271] [<ffffffff8025d255>] __drain_pages+0x33/0x7d
[ 488.264288] PGD 2753ae067 PUD 275f66067 PMD 0
[ 488.264302] Oops: 0000 [1] PREEMPT SMP
[ 488.264315] CPU 2
[ 488.264323] Modules linked in:
[ 488.264332] Pid: 7617, comm: sh Not tainted 2.6.19-rc3 #2
[ 488.264339] RIP: 0010:[<ffffffff8025d255>] [<ffffffff8025d255>] __drain_pages+0x33/0x7d
[ 488.264353] RSP: 0018:ffff81016dd3fdf8 EFLAGS: 00010046
[ 488.264358] RAX: 0000000000000000 RBX: 0000000000000086 RCX: 0000000000000000
[ 488.264366] RDX: 0000000000000010 RSI: 0000000000000000 RDI: ffff810180000000
[ 488.264373] RBP: 0000000000000000 R08: 0000000000000002 R09: 0000000000000001
[ 488.264381] R10: ffffffff8025cb12 R11: 0000000000000000 R12: ffff810180000000
[ 488.264389] R13: 0000000000000001 R14: 0000000000000001 R15: ffff81027a7e5998
[ 488.264397] FS: 00002af4d80a5ae0(0000) GS:ffff81018001ad18(0000) knlGS:0000000000000000
[ 488.264405] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 488.264412] CR2: 0000000000000000 CR3: 00000002767bf000 CR4: 00000000000006a0
[ 488.264420] Process sh (pid: 7617, threadinfo ffff81016dd3e000, task ffff81016e03d080)
[ 488.264427] Stack: 0000000000000001 0000000000000001 0000000000000007 0000000000000001
[ 488.264448] 000000000000000f ffffffff8025e771 ffffffff80538860 ffffffff8023e3c2
[ 488.264466] 0000000000000001 0000000000000001 ffff81016e11a0c0 ffffffff8024ff50
[ 488.264481] Call Trace:
[ 488.264490] [<ffffffff8025e771>] page_alloc_cpu_notify+0x18/0x33
[ 488.264500] [<ffffffff8023e3c2>] notifier_call_chain+0x23/0x32
[ 488.264509] [<ffffffff8024ff50>] _cpu_down+0x184/0x245
[ 488.264516] [<ffffffff8025003c>] cpu_down+0x2b/0x42
[ 488.264525] [<ffffffff803b25b8>] store_online+0x27/0x71
[ 488.264534] [<ffffffff802b7b4a>] sysfs_write_file+0xb6/0xe5
[ 488.264544] [<ffffffff8027c6e5>] vfs_write+0xb2/0x155
[ 488.264551] [<ffffffff8027c83d>] sys_write+0x45/0x70
[ 488.264560] [<ffffffff80209a4e>] system_call+0x7e/0x83
[ 488.264566]
[ 488.264570]
[ 488.264572] Code: 8b 75 00 4c 89 e7 e8 82 f8 ff ff f6 c7 02 c7 45 00 00 00 00
[ 488.264634] RIP [<ffffffff8025d255>] __drain_pages+0x33/0x7d
[ 488.264644] RSP <ffff81016dd3fdf8>
[ 488.264650] CR2: 0000000000000000
[ 488.264655]
Thanks,
Kiran
[ 0.000000] Linux version 2.6.19-rc3 (kiran@x460) (gcc version 3.4.3 20050227 (Red Hat 3.4.3-22.1)) #2 SMP PREEMPT Tue Oct 24 12:24:39 PDT 2006 [ 0.000000] Command line: root=/dev/sda2 splash=silent showopts [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009b800 (usable) [ 0.000000] BIOS-e820: 000000000009b800 - 00000000000a0000 (reserved) [ 0.000000] BIOS-e820: 00000000000d4000 - 0000000000100000 (reserved) [ 0.000000] BIOS-e820: 0000000000100000 - 000000007ff10000 (usable) [ 0.000000] BIOS-e820: 000000007ff10000 - 000000007ff19000 (ACPI data) [ 0.000000] BIOS-e820: 000000007ff19000 - 000000007ff80000 (ACPI NVS) [ 0.000000] BIOS-e820: 000000007ff80000 - 0000000080000000 (reserved) [ 0.000000] BIOS-e820: 00000000e0000000 - 00000000e8000000 (reserved) [ 0.000000] BIOS-e820: 00000000fec00000 - 00000000fec00400 (reserved) [ 0.000000] BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved) [ 0.000000] BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved) [ 0.000000] BIOS-e820: 0000000100000000 - 0000000280000000 (usable) [ 0.000000] Entering add_active_range(0, 0, 155) 0 entries of 3200 used [ 0.000000] Entering add_active_range(0, 256, 524048) 1 entries of 3200 used [ 0.000000] Entering add_active_range(0, 1048576, 2621440) 2 entries of 3200 used [ 0.000000] end_pfn_map = 2621440 [ 0.000000] DMI present. [ 0.000000] ACPI: RSDP (v000 PTLTD ) @ 0x00000000000f7040 [ 0.000000] ACPI: RSDT (v001 PTLTD RSDT 0x06040000 LTP 0x00000000) @ 0x000000007ff153c7 [ 0.000000] ACPI: FADT (v001 NVIDIA CK8S 0x06040000 PTL_ 0x000f4240) @ 0x000000007ff18d36 [ 0.000000] ACPI: SRAT (v001 AMD HAMMER 0x06040000 AMD 0x00000001) @ 0x000000007ff18daa [ 0.000000] ACPI: SPCR (v001 PTLTD $UCRTBL$ 0x06040000 PTL 0x00000001) @ 0x000000007ff18eba [ 0.000000] ACPI: MCFG (v001 PTLTD MCFG 0x06040000 LTP 0x00000000) @ 0x000000007ff18f0a [ 0.000000] ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x000000007ff18f46 [ 0.000000] ACPI: BOOT (v001 PTLTD $SBFTBL$ 0x06040000 LTP 0x00000001) @ 0x000000007ff18fd8 [ 0.000000] ACPI: DSDT (v001 NVIDIA CK8 0x06040000 MSFT 0x0100000e) @ 0x0000000000000000 [ 0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0 [ 0.000000] SRAT: PXM 0 -> APIC 1 -> Node 0 [ 0.000000] SRAT: PXM 1 -> APIC 2 -> Node 1 [ 0.000000] SRAT: PXM 1 -> APIC 3 -> Node 1 [ 0.000000] SRAT: Node 0 PXM 0 0-a0000 [ 0.000000] Entering add_active_range(0, 0, 155) 0 entries of 3200 used [ 0.000000] SRAT: Node 0 PXM 0 0-80000000 [ 0.000000] Entering add_active_range(0, 0, 155) 1 entries of 3200 used [ 0.000000] Entering add_active_range(0, 256, 524048) 1 entries of 3200 used [ 0.000000] SRAT: Node 0 PXM 0 0-180000000 [ 0.000000] Entering add_active_range(0, 0, 155) 2 entries of 3200 used [ 0.000000] Entering add_active_range(0, 256, 524048) 2 entries of 3200 used [ 0.000000] Entering add_active_range(0, 1048576, 1572864) 2 entries of 3200 used [ 0.000000] SRAT: Node 1 PXM 1 180000000-280000000 [ 0.000000] Entering add_active_range(1, 1572864, 2621440) 3 entries of 3200 used [ 0.000000] NUMA: Using 31 for the hash shift. [ 0.000000] Bootmem setup node 0 0000000000000000-0000000180000000 [ 0.000000] Bootmem setup node 1 0000000180000000-0000000280000000 [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0 -> 4096 [ 0.000000] DMA32 4096 -> 1048576 [ 0.000000] Normal 1048576 -> 2621440 [ 0.000000] early_node_map[4] active PFN ranges [ 0.000000] 0: 0 -> 155 [ 0.000000] 0: 256 -> 524048 [ 0.000000] 0: 1048576 -> 1572864 [ 0.000000] 1: 1572864 -> 2621440 [ 0.000000] On node 0 totalpages: 1048235 [ 0.000000] DMA zone: 88 pages used for memmap [ 0.000000] DMA zone: 2151 pages reserved [ 0.000000] DMA zone: 1756 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 22440 pages used for memmap [ 0.000000] DMA32 zone: 497512 pages, LIFO batch:31 [ 0.000000] Normal zone: 11264 pages used for memmap [ 0.000000] Normal zone: 513024 pages, LIFO batch:31 [ 0.000000] On node 1 totalpages: 1048576 [ 0.000000] DMA zone: 0 pages used for memmap [ 0.000000] DMA32 zone: 0 pages used for memmap [ 0.000000] Normal zone: 22528 pages used for memmap [ 0.000000] Normal zone: 1026048 pages, LIFO batch:31 [ 0.000000] Nvidia board detected. Ignoring ACPI timer override. [ 0.000000] ACPI: PM-Timer IO Port: 0x8008 [ 0.000000] ACPI: Local APIC address 0xfee00000 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) [ 0.000000] Processor #0 (Bootup-CPU) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) [ 0.000000] Processor #1 [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) [ 0.000000] Processor #2 [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled) [ 0.000000] Processor #3 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) [ 0.000000] ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0]) [ 0.000000] IOAPIC[0]: apic_id 4, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: IOAPIC (id[0x05] address[0xd8000000] gsi_base[24]) [ 0.000000] IOAPIC[1]: apic_id 5, address 0xd8000000, GSI 24-27 [ 0.000000] ACPI: IOAPIC (id[0x06] address[0xd8001000] gsi_base[28]) [ 0.000000] IOAPIC[2]: apic_id 6, address 0xd8001000, GSI 28-31 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level) [ 0.000000] ACPI: IRQ9 used by override. [ 0.000000] Setting APIC routing to physical flat [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] Nosave address range: 000000000009b000 - 000000000009c000 [ 0.000000] Nosave address range: 000000000009c000 - 00000000000a0000 [ 0.000000] Nosave address range: 00000000000a0000 - 00000000000d4000 [ 0.000000] Nosave address range: 00000000000d4000 - 0000000000100000 [ 0.000000] Nosave address range: 000000007ff10000 - 000000007ff19000 [ 0.000000] Nosave address range: 000000007ff19000 - 000000007ff80000 [ 0.000000] Nosave address range: 000000007ff80000 - 0000000080000000 [ 0.000000] Nosave address range: 0000000080000000 - 00000000e0000000 [ 0.000000] Nosave address range: 00000000e0000000 - 00000000e8000000 [ 0.000000] Nosave address range: 00000000e8000000 - 00000000fec00000 [ 0.000000] Nosave address range: 00000000fec00000 - 00000000fee00000 [ 0.000000] Nosave address range: 00000000fee00000 - 00000000fee01000 [ 0.000000] Nosave address range: 00000000fee01000 - 00000000fff80000 [ 0.000000] Nosave address range: 00000000fff80000 - 0000000100000000 [ 0.000000] Allocating PCI resources starting at 88000000 (gap: 80000000:60000000) [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] PERCPU: Allocating 37632 bytes of per cpu data [ 0.000000] Built 2 zonelists. Total pages: 2038340 [ 0.000000] Kernel command line: root=/dev/sda2 splash=silent showopts [ 0.000000] Initializing CPU#0 [ 0.000000] PID hash table entries: 4096 (order: 12, 32768 bytes) [ 277.730450] Console: colour VGA+ 80x25 [ 277.734534] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar [ 277.734605] ... MAX_LOCKDEP_SUBCLASSES: 8 [ 277.734656] ... MAX_LOCK_DEPTH: 30 [ 277.734706] ... MAX_LOCKDEP_KEYS: 2048 [ 277.734758] ... CLASSHASH_SIZE: 1024 [ 277.734809] ... MAX_LOCKDEP_ENTRIES: 8192 [ 277.734860] ... MAX_LOCKDEP_CHAINS: 8192 [ 277.734911] ... CHAINHASH_SIZE: 4096 [ 277.734963] memory used by lock dependency info: 1328 kB [ 277.735017] per task-struct memory footprint: 1680 bytes [ 277.739261] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes) [ 277.746508] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 277.749122] Checking aperture... [ 277.749175] CPU 0: aperture @ 0 size 32 MB [ 277.751943] No AGP bridge found [ 277.751992] Your BIOS doesn't leave a aperture memory hole [ 277.752046] Please enable the IOMMU option in the BIOS setup [ 277.752099] This costs you 64 MB of RAM [ 277.785927] Mapping aperture over 65536 KB of RAM @ 4000000 [ 277.900475] Memory: 8074684k/10485760k available (2735k kernel code, 312560k reserved, 1281k data, 304k init) [ 278.048243] Calibrating delay using timer specific routine.. 4824.91 BogoMIPS (lpj=24124594) [ 278.048925] Mount-cache hash table entries: 256 [ 278.050110] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) [ 278.050167] CPU: L2 Cache: 1024K (64 bytes/line) [ 278.050220] CPU 0/0 -> Node 0 [ 278.050270] CPU: Physical Processor ID: 0 [ 278.050320] CPU: Processor Core ID: 0 [ 278.050389] lockdep: not fixing up alternatives. [ 278.050441] ACPI: Core revision 20060707 [ 278.158599] Using local APIC timer interrupts. [ 278.200058] result 12557978 [ 278.200107] Detected 12.557 MHz APIC timer. [ 278.209368] lockdep: not fixing up alternatives. [ 278.209740] Booting processor 1/4 APIC 0x1 [ 278.220064] Initializing CPU#1 [ 278.367716] Calibrating delay using timer specific routine.. 4822.38 BogoMIPS (lpj=24111902) [ 278.367722] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) [ 278.367724] CPU: L2 Cache: 1024K (64 bytes/line) [ 278.367726] CPU 1/1 -> Node 0 [ 278.367728] CPU: Physical Processor ID: 0 [ 278.367730] CPU: Processor Core ID: 1 [ 278.367848] Dual Core AMD Opteron(tm) Processor 280 stepping 02 [ 278.377712] CPU 1: Syncing TSC to CPU 0. [ 278.377827] CPU 1: synchronized TSC with CPU 0 (last diff 0 cycles, maxerr 637 cycles) [ 278.378892] lockdep: not fixing up alternatives. [ 278.379570] Booting processor 2/4 APIC 0x2 [ 278.389900] Initializing CPU#2 [ 278.537439] Calibrating delay using timer specific routine.. 4822.38 BogoMIPS (lpj=24111923) [ 278.537446] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) [ 278.537448] CPU: L2 Cache: 1024K (64 bytes/line) [ 278.537451] CPU 2/2 -> Node 1 [ 278.537453] CPU: Physical Processor ID: 1 [ 278.537454] CPU: Processor Core ID: 0 [ 278.537568] Dual Core AMD Opteron(tm) Processor 280 stepping 02 [ 278.547437] CPU 2: Syncing TSC to CPU 0. [ 278.547619] CPU 2: synchronized TSC with CPU 0 (last diff -17 cycles, maxerr 1238 cycles) [ 278.548616] lockdep: not fixing up alternatives. [ 278.549260] Booting processor 3/4 APIC 0x3 [ 278.560124] Initializing CPU#3 [ 278.707160] Calibrating delay using timer specific routine.. 4822.34 BogoMIPS (lpj=24111748) [ 278.707168] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) [ 278.707170] CPU: L2 Cache: 1024K (64 bytes/line) [ 278.707172] CPU 3/3 -> Node 1 [ 278.707175] CPU: Physical Processor ID: 1 [ 278.707176] CPU: Processor Core ID: 1 [ 278.707290] Dual Core AMD Opteron(tm) Processor 280 stepping 02 [ 278.717159] CPU 3: Syncing TSC to CPU 0. [ 278.717342] CPU 3: synchronized TSC with CPU 0 (last diff -17 cycles, maxerr 1238 cycles) [ 278.717348] Brought up 4 CPUs [ 278.718069] testing NMI watchdog ... OK. [ 278.818025] Disabling vsyscall due to use of PM timer [ 278.818079] time.c: Using 3.579545 MHz WALL PM GTOD PM timer. [ 278.818133] time.c: Detected 2411.129 MHz processor. [ 279.037931] migration_cost=344,788 [ 279.040588] NET: Registered protocol family 16 [ 279.041267] ACPI: bus type pci registered [ 279.041322] PCI: Using configuration type 1 [ 279.053074] ACPI: Interpreter enabled [ 279.053128] ACPI: Using IOAPIC for interrupt routing [ 279.055219] ACPI: PCI Root Bridge [PCI0] (0000:00) [ 279.055343] PCI: Probing PCI hardware (bus 00) [ 279.055723] ACPI: Assume root bridge [\_SB_.PCI0] bus is 0 [ 279.061084] Boot video device is 0000:01:07.0 [ 279.061515] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT] [ 279.109544] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P2P0._PRT] [ 279.112529] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.XVR0._PRT] [ 279.115015] ACPI: PCI Interrupt Link [LNK1] (IRQs 16 17 18 19) *0 [ 279.116022] ACPI: PCI Interrupt Link [LNK2] (IRQs 16 17 18 19) *0 [ 279.117040] ACPI: PCI Interrupt Link [LNK3] (IRQs 16 17 18 19) *0 [ 279.118050] ACPI: PCI Interrupt Link [LNK4] (IRQs 16 17 18 19) *0, disabled. [ 279.119099] ACPI: PCI Interrupt Link [LNK5] (IRQs 16 17 18 19) *0, disabled. [ 279.120146] ACPI: PCI Interrupt Link [LSMB] (IRQs 20 21 22 23) *0, disabled. [ 279.121181] ACPI: PCI Interrupt Link [LUS0] (IRQs 20 21 22 23) *0 [ 279.122193] ACPI: PCI Interrupt Link [LUS2] (IRQs 20 21 22 23) *0, disabled. [ 279.123240] ACPI: PCI Interrupt Link [LMAC] (IRQs 20 21 22 23) *0, disabled. [ 279.124280] ACPI: PCI Interrupt Link [LACI] (IRQs 20 21 22 23) *0, disabled. [ 279.125320] ACPI: PCI Interrupt Link [LMCI] (IRQs 20 21 22 23) *0, disabled. [ 279.126360] ACPI: PCI Interrupt Link [LPID] (IRQs 20 21 22 23) *0, disabled. [ 279.127429] ACPI: PCI Interrupt Link [LTID] (IRQs 20 21 22 23) *0 [ 279.128458] ACPI: PCI Interrupt Link [LSI1] (IRQs 20 21 22 23) *0, disabled. [ 279.129520] ACPI: PCI Interrupt Link [APCP] (IRQs 20 21 22 23) *0, disabled. [ 279.130126] ACPI: PCI Root Bridge [PCI2] (0000:08) [ 279.130181] PCI: Probing PCI hardware (bus 08) [ 279.130330] ACPI: Assume root bridge [\_SB_.PCI0] bus is 0 [ 279.137089] ACPI: PCI Interrupt Routing Table [\_SB_.PCI2.G0PA._PRT] [ 279.137629] ACPI: PCI Interrupt Routing Table [\_SB_.PCI2.G0PB._PRT] [ 279.139419] SCSI subsystem initialized [ 279.139687] usbcore: registered new interface driver usbfs [ 279.139874] usbcore: registered new interface driver hub [ 279.141126] usbcore: registered new device driver usb [ 279.141275] PCI: Using ACPI for IRQ routing [ 279.141328] PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report [ 279.141746] PCI-DMA: Disabling AGP. [ 279.141839] PCI-DMA: aperture base @ 4000000 size 65536 KB [ 279.141892] PCI-DMA: using GART IOMMU. [ 279.141945] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture [ 279.144501] PCI: Bridge: 0000:00:09.0 [ 279.144553] IO window: 2000-2fff [ 279.144605] MEM window: c8100000-c9ffffff [ 279.144658] PREFETCH window: 88000000-880fffff [ 279.144711] PCI: Bridge: 0000:00:0e.0 [ 279.144762] IO window: disabled. [ 279.144814] MEM window: ca000000-ca0fffff [ 279.144867] PREFETCH window: ca800000-d7ffffff [ 279.144925] PCI: Setting latency timer of device 0000:00:09.0 to 64 [ 279.144934] PCI: Setting latency timer of device 0000:00:0e.0 to 64 [ 279.144940] PCI: Bridge: 0000:08:0a.0 [ 279.144991] IO window: disabled. [ 279.145042] MEM window: disabled. [ 279.145093] PREFETCH window: disabled. [ 279.145149] PCI: Bridge: 0000:08:0b.0 [ 279.145198] IO window: disabled. [ 279.145251] MEM window: d8100000-d81fffff [ 279.145304] PREFETCH window: 88100000-881fffff [ 279.145897] NET: Registered protocol family 2 [ 279.256530] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes) [ 279.258408] TCP established hash table entries: 65536 (order: 9, 3670016 bytes) [ 279.262505] TCP bind hash table entries: 32768 (order: 8, 1835008 bytes) [ 279.264617] TCP: Hash tables configured (established 65536 bind 32768) [ 279.264711] TCP reno registered [ 279.265327] Simple Boot Flag at 0x36 set to 0x1 [ 279.266948] microcode: CPU0 not a capable Intel processor [ 279.267030] microcode: CPU1 not a capable Intel processor [ 279.267153] microcode: CPU2 not a capable Intel processor [ 279.267291] microcode: CPU3 not a capable Intel processor [ 279.267356] IA-32 Microcode Update Driver: v1.14a <[email protected]> [ 279.272881] SGI XFS with large block/inode numbers, no debug enabled [ 279.273920] io scheduler noop registered [ 279.274003] io scheduler anticipatory registered (default) [ 279.274118] io scheduler deadline registered [ 279.274284] io scheduler cfq registered [ 279.486101] PCI: Found HT MSI mapping on 0000:00:0e.0 with capability disabled [ 279.486186] PCI: Found HT MSI mapping on 0000:00:00.0 with capability enabled [ 279.486256] PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:08:0a.0 subordinate bus. [ 279.486333] PCI: MSI quirk detected. PCI_BUS_FLAGS_NO_MSI set for 0000:08:0b.0 subordinate bus. [ 279.557386] Real Time Clock Driver v1.12ac [ 279.557444] Linux agpgart interface v0.101 (c) Dave Jones [ 279.557500] Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled [ 279.558052] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A [ 279.558652] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A [ 279.561780] RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize [ 279.562996] loop: loaded (max 8 devices) [ 279.563048] Intel(R) PRO/1000 Network Driver - version 7.2.9-k2 [ 279.563103] Copyright (c) 1999-2006 Intel Corporation. [ 279.563396] tg3.c:v3.67 (October 18, 2006) [ 279.563470] ACPI: PCI Interrupt 0000:0a:09.0[A] -> GSI 28 (level, low) -> IRQ 28 [ 279.593970] eth0: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:31:5f:ac [ 279.594232] eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] [ 279.594305] eth0: dma_rwctrl[769f4000] dma_mask[64-bit] [ 279.594411] ACPI: PCI Interrupt 0000:0a:09.1[B] -> GSI 29 (level, low) -> IRQ 29 [ 279.627921] eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:133MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:31:5f:ad [ 279.628182] eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] [ 279.628255] eth1: dma_rwctrl[769f4000] dma_mask[64-bit] [ 279.628843] Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 [ 279.628901] ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx [ 279.629036] NFORCE-CK804: IDE controller at PCI slot 0000:00:06.0 [ 279.629106] NFORCE-CK804: chipset revision 242 [ 279.629158] NFORCE-CK804: not 100% native mode: will probe irqs later [ 279.629217] NFORCE-CK804: 0000:00:06.0 (rev f2) UDMA133 controller [ 279.629277] ide0: BM-DMA at 0x1400-0x1407, BIOS settings: hda:pio, hdb:pio [ 279.629416] ide1: BM-DMA at 0x1408-0x140f, BIOS settings: hdc:pio, hdd:pio [ 279.629552] Probing IDE interface ide0... [ 280.244638] Probing IDE interface ide1... [ 315.230991] ide1: Wait for ready failed before probe ! [ 315.836959] libata version 2.00 loaded. [ 315.837066] sata_nv 0000:00:07.0: version 2.0 [ 315.837804] ACPI: PCI Interrupt Link [LTID] enabled at IRQ 23 [ 315.837867] ACPI: PCI Interrupt 0000:00:07.0[A] -> Link [LTID] -> GSI 23 (level, high) -> IRQ 23 [ 315.838403] PCI: Setting latency timer of device 0000:00:07.0 to 64 [ 315.838733] ata1: SATA max UDMA/133 cmd 0x1440 ctl 0x1436 bmdma 0x1410 irq 23 [ 315.838891] ata2: SATA max UDMA/133 cmd 0x1438 ctl 0x1432 bmdma 0x1418 irq 23 [ 315.838965] scsi0 : sata_nv [ 316.175391] ata1: SATA link down (SStatus 0 SControl 300) [ 316.175469] scsi1 : sata_nv [ 316.504843] ata2: SATA link down (SStatus 0 SControl 300) [ 316.505656] ACPI: PCI Interrupt Link [LSI1] enabled at IRQ 22 [ 316.505719] ACPI: PCI Interrupt 0000:00:08.0[A] -> Link [LSI1] -> GSI 22 (level, high) -> IRQ 22 [ 316.506055] PCI: Setting latency timer of device 0000:00:08.0 to 64 [ 316.506232] ata3: SATA max UDMA/133 cmd 0x1458 ctl 0x144E bmdma 0x1420 irq 22 [ 316.506392] ata4: SATA max UDMA/133 cmd 0x1450 ctl 0x144A bmdma 0x1428 irq 22 [ 316.506465] scsi2 : sata_nv [ 317.004028] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300) [ 317.004687] ata3.00: ATA-7, max UDMA/133, 321672960 sectors: LBA48 NCQ (depth 0/1) [ 317.004759] ata3.00: ata3: dev 0 multi count 16 [ 317.005487] ata3.00: configured for UDMA/133 [ 317.005544] scsi3 : sata_nv [ 317.343453] ata4: SATA link down (SStatus 0 SControl 300) [ 317.344325] scsi 2:0:0:0: Direct-Access ATA WDC WD1600YD-01N 10.0 PQ: 0 ANSI: 5 [ 317.345270] SCSI device sda: 321672960 512-byte hdwr sectors (164697 MB) [ 317.345344] sda: Write Protect is off [ 317.345395] sda: Mode Sense: 00 3a 00 00 [ 317.345428] SCSI device sda: drive cache: write back [ 317.346721] SCSI device sda: 321672960 512-byte hdwr sectors (164697 MB) [ 317.346794] sda: Write Protect is off [ 317.346845] sda: Mode Sense: 00 3a 00 00 [ 317.346878] SCSI device sda: drive cache: write back [ 317.347108] sda: sda1 sda2 sda3 [ 317.354142] sd 2:0:0:0: Attached scsi disk sda [ 317.354542] sd 2:0:0:0: Attached scsi generic sg0 type 0 [ 317.354671] USB Universal Host Controller Interface driver v3.0 [ 317.354895] Initializing USB Mass Storage driver... [ 317.355054] usbcore: registered new interface driver usb-storage [ 317.355111] USB Mass Storage support registered. [ 317.355268] usbcore: registered new interface driver usbhid [ 317.355324] drivers/usb/input/hid-core.c: v2.6:USB HID core driver [ 317.357957] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 317.358056] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 317.358437] mice: PS/2 mouse device common for all mice [ 317.358587] md: linear personality registered for level -1 [ 317.358906] device-mapper: ioctl: 4.10.0-ioctl (2006-09-14) initialised: [email protected] [ 317.359082] TCP cubic registered [ 317.359168] NET: Registered protocol family 1 [ 317.359233] NET: Registered protocol family 17 [ 317.359702] ACPI: (supports S0 S1 S4 S5) [ 317.398060] input: AT Translated Set 2 keyboard as /class/input/input0 [ 318.285432] input: ImExPS/2 Generic Explorer Mouse as /class/input/input1 [ 318.353007] md: Autodetecting RAID arrays. [ 318.353059] md: autorun ... [ 318.353107] md: ... autorun DONE. [ 318.361835] EXT3-fs: INFO: recovery required on readonly filesystem. [ 318.361891] EXT3-fs: write access will be enabled during recovery. [ 320.769049] kjournald starting. Commit interval 5 seconds [ 320.769189] EXT3-fs: sda2: orphan cleanup on readonly fs [ 320.769252] ext3_orphan_cleanup: deleting unreferenced inode 1444609 [ 320.770564] ext3_orphan_cleanup: deleting unreferenced inode 1444611 [ 320.770582] ext3_orphan_cleanup: deleting unreferenced inode 2611585 [ 320.770821] EXT3-fs: sda2: 3 orphan inodes deleted [ 320.770873] EXT3-fs: recovery complete. [ 320.778722] EXT3-fs: mounted filesystem with ordered data mode. [ 320.779037] VFS: Mounted root (ext3 filesystem) readonly. [ 320.779181] Freeing unused kernel memory: 304k freed [ 321.294430] warning: process `showconsole' used the removed sysctl system call [ 321.663630] warning: process `showconsole' used the removed sysctl system call [ 321.807213] EXT3 (no)acl options not supported [ 321.807271] EXT3 (no)user_xattr options not supported [ 322.029072] EXT3 (no)acl options not supported [ 322.029130] EXT3 (no)user_xattr options not supported [ 322.029572] EXT3 FS on sda2, internal journal [ 322.693490] warning: process `ls' used the removed sysctl system call [ 322.694815] warning: process `ls' used the removed sysctl system call [ 322.764619] warning: process `ls' used the removed sysctl system call [ 323.789944] EXT3 (no)acl options not supported [ 323.790001] EXT3 (no)user_xattr options not supported [ 323.794042] kjournald starting. Commit interval 5 seconds [ 323.794279] EXT3 FS on sda1, internal journal [ 323.794365] EXT3-fs: mounted filesystem with ordered data mode. [ 324.195824] XFS mounting filesystem sda3 [ 324.272445] Starting XFS recovery on filesystem: sda3 (logdev: internal) [ 324.427368] Ending XFS recovery on filesystem: sda3 (logdev: internal) [ 327.851923] PM: Writing back config space on device 0000:0a:09.0 at offset b (was 164814e4, writing 164414e4) [ 327.851937] PM: Writing back config space on device 0000:0a:09.0 at offset 3 (was 804000, writing 804010) [ 327.851943] PM: Writing back config space on device 0000:0a:09.0 at offset 2 (was 2000000, writing 2000003) [ 327.851948] PM: Writing back config space on device 0000:0a:09.0 at offset 1 (was 2b00000, writing 2b00106) [ 330.623384] tg3: eth0: Link is up at 1000 Mbps, full duplex. [ 330.623389] tg3: eth0: Flow control is on for TX and on for RX. [ 469.336719] [ 469.336721] ============================================= [ 469.336728] [ INFO: possible recursive locking detected ] [ 469.336731] 2.6.19-rc3 #2 [ 469.336737] --------------------------------------------- [ 469.336745] rm/4538 is trying to acquire lock: [ 469.336753] (&(&ip->i_lock)->mr_lock){----}, at: [<ffffffff80320050>] xfs_ilock+0x52/0x77 [ 469.336782] [ 469.336784] but task is already holding lock: [ 469.336791] (&(&ip->i_lock)->mr_lock){----}, at: [<ffffffff80320050>] xfs_ilock+0x52/0x77 [ 469.336816] [ 469.336817] other info that might help us debug this: [ 469.336825] 3 locks held by rm/4538: [ 469.336832] #0: (&inode->i_mutex/1){--..}, at: [<ffffffff80286055>] do_unlinkat+0x75/0x136 [ 469.336866] #1: (&inode->i_mutex){--..}, at: [<ffffffff80285f98>] vfs_unlink+0x45/0x8d [ 469.336897] #2: (&(&ip->i_lock)->mr_lock){----}, at: [<ffffffff80320050>] xfs_ilock+0x52/0x77 [ 469.336928] [ 469.336929] stack backtrace: [ 469.336936] [ 469.336938] Call Trace: [ 469.336947] [<ffffffff8024b4cc>] __lock_acquire+0x381/0xb8c [ 469.336957] [<ffffffff8024bf80>] lock_acquire+0x4b/0x66 [ 469.336966] [<ffffffff80320050>] xfs_ilock+0x52/0x77 [ 469.336976] [<ffffffff80247c97>] down_write+0x1e/0x27 [ 469.336985] [<ffffffff80320050>] xfs_ilock+0x52/0x77 [ 469.336993] [<ffffffff8033cad1>] xfs_lock_dir_and_entry+0x92/0xce [ 469.337002] [<ffffffff8033cd23>] xfs_remove+0x216/0x44a [ 469.337013] [<ffffffff80346c96>] xfs_vn_unlink+0x21/0x4f [ 469.337024] [<ffffffff804a72b9>] __mutex_lock_slowpath+0x249/0x279 [ 469.337034] [<ffffffff8024aaaf>] mark_held_locks+0x65/0x84 [ 469.337043] [<ffffffff804a72b9>] __mutex_lock_slowpath+0x249/0x279 [ 469.337053] [<ffffffff8024ac7c>] trace_hardirqs_on+0x101/0x128 [ 469.337062] [<ffffffff804a72dc>] __mutex_lock_slowpath+0x26c/0x279 [ 469.337072] [<ffffffff802832d9>] permission+0xa0/0xa9 [ 469.337081] [<ffffffff80285fb4>] vfs_unlink+0x61/0x8d [ 469.337090] [<ffffffff80286098>] do_unlinkat+0xb8/0x136 [ 469.337100] [<ffffffff804a8649>] trace_hardirqs_on_thunk+0x35/0x37 [ 469.337109] [<ffffffff8024ac7c>] trace_hardirqs_on+0x101/0x128 [ 469.337118] [<ffffffff804a8649>] trace_hardirqs_on_thunk+0x35/0x37 [ 469.337129] [<ffffffff80209a4e>] system_call+0x7e/0x83 [ 469.337137] [ 487.063562] CPU 1 is now offline [ 487.134710] Breaking affinity for irq 22 [ 487.243265] CPU 2 is now offline [ 487.422969] CPU 3 is now offline [ 487.422978] lockdep: not fixing up alternatives. [ 487.472345] lockdep: not fixing up alternatives. [ 487.472637] Booting processor 1/4 APIC 0x1 [ 487.483163] Initializing CPU#1 [ 487.625211] Calibrating delay using timer specific routine.. 4822.28 BogoMIPS (lpj=24111431) [ 487.625217] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) [ 487.625219] CPU: L2 Cache: 1024K (64 bytes/line) [ 487.625221] CPU 1/1 -> Node 0 [ 487.625223] CPU: Physical Processor ID: 0 [ 487.625225] CPU: Processor Core ID: 1 [ 487.625321] Dual Core AMD Opteron(tm) Processor 280 stepping 02 [ 487.635210] CPU 1: Syncing TSC to CPU 0. [ 487.635306] CPU 1: synchronized TSC with CPU 0 (last diff 0 cycles, maxerr 638 cycles) [ 487.635487] microcode: CPU1 not a capable Intel processor [ 487.685423] lockdep: not fixing up alternatives. [ 487.685433] Booting processor 2/4 APIC 0x2 [ 487.695710] Initializing CPU#2 [ 487.844852] Calibrating delay using timer specific routine.. 4822.40 BogoMIPS (lpj=24112032) [ 487.844859] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) [ 487.844861] CPU: L2 Cache: 1024K (64 bytes/line) [ 487.844864] CPU 2/2 -> Node 1 [ 487.844866] CPU: Physical Processor ID: 1 [ 487.844867] CPU: Processor Core ID: 0 [ 487.844978] Dual Core AMD Opteron(tm) Processor 280 stepping 02 [ 487.854844] CPU 2: Syncing TSC to CPU 0. [ 487.855026] CPU 2: synchronized TSC with CPU 0 (last diff -17 cycles, maxerr 1238 cycles) [ 487.855237] microcode: CPU2 not a capable Intel processor [ 487.904974] lockdep: not fixing up alternatives. [ 487.904984] Booting processor 3/4 APIC 0x3 [ 487.915261] Initializing CPU#3 [ 488.064492] Calibrating delay using timer specific routine.. 4822.42 BogoMIPS (lpj=24112120) [ 488.064500] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) [ 488.064502] CPU: L2 Cache: 1024K (64 bytes/line) [ 488.064504] CPU 3/3 -> Node 1 [ 488.064507] CPU: Physical Processor ID: 1 [ 488.064508] CPU: Processor Core ID: 1 [ 488.064619] Dual Core AMD Opteron(tm) Processor 280 stepping 02 [ 488.074489] CPU 3: Syncing TSC to CPU 0. [ 488.074673] CPU 3: synchronized TSC with CPU 0 (last diff -3 cycles, maxerr 1239 cycles) [ 488.074875] microcode: CPU3 not a capable Intel processor [ 488.264171] CPU 1 is now offline [ 488.264267] Unable to handle kernel NULL pointer dereference at 0000000000000000 RIP: [ 488.264271] [<ffffffff8025d255>] __drain_pages+0x33/0x7d [ 488.264288] PGD 2753ae067 PUD 275f66067 PMD 0 [ 488.264302] Oops: 0000 [1] PREEMPT SMP [ 488.264315] CPU 2 [ 488.264323] Modules linked in: [ 488.264332] Pid: 7617, comm: sh Not tainted 2.6.19-rc3 #2 [ 488.264339] RIP: 0010:[<ffffffff8025d255>] [<ffffffff8025d255>] __drain_pages+0x33/0x7d [ 488.264353] RSP: 0018:ffff81016dd3fdf8 EFLAGS: 00010046 [ 488.264358] RAX: 0000000000000000 RBX: 0000000000000086 RCX: 0000000000000000 [ 488.264366] RDX: 0000000000000010 RSI: 0000000000000000 RDI: ffff810180000000 [ 488.264373] RBP: 0000000000000000 R08: 0000000000000002 R09: 0000000000000001 [ 488.264381] R10: ffffffff8025cb12 R11: 0000000000000000 R12: ffff810180000000 [ 488.264389] R13: 0000000000000001 R14: 0000000000000001 R15: ffff81027a7e5998 [ 488.264397] FS: 00002af4d80a5ae0(0000) GS:ffff81018001ad18(0000) knlGS:0000000000000000 [ 488.264405] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 488.264412] CR2: 0000000000000000 CR3: 00000002767bf000 CR4: 00000000000006a0 [ 488.264420] Process sh (pid: 7617, threadinfo ffff81016dd3e000, task ffff81016e03d080) [ 488.264427] Stack: 0000000000000001 0000000000000001 0000000000000007 0000000000000001 [ 488.264448] 000000000000000f ffffffff8025e771 ffffffff80538860 ffffffff8023e3c2 [ 488.264466] 0000000000000001 0000000000000001 ffff81016e11a0c0 ffffffff8024ff50 [ 488.264481] Call Trace: [ 488.264490] [<ffffffff8025e771>] page_alloc_cpu_notify+0x18/0x33 [ 488.264500] [<ffffffff8023e3c2>] notifier_call_chain+0x23/0x32 [ 488.264509] [<ffffffff8024ff50>] _cpu_down+0x184/0x245 [ 488.264516] [<ffffffff8025003c>] cpu_down+0x2b/0x42 [ 488.264525] [<ffffffff803b25b8>] store_online+0x27/0x71 [ 488.264534] [<ffffffff802b7b4a>] sysfs_write_file+0xb6/0xe5 [ 488.264544] [<ffffffff8027c6e5>] vfs_write+0xb2/0x155 [ 488.264551] [<ffffffff8027c83d>] sys_write+0x45/0x70 [ 488.264560] [<ffffffff80209a4e>] system_call+0x7e/0x83 [ 488.264566] [ 488.264570] [ 488.264572] Code: 8b 75 00 4c 89 e7 e8 82 f8 ff ff f6 c7 02 c7 45 00 00 00 00 [ 488.264634] RIP [<ffffffff8025d255>] __drain_pages+0x33/0x7d [ 488.264644] RSP <ffff81016dd3fdf8> [ 488.264650] CR2: 0000000000000000 [ 488.264655]
Attachment:
config.gz
Description: GNU Zip compressed data
- Prev by Date: 2.6.19-rc3: known unfixed regressions
- Next by Date: Re: ext3: bogus i_mode errors with 2.6.18.1
- Previous by thread: [PATCH] i386: Mark CONFIG_RELOCATABLE EXPERIMENTAL
- Next by thread: v4l compat ioctls broken
- Index(es):
 |