Kernel BUG at mm/slab.c:2564 - 2.6.16-rc5-g7b14e3b5

[Paolo Ornati <ornati@fastwebnet.it>]

It's the first time I see this, I don't think it is reproducible...

I was emerging (Gentoo) something and working on a reiserfs partition
when this happened:


grep[21235]: segfault at 00000079ecf16039 rip 0000003cf6701274 rsp
00007fffffd95e80 error 4 slab: Internal list corruption detected in
cache 'anon_vma'(92), slabp ffff810000a07000(16). Hexdump:

000: 00 01 10 00 00 00 00 00 00 02 20 00 00 00 00 00
010: a0 01 00 00 00 00 00 00 a0 71 a0 00 00 81 ff ff
020: 10 00 00 00 10 00 00 00 00 00 00 00 00 00 00 00
030: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
040: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
050: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
060: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
070: 11 00 00 00 12 00 00 00 13 00 00 00 14 00 00 00
080: 15 00 00 00 16 00 00 00 17 00 00 00 18 00 00 00
090: 19 00 00 00 1a 00 00 00 1b 00 00 00 1c 00 00 00
0a0: 1d 00 00 00 1e 00 00 00 7f 00 00 00 20 00 00 00
0b0: 21 00 00 00 22 00 00 00 23 00 00 00 24 00 00 00
0c0: 25 00 00 00 26 00 00 00 27 00 00 00 28 00 00 00
0d0: 29 00 00 00 2a 00 00 00 2b 00 00 00 2c 00 00 00
0e0: 2d 00 00 00 2e 00 00 00 2f 00 00 00 30 00 00 00
0f0: 31 00 00 00 32 00 00 00 33 00 00 00 34 00 00 00
100: 35 00 00 00 36 00 00 00 37 00 00 00 38 00 00 00
110: 39 00 00 00 3a 00 00 00 3b 00 00 00 3c 00 00 00
120: 3d 00 00 00 3e 00 00 00 3f 00 00 00 40 00 00 00
130: 41 00 00 00 42 00 00 00 43 00 00 00 44 00 00 00
140: 45 00 00 00 46 00 00 00 47 00 00 00 48 00 00 00
150: 49 00 00 00 4a 00 00 00 4b 00 00 00 4c 00 00 00
160: 4d 00 00 00 4e 00 00 00 4f 00 00 00 50 00 00 00
170: 51 00 00 00 52 00 00 00
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at mm/slab.c:2564
invalid opcode: 0000 [1] 
CPU 0 
Modules linked in: xt_state ip_queue ip_conntrack iptable_filter
ip_tables Pid: 21232, comm: as Not tainted 2.6.16-rc5-g7b14e3b5 #7
RIP: 0010:[<ffffffff80156c5e>] <ffffffff80156c5e>{check_slabp+188}
RSP: 0018:ffff81000a2cbdc8  EFLAGS: 00010096
RAX: 0000000000000001 RBX: 0000000000000178 RCX: 0000000000003ee7
RDX: 00000000ffffff01 RSI: 0000000000003ee7 RDI: ffffffff803f24c0
RBP: ffff810000a07000 R08: 00000000fffffffe R09: ffff810000a07000
R10: 0000000000000046 R11: 0000000000000000 R12: ffff81001ff2cec0
R13: ffff810000a071a0 R14: 0000000000000000 R15: ffff81000a2cbee0
FS:  00002af4db709dc0(0000) GS:ffffffff804cb000(0000)
knlGS:00000000f6cd7bb0 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000003cf698c600 CR3: 000000000c1a3000 CR4: 00000000000006e0
Process as (pid: 21232, threadinfo ffff81000a2ca000, task
ffff810017e7a100) Stack: ffff810000a07000 ffff81001ff2bf20
ffff81001ff2cec0 ffffffff80157763 ffff81001ff280a8 ffff810005f24000
0000000000000010 0000000000000010 ffff81001ff28098 ffff81001ff2bf20 
Call Trace: <ffffffff80157763>{free_block+154}
<ffffffff8015796c>{cache_flusharray+111}
<ffffffff80157585>{kmem_cache_free+78}
<ffffffff80148cac>{free_pgtables+45} <ffffffff8014e1fc>{exit_mmap+119}
<ffffffff8012210c>{mmput+27} <ffffffff801263d6>{do_exit+519}
<ffffffff801269d7>{sys_exit_group+0} <ffffffff8010a5b2>{system_call+126}

Code: 0f 0b 68 4f e6 38 80 c2 04 0a 5b 5d 41 5c c3 41 55 31 c0 48 
RIP <ffffffff80156c5e>{check_slabp+188} RSP <ffff81000a2cbdc8>
 <1>Fixing recursive fault but reboot is needed!


Full dmesg & config attached.


PS: the machine is still running and other messages are appearing:


slab: double free detected in cache 'anon_vma', objp ffff81000511ca38
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at mm/slab.c:2329
invalid opcode: 0000 [3] 
CPU 0 
Modules linked in: xt_state ip_queue ip_conntrack iptable_filter
ip_tables Pid: 21233, comm: bash Not tainted 2.6.16-rc5-g7b14e3b5 #7
RIP: 0010:[<ffffffff801577ca>] <ffffffff801577ca>{free_block+257}
RSP: 0018:ffff810000dc5c78  EFLAGS: 00010092
RAX: 0000000000000049 RBX: ffff81000511c000 RCX: ffffffff803f2490
RDX: ffffffff803f2490 RSI: 0000000000000001 RDI: 0000000100000000
RBP: ffff81001ff2bf20 R08: 000000003b9aca00 R09: 000000000000000f
R10: 0000000000000000 R11: ffff81001ff2bf20 R12: ffff81001ff2cec0
R13: ffff81000511ca38 R14: 0000000000000037 R15: ffff81000511c030
FS:  00002b9e31461e60(0000) GS:ffffffff804cb000(0000)
knlGS:00000000f77378e0 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000000000071359f CR3: 0000000000614000 CR4: 00000000000006e0
Process bash (pid: 21233, threadinfo ffff810000dc4000, task
ffff81001d018e60) Stack: ffff81001ff280a8 0000003700615065
0000000000000010 0000000000000010 ffff81001ff28098 ffff81001ff2bf20
ffff81001ff2cec0 0000000000000000 ffff810000dc5d70 ffffffff8015796c 
Call Trace: <ffffffff8015796c>{cache_flusharray+111}
       <ffffffff80157585>{kmem_cache_free+78}
<ffffffff80148cac>{free_pgtables+45} <ffffffff8014e1fc>{exit_mmap+119}
<ffffffff8012210c>{mmput+27} <ffffffff801263d6>{do_exit+519}
<ffffffff801269d7>{sys_exit_group+0}
<ffffffff8012deb3>{get_signal_to_deliver+1216}
<ffffffff80109aaa>{do_signal+110} <ffffffff8012d591>{kill_proc_info+41}
<ffffffff8012e601>{sys_kill+263}
<ffffffff8010a476>{sys_rt_sigreturn+654}
<ffffffff8010a82e>{int_signal+18}

Code: 0f 0b 68 4f e6 38 80 c2 19 09 8b 43 24 48 89 de 4c 89 e7 43 
RIP <ffffffff801577ca>{free_block+257} RSP <ffff810000dc5c78>
 <1>Fixing recursive fault but reboot is needed!
Unable to handle kernel paging request at 0000000000100108 RIP: 
<ffffffff80157748>{free_block+127}
PGD 0 
Oops: 0002 [4] 
CPU 0 
Modules linked in: xt_state ip_queue ip_conntrack iptable_filter
ip_tables Pid: 18669, comm: konsole Not tainted 2.6.16-rc5-g7b14e3b5 #7
RIP: 0010:[<ffffffff80157748>] <ffffffff80157748>{free_block+127}
RSP: 0018:ffff81001d60bde8  EFLAGS: 00010012
RAX: 0000000000100100 RBX: ffff81000511c000 RCX: ffff810001000000
RDX: 0000000000200200 RSI: ffff81000511c000 RDI: ffff81001ff2cec0
RBP: ffff81001ff2bf20 R08: ffff81001ff2eae8 R09: ffff8100169f1000
R10: 0000000000000212 R11: 00000000000000aa R12: ffff81001ff2cec0
R13: ffff81000511c5b0 R14: 0000000000000006 R15: ffff8100098ef030
FS:  00002b6843b30ce0(0000) GS:ffffffff804cb000(0000)
knlGS:00000000f77378e0 CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000100108 CR3: 000000000f55a000 CR4: 00000000000006e0
Process konsole (pid: 18669, threadinfo ffff81001d60a000, task
ffff81001e17b750) Stack: ffff81001ff28100 000000060513e000
0000000000000005 0000000000000010 ffff81001ff28098 ffff81001ff2bf20
ffff81001ff2cec0 0000000000000000 ffff81001d60bee0 ffffffff8015796c 
Call Trace: <ffffffff8015796c>{cache_flusharray+111}
       <ffffffff80157585>{kmem_cache_free+78}
<ffffffff80148cac>{free_pgtables+45} <ffffffff8014e1fc>{exit_mmap+119}
<ffffffff8012210c>{mmput+27} <ffffffff801263d6>{do_exit+519}
<ffffffff801269d7>{sys_exit_group+0} <ffffffff8010a5b2>{system_call+126}

Code: 48 89 50 08 48 89 02 48 c7 43 08 00 02 20 00 48 c7 03 00 01 
RIP <ffffffff80157748>{free_block+127} RSP <ffff81001d60bde8>
CR2: 0000000000100108
 <1>Fixing recursive fault but reboot is needed!
slab error in cache_alloc_debugcheck_after(): cache `anon_vma': double
free, or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014af5f>{__handle_mm_fault+899}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff801ddede>{prio_tree_insert+432}
<ffffffff801566a0>{check_poison_obj+48}
<ffffffff8010aee1>{error_exit+0} <ffffffff801e1896>{__clear_user+60}
<ffffffff801e187a>{__clear_user+32} <ffffffff80182b0d>{padzero+24}
<ffffffff80183b4c>{load_elf_binary+2866}
<ffffffff80163dd4>{search_binary_handler+110}
<ffffffff801640c7>{do_execve+375} <ffffffff8010a5b2>{system_call+126}
<ffffffff8010929c>{sys_execve+51} <ffffffff8010a9d6>{stub_execve+106}
ffff8100098efb78: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014af5f>{__handle_mm_fault+899}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff80141857>{generic_file_aio_read+52}
<ffffffff801ddede>{prio_tree_insert+432}
<ffffffff8010aee1>{error_exit+0} <ffffffff801e1896>{__clear_user+60}
<ffffffff801e187a>{__clear_user+32} <ffffffff80182b0d>{padzero+24}
<ffffffff80182f2b>{load_elf_interp+803}
<ffffffff80183d87>{load_elf_binary+3437}
<ffffffff80163dd4>{search_binary_handler+110}
<ffffffff801640c7>{do_execve+375} <ffffffff8010a5b2>{system_call+126}
<ffffffff8010929c>{sys_execve+51} <ffffffff8010a9d6>{stub_execve+106}
ffff8100098ef970: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014ad64>{__handle_mm_fault+392}
<ffffffff801566a0>{check_poison_obj+48}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff8014dc68>{do_mmap_pgoff+1507} <ffffffff8010aee1>{error_exit+0}
ffff8100098ef3d0: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014af5f>{__handle_mm_fault+899}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff8014dc68>{do_mmap_pgoff+1507} <ffffffff8010aee1>{error_exit+0}
ffff8100098ef920: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014ad64>{__handle_mm_fault+392}
<ffffffff801566a0>{check_poison_obj+48}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff8014dc68>{do_mmap_pgoff+1507} <ffffffff8010aee1>{error_exit+0}
ffff8100098ef808: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014ad64>{__handle_mm_fault+392}
<ffffffff80117ee8>{do_page_fault+937} <ffffffff8014c436>{remove_vma+90}
<ffffffff8014d66a>{do_munmap+601} <ffffffff8010aee1>{error_exit+0}
ffff8100098eff10: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014ad64>{__handle_mm_fault+392}
<ffffffff80117ee8>{do_page_fault+937} <ffffffff8016d5f8>{dput+59}
<ffffffff8015b791>{__fput+302} <ffffffff8010aee1>{error_exit+0}
ffff8100098ef718: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014ad64>{__handle_mm_fault+392}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff8014dc68>{do_mmap_pgoff+1507} <ffffffff8010aee1>{error_exit+0}
ffff8100098ef740: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014af5f>{__handle_mm_fault+899}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff801ddede>{prio_tree_insert+432}
<ffffffff801566a0>{check_poison_obj+48}
<ffffffff8010aee1>{error_exit+0} <ffffffff801e1896>{__clear_user+60}
<ffffffff801e187a>{__clear_user+32} <ffffffff80182b0d>{padzero+24}
<ffffffff80183b4c>{load_elf_binary+2866}
<ffffffff80163dd4>{search_binary_handler+110}
<ffffffff801640c7>{do_execve+375} <ffffffff8010a5b2>{system_call+126}
<ffffffff8010929c>{sys_execve+51} <ffffffff8010a9d6>{stub_execve+106}
ffff8100098ef858: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014af5f>{__handle_mm_fault+899}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff80141857>{generic_file_aio_read+52}
<ffffffff801ddede>{prio_tree_insert+432}
<ffffffff8010aee1>{error_exit+0} <ffffffff801e1896>{__clear_user+60}
<ffffffff801e187a>{__clear_user+32} <ffffffff80182b0d>{padzero+24}
<ffffffff80182f2b>{load_elf_interp+803}
<ffffffff80183d87>{load_elf_binary+3437}
<ffffffff80163dd4>{search_binary_handler+110}
<ffffffff801640c7>{do_execve+375} <ffffffff8010a5b2>{system_call+126}
<ffffffff8010929c>{sys_execve+51} <ffffffff8010a9d6>{stub_execve+106}
ffff8100098ef768: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5. slab
error in cache_alloc_debugcheck_after(): cache `anon_vma': double free,
or memory outside object was overwritten

Call Trace: <ffffffff80156d06>{cache_alloc_debugcheck_after+153}
       <ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff80156e27>{kmem_cache_alloc+140}
<ffffffff8014f939>{anon_vma_prepare+73}
<ffffffff8014ad64>{__handle_mm_fault+392}
<ffffffff801566a0>{check_poison_obj+48}
<ffffffff80117ee8>{do_page_fault+937}
<ffffffff8014dc68>{do_mmap_pgoff+1507} <ffffffff8010aee1>{error_exit+0}
ffff8100098ef290: redzone 1: 0x170fc2a5, redzone 2: 0x170fc2a5.


I'm going to reboot ;)

-- 
	Paolo Ornati
	Linux 2.6.16-rc5-g7b14e3b5 on x86_64

Attachment: DMESG.gz
Description: GNU Zip compressed data

Attachment: config.gz
Description: GNU Zip compressed data