Several apps are crashing almost at the same time!
FC3 2.6.9-1.667 installed from ISO's
This was a stable solid system until yum update to 2.6.10-1.770_FC3
About an hour after booting I get segmentation faults and
illegal instruction crashes from firefox, mozilla, yum, bug-buddy, ..
I used strace to find out how far they get, and saw pages of trace before
the crashes
(the tails of the traces are below)
I have disabled screensavers and power saving mode.
At times, the desktop icons would be gone and there would be no
right-click menu.
Sometimes the desktop would freeze.
Sometimes in this state the machine can still respond to pings.
memtest86 shows no hardware errors, but it might not be testing things
like MMU, disk controller, ..
HP P4 1.6Ghz 512 Mbyte
Might it be a problem with prelink?
Should I revert to the FC3 ISO's?
This may be a kernel bug, and if so
I would like to help track it down.
How would I go about narrowing down the fault?
-- next day --
Now everything is working fine, and I am using the updated kernel. What
happened?
cheers -- Rick rick
ooo at leirtech
ooo dot com
P.S. Below is the output of strace for several apps. Only the last few
lines of each strace is shown.
The output of dmesg follows.
seg.automake
::::::::::::::
_llseek(9, 0, [0], SEEK_CUR) = 0
read(9, "# Generated from XSLoader.pm.PL "..., 4096) = 4096
_llseek(9, 2901, [2901], SEEK_SET) = 0
_llseek(9, 0, [2901], SEEK_CUR) = 0
close(9) = 0
_llseek(8, 412, [412], SEEK_SET) = 0
_llseek(8, 0, [412], SEEK_CUR) = 0
close(8) = 0
stat64("/usr/lib/perl5/5.8.5/i386-linux-thread-multi/auto/IO/IO.so",
{st_mode=S_IFREG|0
555, st_size=16816, ...}) = 0
stat64("/usr/lib/perl5/5.8.5/i386-linux-thread-multi/auto/IO/IO.bs",
0x963b0c8) = -1 EN
OENT (No such file or directory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
seg.bug-buddy
::::::::::::::
rt_sigaction(SIGFPE, {0x5460ec2, [], SA_RESTORER, 0xa278d8}, NULL, 8) = 0
rt_sigaction(SIGBUS, {0x5460ec2, [], SA_RESTORER, 0xa278d8}, NULL, 8) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=39538288, ...}) = 0
mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7d89000
close(3) getresuid32([0], [0], [0]) = 0
getresgid32([0], [0], [0]) = 0
--- SIGILL (Illegal instruction) @ 0 (0) ---
+++ killed by SIGILL +++
seg.evolution
::::::::::::::
rt_sigaction(SIGFPE, {0x5460ec2, [], SA_RESTORER, 0xa278d8}, NULL, 8) = 0
rt_sigaction(SIGBUS, {0x5460ec2, [], SA_RESTORER, 0xa278d8}, NULL, 8) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=39538288, ...}) = 0
mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7d07000
close(3) = 0
getresuid32([0], [0], [0]) = 0
getresgid32([0], [0], [0]) = 0
--- SIGILL (Illegal instruction) @ 0 (0) ---
+++ killed by SIGILL +++
seg.mozilla
::::::::::::::
rt_sigaction(SIGRT_1, {0xa20430, [], SA_RESTORER|SA_RESTART|SA_SIGINFO,
0xa278d0}, NULL
, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
_sysctl({{CTL_KERN, KERN_VERSION}, 2, 0xbfe71018, 31, (nil), 0}) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
seg.nautilus
::::::::::::::
rt_sigaction(SIGBUS, {0x5460ec2, [], SA_RESTORER, 0xa278d8}, NULL, 8) = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=39538288, ...}) = 0
mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7d62000
close(3) = 0
getresuid32([0], [0], [0]) = 0
getresgid32([0], [0], [0]) = 0
--- SIGILL (Illegal instruction) @ 0 (0) ---
+++ killed by SIGILL +++
seg.yum
::::::::::::::
open("/usr/lib/python2.3/lib-dynload/_localemodule.so",
O_RDONLY|O_LARGEFILE) = 5
fstat64(5, {st_mode=S_IFREG|0555, st_size=15852, ...}) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
seg.zenity
::::::::::::::
open("/usr/lib/charset.alias", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such
file or direc
tory)
open("/etc/popt", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or
directory)
open("/root/.popt", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or
directory)
getresuid32([0], [0], [0]) = 0
getresgid32([0], [0], [0]) = 0
--- SIGILL (Illegal instruction) @ 0 (0) ---
+++ killed by SIGILL +++
===========================================
$ m /tmp/dmesg
Linux version 2.6.10-1.770_FC3 (bhcompile@xxxxxxxxxxxxxxxxxxxxxx) (gcc
version 3.4.2 20
041017 (Red Hat 3.4.2-6.fc3)) #1 Thu Feb 24 14:00:06 EST 2005
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000ca000 - 00000000000ce000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000001fff0000 (usable)
BIOS-e820: 000000001fff0000 - 000000001fff8000 (ACPI data)
BIOS-e820: 000000001fff8000 - 0000000020000000 (ACPI NVS)
BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
0MB HIGHMEM available.
511MB LOWMEM available.
Using x86 segment limits to approximate NX protection
On node 0 totalpages: 131056
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 126960 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
DMI 2.3 present.
ACPI: RSDP (v000 AMI ) @ 0x000f9e10
ACPI: RSDT (v001 HP-BDD HPBDD-JA 0x00000010 MSFT 0x00000097) @ 0x1fff0000
ACPI: FADT (v001 HP-BDD HPBDD-JA 0x00000001 MSFT 0x00000097) @ 0x1fff0030
ACPI: MADT (v001 HP-BDD 0x00000009 MSFT 0x00000097) @ 0x1fff00e0
ACPI: BOOT (v001 HP-BDD INTEL845 0x00000009 MSFT 0x00000097) @ 0x1fff00b0
ACPI: DSDT (v001 INTEL BROKDALE 0x00001014 MSFT 0x0100000d) @ 0x00000000
ACPI: PM-Timer IO Port: 0x808
Built 1 zonelists
Kernel command line: ro root=LABEL=/
Initializing CPU#0
CPU 0 irqstacks, hard=c03d3000 soft=c03d2000
PID hash table entries: 2048 (order: 11, 32768 bytes)
Detected 1594.200 MHz processor.
Using pmtmr for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Memory: 515004k/524224k available (2045k kernel code, 8568k reserved, 655k
data, 160k i
nit, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode...
Ok.
Calibrating delay loop... 3153.92 BogoMIPS (lpj=1576960)
Security Framework v1.0.0 initialized
SELinux: Initializing.
SELinux: Starting in permissive mode
selinux_register_security: Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
CPU: After generic identify, caps: 3febfbff 00000000 00000000 00000000
CPU: After vendor identify, caps: 3febfbff 00000000 00000000 00000000
CPU: Trace cache: 12K uops, L1 D cache: 8K
CPU: L2 cache: 256K
CPU: After all inits, caps: 3febf3ff 00000000 00000000 00000080
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (12) available
CPU: Intel(R) Pentium(R) 4 CPU 1.60GHz stepping 02
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
ACPI: setting ELCR to 0200 (from 0e20)
checking if image is initramfs... it is
Freeing initrd memory: 384k freed
NET: Registered protocol family 16
PCI: PCI BIOS revision 2.10 entry at 0xfdab1, last bus=2
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20041105
ACPI: Interpreter enabled
ACPI: Using PIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.ICHB._PRT]
ACPI: Power Resource [URP1] (off)
ACPI: Power Resource [URP2] (off)
ACPI: Power Resource [FDDP] (off)
ACPI: Power Resource [LPTP] (off)
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0,
disabled.
ACPI: PCI Interrupt Link [LNKD] (IRQs 9) *5
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0,
disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0,
disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 *9 10 11 12 14 15)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
pnp: PnP ACPI: found 11 devices
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
** PCI interrupts are no longer routed automatically. If this
** causes a device to stop working, it is probably because the
** driver failed to call pci_enable_device(). As a temporary
** workaround, the "pci=routeirq" argument restores the old
** behavior. If this argument makes the device work again,
** please email the output of "lspci" to bjorn.helgaas@xxxxxx
** so I can fix the driver.
Simple Boot Flag at 0x79 set to 0x1
apm: BIOS version 1.2 Flags 0x03 (Driver version 1.16ac)
apm: overridden by ACPI.
audit: initializing netlink socket (disabled)
audit(1112867241.611:0): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
SELinux: Registering netfilter hooks
Initializing Cryptographic API
ksign: Installing public key data
Loading keyring
- Added public key C1F2FA57F7EECD64
- User ID: Red Hat, Inc. (Kernel Module GPG key)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
ACPI: Processor [CPU1] (supports 8 throttling states)
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Real Time Clock Driver v1.12
Linux agpgart interface v0.100 (c) Dave Jones
agpgart: Detected an Intel i845 Chipset.
agpgart: Maximum main memory to use for agp memory: 439M
agpgart: AGP aperture is 64M @ 0xe0000000
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
Serial: 8250/16550 driver $Revision: 1.90 $ 76 ports, IRQ sharing enabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
divert: not allocating divert_blk for non-ethernet device lo
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
ICH2: IDE controller at PCI slot 0000:00:1f.1
ICH2: chipset revision 18
ICH2: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0xfc00-0xfc07, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0xfc08-0xfc0f, BIOS settings: hdc:DMA, hdd:pio
Probing IDE interface ide0...
hda: ST3120026A, ATA DISK drive
elevator: using anticipatory as default io scheduler
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
hdc: CRD-8482B, ATAPI CD/DVD-ROM drive
hdc: Disabling (U)DMA for CRD-8482B (blacklisted)
ide1 at 0x170-0x177,0x376 on irq 15
hda: max request size: 1024KiB
hda: 234441648 sectors (120034 MB) w/8192KiB Cache, CHS=16383/255/63,
UDMA(100)
hda: cache flushes supported
hda: hda1 hda2 hda3
hdc: ATAPI 48X CD-ROM drive, 128kB Cache
Uniform CD-ROM driver Revision: 3.20
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.0:USB HID core driver
mice: PS/2 mouse device common for all mice
input: AT Translated Set 2 keyboard on isa0060/serio0
input: ImExPS/2 Generic Explorer Mouse on isa0060/serio1
md: md driver 0.90.1 MAX_MD_DEVS=256, MD_SB_DISKS=27
NET: Registered protocol family 2
IP: routing cache hash table of 1024 buckets, 32Kbytes
TCP: Hash tables configured (established 32768 bind 9362)
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
ACPI wakeup devices:
ICHB PS2K UAR1 UAR2 USB1 USB2 SLPB
ACPI: (supports S0 S1 S3 S4 S5)
Freeing unused kernel memory: 160k freed
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: hda2: orphan cleanup on readonly fs
ext3_orphan_cleanup: deleting unreferenced inode 215185
ext3_orphan_cleanup: deleting unreferenced inode 12828684
EXT3-fs: hda2: 2 orphan inodes deleted
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: Disabled at runtime.
SELinux: Unregistering netfilter hooks
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
e100: Intel(R) PRO/100 Network Driver, 3.2.3-k2-NAPI
e100: Copyright(c) 1999-2004 Intel Corporation
ACPI: PCI Interrupt Link [LNKE] enabled at IRQ 10
PCI: setting IRQ 10 as level-triggered
ACPI: PCI interrupt 0000:02:08.0[A] -> GSI 10 (level, low) -> IRQ 10
divert: allocating divert_blk for eth0
e100: eth0: e100_probe: addr 0xdfeff000, irq 10, MAC addr
00:04:23:0B:CF:F3
ip_tables: (C) 2000-2002 Netfilter core team
e100: eth0: e100_watchdog: link up, 10Mbps, half-duplex
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
ACPI: PCI interrupt 0000:00:1f.5[B] -> GSI 10 (level, low) -> IRQ 10
PCI: Setting latency timer of device 0000:00:1f.5 to 64
intel8x0_measure_ac97_clock: measured 49148 usecs
intel8x0: clocking to 48000
hw_random hardware driver 1.0.0 loaded
USB Universal Host Controller Interface driver v2.2
ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 9
PCI: setting IRQ 9 as level-triggered
ACPI: PCI interrupt 0000:00:1f.2[D] -> GSI 9 (level, low) -> IRQ 9
uhci_hcd 0000:00:1f.2: UHCI Host Controller
PCI: Setting latency timer of device 0000:00:1f.2 to 64
uhci_hcd 0000:00:1f.2: irq 9, io base 0xd000
uhci_hcd 0000:00:1f.2: new USB bus registered, assigned bus number 1
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
ACPI: PCI Interrupt Link [LNKH] enabled at IRQ 9
ACPI: PCI interrupt 0000:00:1f.4[C] -> GSI 9 (level, low) -> IRQ 9
uhci_hcd 0000:00:1f.4: UHCI Host Controller
PCI: Setting latency timer of device 0000:00:1f.4 to 64
uhci_hcd 0000:00:1f.4: irq 9, io base 0xd400
uhci_hcd 0000:00:1f.4: new USB bus registered, assigned bus number 2
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
ACPI: Power Button (FF) [PWRF]
ACPI: Sleep Button (CM) [SLPB]
ibm_acpi: ec object not found
EXT3 FS on hda2, internal journal
device-mapper: 4.3.0-ioctl (2004-09-30) initialised: dm-devel@xxxxxxxxxx
kjournald starting. Commit interval 5 seconds
EXT3 FS on hda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Adding 257032k swap on /dev/hda3. Priority:-1 extents:1
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378 (0x778), irq 7 [PCSPP,TRISTATE]
Trying to free free DMA3
pnp: Device 00:0a disabled.
ieee1394: Initialized config rom entry `ip1394'
ip_tables: (C) 2000-2002 Netfilter core team
ip_tables: (C) 2000-2002 Netfilter core team
Bluetooth: Core ver 2.7
NET: Registered protocol family 31
Bluetooth: HCI device and connection manager initialized
Bluetooth: HCI socket layer initialized
Bluetooth: L2CAP ver 2.6
Bluetooth: L2CAP socket layer initialized
Bluetooth: RFCOMM ver 1.3
Bluetooth: RFCOMM socket layer initialized
Bluetooth: RFCOMM TTY layer initialized
pnp: Device 00:0a activated.
parport: PnPBIOS parport detected.
parport0: PC-style at 0x378 (0x778), irq 7 [PCSPP,TRISTATE]
lp0: using parport0 (interrupt-driven).
lp0: console ready
NET: Registered protocol family 10
Disabled Privacy Extensions on device c03697c0(lo)
IPv6 over IPv4 tunneling driver
divert: not allocating divert_blk for non-ethernet device sit0
eth0: no IPv6 routers present
ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 11
PCI: setting IRQ 11 as level-triggered
ACPI: PCI interrupt 0000:01:00.0[A] -> GSI 11 (level, low) -> IRQ 11
[drm] Initialized r128 2.5.0 20030725 on minor 0:
agpgart: Found an AGP 2.0 compliant device at 0000:00:00.0.
agpgart: Putting AGP V2 device at 0000:00:00.0 into 1x mode
agpgart: Putting AGP V2 device at 0000:01:00.0 into 1x mode
application chromium uses obsolete OSS audio interface
