Re: [2.6.13-rc6-git13/sparc64]: Slab corruption (possible stack or buffer-cache corruption)

[Tomasz Kłoczko <kloczek@rudy.mif.pg.gda.pl>]

On Tue, 13 Sep 2005, David S. Miller wrote:

> From: Tomasz Kłoczko <kloczek@rudy.mif.pg.gda.pl>
> Date: Tue, 13 Sep 2005 12:12:22 +0200 (CEST)
>
> > # grep "^Sep 12" /var/log/messages | grep kernel: | uniq | cut -d " " -f 6- | sort | uniq -c | sort -n | tail -n 2
> >      509 svc: bad direction 268435456, dropping request
> >      653 eth0: Happy Meal out of receive descriptors, packet dropped.
> >
> > As you see one of this two messagess occures avarange one time per ~two
> > minutes.
> > Second looks like some error in sunhme.c. eth0 it is:
>
> It's not a bug, per se, your system is simply receiving more
> network traffic than the kernel can receive.
> So the network adapter runs out of receive descriptors in which to receive
> new packets, and starts dropping them until the kernel catches
> up again.  That's what that message means.

Dave you touch core of problems .. messages occures constantly but 
high network activity occures only at night on this interface (backup). In 
attachment is weekly graph network acivity measured on SNMP agent on 
switch on this port.
At day usualaly on this interface activity is is very low (5-40Kb/s 
in/out).

> The "svc: " one I've seen before, it looks like something is
> clobbering the sunrpc message, for example a freed up buffer is having
> some bit set in one of it's words.  This 268435456 value is
> "0x10000000" hexadecimal.  The code expects the value zero, and we
> have a stray bit being set in there, bit 28 to be exact.  This would
> actually be expected after you trigger something like that
> destroy_inode() bug as a buffer is being free'd up twice.

Good news. After switch to kernel 2.6.13-1.1552sp1 from corona with your 
patch I don't see "svc: .." messages, but "eth0: Happy Meal .." still 
occures (seems less but still).
After ~half of day (and pass one time backup at night):

[root@boss ~]# grep "^Sep 14" /var/log/kernel | grep kernel: | uniq | cut -d " " -f 6- | sort | uniq -c | sort -n | tail -n 3
      2 udp v4 hw csum failure.
     15 eth0: Happy Meal out of receive descriptors, packet dropped.
     22 hw tcp v4 csum failed
[root@boss ~]# LANG= date
Wed Sep 14 13:15:01 CEST 2005

I'll try count of this for compare again at ~12:00 pm.

kloczek
--
-----------------------------------------------------------
*Ludzie nie mają problemów, tylko sobie sami je stwarzają*
-----------------------------------------------------------
Tomasz Kłoczko, sys adm @zie.pg.gda.pl|*e-mail: kloczek@rudy.mif.pg.gda.pl*

Attachment: sunset163_2-week.png
Description: Binary data