Re: Case: 7454422: Re: Kernel 2.6.21.3 does not work with 8GB of RAM on Intel 965WH motherboards. (FULL DMESG)

[Justin Piszcz <jpiszcz@lucidpixels.com>]

On Fri, 1 Jun 2007, Justin Piszcz wrote:

> On Fri, 1 Jun 2007, Justin Piszcz wrote:
>
> > On Wed, 30 May 2007, Robert Hancock wrote:
> >
> > > Parag Warudkar wrote:
> > > > Robert Hancock wrote:
> > > >
> > > >
> > > > > 0-3319MB
> > > > > 4096-8832MB
> > > > >
> > > > > leaving 64MB of memory at the top of RAM uncached. What do you want to
> > > > > bet that something important (kernel code?) is getting loaded there..
> > > > >
> > > > > So essentially it's a BIOS problem, it's not setting up the MTRRs
> > > > > properly in order to map all of RAM as cacheable. As Andi says, complain
> > > > > to Intel.
> > > >
> > > > Could the BADRAM patch be useful for him?
> > > > http://rick.vanrein.org/linux/badram/download.html has 2.6.21 version.
> > > > It says it supports x86_64. May be using this patch he can exclude
> > > > that RAM from being used/accessed?
> > >
> > > I think that mem=8832M would work as well, to make the kernel use only the 
> > > memory that is marked cacheable. (It looks like this parameter takes the 
> > > highest memory address we want the kernel to use, not the highest memory 
> > > amount.)
> > >
> > > --
> > > Robert Hancock      Saskatoon, SK, Canada
> > > To email, remove "nospam" from hancockr@nospamshaw.ca
> > > Home Page: http://www.roberthancock.com/
> >
> > Is 8832MB a typo?  8GB of memory is ~8192MB right?  Did you mean 8132MB or? 
> > Intel wants me to flash my bios and reset everything to the defaults to see 
> > if it is still an issue, I'd prefer to try the mem= option first.
> >
> > Justin.
>
> mem=8064M" [top: Mem:   7264144k total]
> mem=8832M" [top: Mem:   8039820k total]
>
> I am using 8832MB and it does not have the bug/slowness!  How did you 
> calculate that from the MTRR output?
>
> $ cat /proc/mtrr
> reg00: base=0x00000000 (   0MB), size=2048MB: write-back, count=1
> reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
> reg02: base=0xc0000000 (3072MB), size= 256MB: write-back, count=1
> reg03: base=0xcf800000 (3320MB), size=   8MB: uncachable, count=1
> reg04: base=0xcf700000 (3319MB), size=   1MB: uncachable, count=1
> reg05: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
> reg06: base=0x200000000 (8192MB), size= 512MB: write-back, count=1
> reg07: base=0x220000000 (8704MB), size= 128MB: write-back, count=1
>
> It sees 7.66GB now!
>
> 8039820 / 1024 / 1024
> 7.66GB
>
> top - 13:56:48 up 3 min,  5 users,  load average: 0.12, 0.14, 0.06
> Tasks: 163 total,   1 running, 162 sleeping,   0 stopped,   0 zombie
> Cpu0  :  4.4%us,  1.4%sy,  0.2%ni, 89.7%id,  4.1%wa,  0.0%hi,  0.1%si, 
> 0.0%st
> Cpu1  :  0.1%us,  0.5%sy,  0.6%ni, 98.4%id,  0.3%wa,  0.0%hi,  0.0%si, 
> 0.0%st
> Cpu2  :  1.2%us,  0.3%sy,  0.1%ni, 97.9%id,  0.5%wa,  0.0%hi,  0.0%si, 
> 0.0%st
> Cpu3  :  0.2%us,  1.9%sy,  0.7%ni, 96.0%id,  1.1%wa,  0.0%hi,  0.0%si, 
> 0.0%st
> Mem:   8039820k total,  1039588k used,  7000232k free,     3552k buffers
> Swap: 16787768k total,        0k used, 16787768k free,   141480k cached

Ahh here we go:

Justin Piszcz wrote:
> That output looked nasty, attaching entries from syslog.
>
> Justin.

Here's your E820 memory map, from dmesg:

BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
BIOS-e820: 000000000008f000 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000cf58f000 (usable)
BIOS-e820: 00000000cf58f000 - 00000000cf59c000 (reserved)
BIOS-e820: 00000000cf59c000 - 00000000cf653000 (usable)
BIOS-e820: 00000000cf653000 - 00000000cf6a5000 (ACPI NVS)
BIOS-e820: 00000000cf6a5000 - 00000000cf6a8000 (ACPI data)
BIOS-e820: 00000000cf6a8000 - 00000000cf6ef000 (ACPI NVS)
BIOS-e820: 00000000cf6ef000 - 00000000cf6f1000 (ACPI data)
BIOS-e820: 00000000cf6f1000 - 00000000cf6f2000 (usable)
BIOS-e820: 00000000cf6f2000 - 00000000cf6ff000 (ACPI data)
BIOS-e820: 00000000cf6ff000 - 00000000cf700000 (usable)
BIOS-e820: 00000000cf700000 - 00000000d0000000 (reserved)
BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 000000022c000000 (usable)

so the usable memory ranges are:

0-572K
1MB-3317.55MB
3317.60MB-3317.75MB
3318.94MB-3318.945MB
3318.996MB-3319MB
4096MB-8896MB

and the MTRRs (from /proc/mtrr, from private email):

reg00: base=0x00000000 (   0MB), size=2048MB: write-back, count=1
reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
reg02: base=0xc0000000 (3072MB), size= 256MB: write-back, count=1
reg03: base=0xcf800000 (3320MB), size=   8MB: uncachable, count=1
reg04: base=0xcf700000 (3319MB), size=   1MB: uncachable, count=1
reg05: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg06: base=0x200000000 (8192MB), size= 512MB: write-back, count=1
reg07: base=0x220000000 (8704MB), size= 128MB: write-back, count=1

so the ranges mapped as cacheable are:

0-3319MB
4096-8832MB

leaving 64MB of memory at the top of RAM uncached. What do you want to
bet that something important (kernel code?) is getting loaded there..

So essentially it's a BIOS problem, it's not setting up the MTRRs
properly in order to map all of RAM as cacheable. As Andi says, complain
to Intel.

--
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/