On Thu, 25 Oct 2007 14:58:10 -0700, "H. Peter Anvin" <[email protected]> wrote:
> J.A. Magallon wrote:
> > Hi...
> >
> > I have some Quad-Opteron boxes with 4Gb memory and two of them are
> > running two different Linux distros.
> >
> > Box one sees 4Gb of memory, but box two just sees 3.
> > Their mtrr setups are different:
> >
> > Why ? Is it a bios setup problem ? A kernel problem ?
> > grep HIGHMEN in configs for both kernels does not give anything, so
> > I still understand less this thing...
> >
>
> It would depend on how the BIOS programmed the memory controllers. For
> 32-bit (and lots of device) compatibility, a memory hole is required
> below 4 GB. Not all memory controllers can remap memory in the 3-4 GB
> range above the 4 GB memory; I'm not sure if that varies with the
> different Opteron processors.
I have collected several pieces of info around the internet...
- Some people uses this options in the BIOS:
Node interleave: off
Bank interleave: auto
SW memory hole: disable
HW memory hole: enable
MTRR: Continuous
- Node Memory Interleaving DISABLES NUMA and generally is a bad thing
- MTRR setting -should be set to "discrete" for Linux, and probably for Windows too.
- This is what SuperMicro's tech support said about 2.96GB vs. 4GB.
"This is as expected, as soon as you set "software memory hole" to disabled,
you also disable option ROM remapping functionality, this option normally
remaps used option rom (option rom= raid bios, lan pxe ; usb legacy, bioses
on add-on cards, etc) in the 4GB region, so no basis memory is lost, while
this feature is now disabled the option rom space occupies the space between
3 and 4 GB which results in lower main memory availability.
There is no solution or work around for this phenomenon"
so software memory hole enabled might be needed to get all 4GB to show up
>From mobo manual:
Software Memory Hole
When "Enabled", allows software memory remapping around the memory
hole. Options are Enabled and Disabled.
Hardware Memory Hole
When "Enabled", allows software memory remapping around the memory
hole. Options are Enabled and Disabled. Note: this is only supported by
Rev E0 processors and above.
( I have two Opteron 275 processors, no idea about revision)
So _my_ conclussion is:
Node interleave: off (numa mode)
Bank interleave: auto
SW memory hole: disable |
HW memory hole: enable | allow remapping
MTRR: Discrete |
But then, do I need to enable NUMA options in the kernel ?
>
> Also, if you run a 32-bit distribution, you need to have HIGHMEM_64G
> enabled in the kernel.
>
I run a 64 bit one, then I don't need anything, isn't it ? That's why I
don't see any _HIGHMEM in the kernel configs...
Some day I will understand this crappy BIOS thing (or burn a photo of its
inventor...).
Why can't we have OpenFirmware PC's, like my MacPro and Sparcs ?
--
J.A. Magallon <jamagallon()ono!com> \ Software is like sex:
\ It's better when it's free
Mandriva Linux release 2008.1 (Cooker) for i586
Linux 2.6.23-jam01 (gcc 4.2.2 20070909 (4.2.2-0.RC.1mdv2008.0)) SMP PREEMPT
09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
