* Pekka Enberg <[email protected]> wrote:
> I mostly live in the legacy 32-bit UMA/UP land still so I cannot
> verify this myself but the kind folks at SGI claim the following
> (again from the announcement):
>
> "On our systems with 1k nodes / processors we have several gigabytes
> just tied up for storing references to objects for those queues This
> does not include the objects that could be on those queues. One fears
> that the whole memory of the machine could one day be consumed by
> those queues."
Yes, you can find gigs tied up on systems that have 100 GB of RAM, or
you can have gigs tied up if you over-size your caches. I'd like to see
an accurate calculation done on this.
> The problem is that for each cache, you have an "per-node alien
> queues" for each node (see struct kmem_cache nodelists -> struct
> kmem_list3 alien). Moving slab metadata to struct page solves this but
> now you can only have one "queue" thats part of the same struct.
yes, it's what i referred to as "distributed, per node cache". It has no
"quadratic overhead". It has SLAB memory spread out amongst nodes. I.e.
1 million pages are distributed amongst 1k nodes with 1000 pages per
node with each node having 1 page.
But that memory is not lost and it's disingenous to call it 'overhead'
and it very much comes handy for performance _IF_ there's global
workload that involves cross-node allocations. It's simply a cache that
is mis-sized and mis-constructed on large node count systems but i bet
it makes quite a performance difference on low-node-count systems.
On high node-count systems it might make sense to reduce the amount of
cross-node caching and to _structure_ the distributed NUMA SLAB cache in
an intelligent way (perhaps along cpuset boundaries) - but a total,
design level _elimination_ of this caching concept, using very
misleading arguments, just looks stupid to me ...
Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
- References:
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
- Re: tipc_init(), WARNING: at arch/x86/mm/highmem_32.c:52, [2.6.24-rc4-git5: Reported regressions from 2.6.23]
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
