Re: [PATCH] remove empty node at boot time

[KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>]

On Mon, 10 Jul 2006 11:03:03 -0600
Bjorn Helgaas <bjorn.helgaas@hp.com> wrote:

> On Sunday 09 July 2006 23:19, KAMEZAWA Hiroyuki wrote:
> > Could you try this patch ? (against 2.6.18-rc1)
> 
> Your patch does fix it.  But I'm worried about removing
> empty nodes at boot-time.  I want to support the following
> scenario:
> 
>   node 0: 1 enabled CPU, 3 disabled CPUs, no local memory
>   node 1: 4 disabled CPUs, no local memory
>   node 2: no CPUs, big interleaved memory across nodes 0 & 1
> 
> At run-time, I'd like to be able to enable any or all of the
> 7 disabled CPUs.  If you remove the "empty" node 1 at boot-time,
> it sounds like I won't be able to enable its CPUs later.
> 

Hmm.. in my understanding, all structures for *possible* cpus are allocated
at boot time. Then, only problem seems that a cpu is tied to 
not-exisiting-node at boot time.
(see arch/ia64/kernel/numa.c, build_cpu_to_node_map())

====
void __init build_cpu_to_node_map(void)
{
        int cpu, i, node;

        for(node=0; node < MAX_NUMNODES; node++)
                cpus_clear(node_to_cpu_mask[node]);

        for(cpu = 0; cpu < NR_CPUS; ++cpu) {
                node = -1;
                for (i = 0; i < NR_CPUS; ++i)
                        if (cpu_physical_id(cpu) == node_cpuid[i].phys_id) {
                                node = node_cpuid[i].nid;
                                break;
                        }
                cpu_to_node_map[cpu] = (node >= 0) ? node : 0; 
                if (node >= 0)
                        cpu_set(cpu, node_to_cpu_mask[node]);
        }
}
=====

Then what we have to do here are
1. remap cpu to the first existing node at hot-add event
or
2. implement node-hot-add triggered by cpu-hot-add.
   Because we already have implemented node-hot-add triggered by memory-hotadd
   we can do it by small effort.

I think above will work for your environment.
do you have any idea other than "don't remove empty node at boot time" ?
or reserve empty node is the best way ?

- Kame

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/