[BUGFIX] [PATCH] [IA64] node hotplug : fixup cpu-to-node relationship

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I tested node hoplug with NUMA machine whcih equips physical node-hotplug.
And found panic (see below). This is bacause node_to_cpu_mask is not updated to valid
value. patch is attached.
==
Sep 21 14:15:26 drpq kernel:  [<a00000010000c700>] __ia64_leave_kernel+0x0/0x280
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d426d80 bsp=e00001401d421618
Sep 21 14:15:26 drpq kernel:  [<a000000100062040>] sd_degenerate+0x80/0xe0
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d426f50 bsp=e00001401d4215e0
Sep 21 14:15:26 drpq kernel:  [<a000000100065710>] cpu_attach_domain+0x90/0x1e0
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d426f50 bsp=e00001401d421598
Sep 21 14:15:26 drpq kernel:  [<a00000010006da20>] build_sched_domains+0x14e0/0x2360
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d426f50 bsp=e00001401d4214c0
Sep 21 14:15:26 drpq kernel:  [<a00000010006e8f0>] arch_init_sched_domains+0x50/0x80
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d427da0 bsp=e00001401d4214a0
Sep 21 14:15:26 drpq kernel:  [<a00000010006e9c0>] update_sched_domains+0xa0/0xe0
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d421478
Sep 21 14:15:26 drpq kernel:  [<a0000001006201b0>] notifier_call_chain+0x50/0xc0
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d421440
Sep 21 14:15:26 drpq kernel:  [<a00000010009ede0>] blocking_notifier_call_chain+0x40/0x80
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d421408
Sep 21 14:15:26 drpq kernel:  [<a0000001000bccb0>] cpu_up+0x290/0x300
Sep 21 14:15:26 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d4213c8
Sep 21 14:15:27 drpq kernel:  [<a0000001003d2b90>] store_online+0x70/0xe0
Sep 21 14:15:27 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d421398
Sep 21 14:15:27 drpq kernel:  [<a0000001003c9e20>] sysdev_store+0x60/0xa0
Sep 21 14:15:27 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d421360
Sep 21 14:15:27 drpq kernel:  [<a0000001001edfa0>] sysfs_write_file+0x240/0x2e0
Sep 21 14:15:27 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d421310
Sep 21 14:15:27 drpq kernel:  [<a000000100156ae0>] vfs_write+0x200/0x3a0
Sep 21 14:15:27 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d4212c0
Sep 21 14:15:27 drpq kernel:  [<a000000100157630>] sys_write+0x70/0xe0
Sep 21 14:15:27 drpq kernel:                                 sp=e00001401d427e20 bsp=e00001401d421248
==

-Kame
==
When a cpu is not tied to its node at(before) cpu-onlining, the system panics.
node_to_cpu_mask[] should be set to valid value before notifier of CPU_ONLINE
is called.(if not, the system panics.)

It can happen when a cpu is physically offlined at boot time (ia64).
(*)See arch/ia64/kernel/numa.c. To bind a cpu to a node, we need physical_id
   of a cpu. But smp_build_cpu_map() in smpboot.c will not set physical_id of
   *not present* cpus.

This patch updates node_to_cpumask() and cpu_to_nid_map[] just after
onlining and offlining a cpu.

Also fixes pxm_to_nid usage. acpi_map_pxm_to_nid() should be called here.

Tested on a NUMA machine which supports hardware-node-hot-add.

Signed-Off-By: KAMEZAWA Hiroyuki <[email protected]>

 arch/ia64/kernel/acpi.c    |    5 +++--
 arch/ia64/kernel/smpboot.c |   18 ++++++++++++++++++
 2 files changed, 21 insertions(+), 2 deletions(-)

Index: linux-2.6.18/arch/ia64/kernel/smpboot.c
===================================================================
--- linux-2.6.18.orig/arch/ia64/kernel/smpboot.c	2006-09-20 12:42:06.000000000 +0900
+++ linux-2.6.18/arch/ia64/kernel/smpboot.c	2006-09-22 09:33:55.000000000 +0900
@@ -59,6 +59,7 @@
 #include <asm/system.h>
 #include <asm/tlbflush.h>
 #include <asm/unistd.h>
+#include <asm/topology.h>
 
 #define SMP_DEBUG 0
 
@@ -377,6 +378,7 @@
 	int cpuid, phys_id, itc_master;
 	extern void ia64_init_itm(void);
 	extern volatile int time_keeper_id;
+	int nid;
 
 #ifdef CONFIG_PERFMON
 	extern void pfm_init_percpu(void);
@@ -396,6 +398,17 @@
 
 	lock_ipi_calllock();
 	cpu_set(cpuid, cpu_online_map);
+#ifdef CONFIG_NUMA
+	nid = node_cpuid[cpuid].nid;
+	/* this can be removed when node-hptplug by cpu-hot-add is implemented */
+	if (!node_online(nid))
+		nid = 0;
+	/* see also build_cpu_to_node_map() in numa.c */
+	if (!cpu_isset(cpuid, node_to_cpumask(nid))) {
+		cpu_set(cpuid, node_to_cpumask(nid));
+		cpu_to_node_map[cpuid] = nid;
+	}
+#endif
 	unlock_ipi_calllock();
 	per_cpu(cpu_state, cpuid) = CPU_ONLINE;
 
@@ -701,6 +714,7 @@
 int __cpu_disable(void)
 {
 	int cpu = smp_processor_id();
+	int nid = cpu_to_node(cpu);
 
 	/*
 	 * dont permit boot processor for now
@@ -719,6 +733,10 @@
 
 	remove_siblinginfo(cpu);
 	cpu_clear(cpu, cpu_online_map);
+#ifdef CONFIG_NUMA
+	cpu_clear(cpu, node_to_cpumask(nid));
+	cpu_to_node_map[cpu] = 0;
+#endif
 	fixup_irqs();
 	local_flush_tlb_all();
 	cpu_clear(cpu, cpu_callin_map);
Index: linux-2.6.18/arch/ia64/kernel/acpi.c
===================================================================
--- linux-2.6.18.orig/arch/ia64/kernel/acpi.c	2006-09-20 12:42:06.000000000 +0900
+++ linux-2.6.18/arch/ia64/kernel/acpi.c	2006-09-22 09:28:00.000000000 +0900
@@ -771,14 +771,15 @@
 {
 #ifdef CONFIG_ACPI_NUMA
 	int pxm_id;
+	int nid;
 
 	pxm_id = acpi_get_pxm(handle);
-
+	nid = acpi_map_pxm_to_node(pxm_id);
 	/*
 	 * Assuming that the container driver would have set the proximity
 	 * domain and would have initialized pxm_to_node(pxm_id) && pxm_flag
 	 */
-	node_cpuid[cpu].nid = (pxm_id < 0) ? 0 : pxm_to_node(pxm_id);
+	node_cpuid[cpu].nid = nid;
 
 	node_cpuid[cpu].phys_id = physid;
 #endif

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Stuff]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]     [Linux Resources]
  Powered by Linux