Re: smp/up alternatives crash when CONFIG_HOTPLUG_CPU

[Linus Torvalds <torvalds@osdl.org>]

On Thu, 20 Apr 2006, Ingo Molnar wrote:
> 
> but ... a more fundamental question is, where does the SMP-alternatives 
> code flush the icache? I dont think it's generally guaranteed on x86 
> CPUs that MESI updates to code get propagated into the icache of other 
> CPUs/cores.

It's guaranteed that the _caches_ get updated, but what isn't guaranteed 
is (a) when they do, in the absense of some synchronization primitive, 
anbd (b) that the prefetch queues do.

So you cannot rewrite code that some other CPU is executing, without some 
kind of locking (to make sure that it's not executing right then and 
there) and without having the other CPU do a serializing instruction 
(which the lock will normally generate, of course).

Now for the smp/up thing, we _should_ be safe, because the rewriting from 
UP back to SMP versions should happen before the other CPU even comes 
online (since we're the one bringing it up). And when we have more CPU's 
(and another CPU may be running), we've obviously already done it.

There must be another bug there somewhere. It's not about cache coherency.

(If it was, it would have more serious problems in the bootup sequence: 
think of just the trampoline that it generates to boot the second CPU, 
which is also written just before we actually tell the second CPU to boot 
up).

I can see two potential bugs:

 - we will do the alternatives_smp_switch(1) for each CPU that comes up, 
   even though we should do it only when we bring up the _first_ CPU 
   (which is obviously the second one - we never bring up the boot CPU).

   This should be ok, though. The subsequent calls should replace the code 
   with itself. I don't like writing to code even if it's writing with the 
   same contents, though.

   And this is not the issue you have, if you just have two cores (you 
   won't be bringing up a third CPU ;)

 - we simply save the smp info incorrectly, and don't bring it back 
   properly.

But cache coherency problems I don't believe in.

		Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/