_cpu_down deadlock [was Re: 2.6.19-rc1-mm1]

[Neil Brown <neilb@suse.de>]

On Wednesday October 11, arjan@infradead.org wrote:
> 
> > > blocking_notifier_call_chain is
> > >         down_read(&nh->rwsem);
> > >         ret = notifier_call_chain(&nh->head, val, v);
> > >         up_read(&nh->rwsem);
> > >
> > > and so holds ->rwsem while calling the callback.
> > > So the locking sequence ends up as:
> > >
> > >  down_read(&cpu_chain.rwsem);
> > >  mutex_lock(&workqueue_mutex);
> > >  up_read(&cpu_chain.rwsem);
> > >
> > >  down_read(&cpu_chain.rwsem);
> > >  mutex_unlock(&workqueue_mutex);
> > >  up_read(&workqueue_mutex);
> > >
> > > and lockdep doesn't seem to like this.  It sees workqueue_mutex
> > > claimed while cpu_chain.rwsem is held. and then it sees
> > > cpu_chain.rwsem claimed while workqueue_mutex is held, which looks a
> > > bit like a class ABBA deadlock.
> > > Of course because it is a 'down_read' rather than a 'down', it isn't
> > > really a dead lock.
> 
> ok can you explain to me why "down_read" doesn't make this a deadlock
> while "down" would make it a deadlock? I have trouble following your
> reasoning.....
> 
> (remember that rwsems are strictly fair)

I see your point.

While thread A holds just workqueue_mutex,
thread B takes cpu_chain.rwsem for read then tries to take
workqueue_mutex and blocks.
Now thread C tries to get a write lock on cpu_chain.rwsem and blocks
as well.
Finally thread A moves on to try to get a read lock on cpu_chain.rwsem
and this blocks because thread C is waiting for a write lock.

So A waits on B and C, C waits on B, B waits on A.
Deadlock.

I guess _cpu_down should
	down_read(&cpu_chain.rwsem);
and then call notifier_call_chain multiple times.  I wonder if that
would be safe.

Who do we blame this on?  Are you still the cpu-hot-plug guy Rusty?

NeilBrown
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/