status of: tasklet_unlock_wait() causes soft lockup with -rt and ieee1394 audio

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dear all,

What is the status with respect to this problem? I see that in the current -rt patch the problematic code piece is different. I personally haven't tried to reproduce this myself on a more recent kernel, but I just got a report from one of our users who experienced the same problem with 2.6.19-rt15 and RT preemption (desktop preemption works fine).

Should the latest -rt patches be fixed with respect to this issue? If so I'll try and test them, otherwise I omit the effort.

Thanks,

Pieter

Lee Revell wrote:
Pieter has found this bug in -rt:

We are experiencing 'soft' deadlocks when running our code (Freebob, i.e. userspace lib for firewire audio) on RT kernels. After a few seconds of system freeze, I get a kernel panic message that signals a soft lockup.

I've uploaded the photo's of the panic here:
http://freebob.sourceforge.net/old/img_3378.jpg (without flash)
http://freebob.sourceforge.net/old/img_3377.jpg (with flash)
both are of suboptimal quality unfortunately, but all info is readable on one or the other.

The problems occur when an ISO stream (receive and/or transmit) is shut down in a SCHED_FIFO thread. More precisely when running the freebob jackd backend in real-time mode. And even more precise: they only seem to occur when jackd is shut down. There are no problems when jackd is started without RT scheduling.

I havent been able to reproduce this with other test programs that are shutting down streams in a SCHED_FIFO thread.

The problem is not reproducible on non-RT kernels, and it only occurs on those configured for PREEMPT_RT. If I use PREEMPT_DESKTOP, there is no problem. The PREEMPT_DESKTOP setting was the only change between the two tests, all other kernel settings (threaded irq's etc...) were unchanged.

My tests are performed on 2.6.17-rt1, but the lockups are confirmed for PREEMPT_RT configured kernels 2.6.14 and 2.6.16.

Some extra information:

Lee Revell wrote:

<...>

It seems that the -rt patch changes tasklet_kill:

Unpatched 2.6.17:

void tasklet_kill(struct tasklet_struct *t)
{
        if (in_interrupt())
                printk("Attempt to kill tasklet from interrupt\n");

        while (test_and_set_bit(TASKLET_STATE_SCHED, &t->state)) {
                do
                        yield();
                while (test_bit(TASKLET_STATE_SCHED, &t->state));
        }
        tasklet_unlock_wait(t);
        clear_bit(TASKLET_STATE_SCHED, &t->state);
}

2.6.17-rt:

void tasklet_kill(struct tasklet_struct *t)
{
        if (in_interrupt())
                printk("Attempt to kill tasklet from interrupt\n");

        while (test_and_set_bit(TASKLET_STATE_SCHED, &t->state)) {
                do                              msleep(1);
                while (test_bit(TASKLET_STATE_SCHED, &t->state));
        }
        tasklet_unlock_wait(t);
        clear_bit(TASKLET_STATE_SCHED, &t->state);
}

You should ask Ingo & the other -rt developers what the intent of this
change was.  Obviously it loops forever waiting for the state bit to
change.

On Thu, 2006-07-06 at 22:14 +0200, Pieter Palmers wrote:

I've put the debugging printk's into tasklet_kill. One interesting remark is that now that they are in place, I had to start/stop jackd multiple times before deadlock occurs. Without the printk's the machine always locked up on the first pass. However I stopped after the first lockup, so maybe this is not really significant.

Anyway, the new tasklet_kill looks like this:

void tasklet_kill(struct tasklet_struct *t)
{
	printk("enter tasklet_kill\n");
	if (in_interrupt())
		printk("Attempt to kill tasklet from interrupt\n");
	
	printk("passed interrupt check\n");

	while (test_and_set_bit(TASKLET_STATE_SCHED, &t->state)) {
		do
			msleep(1);
		while (test_bit(TASKLET_STATE_SCHED, &t->state));
	}
	printk("passed test_and_set_bit\n");
	
	tasklet_unlock_wait(t);
	printk("passed tasklet_unlock_wait\n");
	
	clear_bit(TASKLET_STATE_SCHED, &t->state);
}

And the last line printed before lockup is:
"passed test_and_set_bit"
This makes the change in tasklet_unlock_wait() as the prime suspect for this problem.








-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[Index of Archives]     [Kernel Newbies]     [Netfilter]     [Bugtraq]     [Photo]     [Stuff]     [Gimp]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Video 4 Linux]     [Linux for the blind]     [Linux Resources]
  Powered by Linux