Re: Possible regression: MSI vector leakage since 2.6.18-rc5ish (Unable to repeatedly allocate/free MSI interrupt)

[ebiederm@xmission.com (Eric W. Biederman)]

Auke Kok <auke-jan.h.kok@intel.com> writes:

> Hi,
>
> I've established a regression in the MSI vector/irq allocation routine for both
> i386 and x86_64. Our test labs repeatedly modprobe/rmmod the e1000 driver for
> serveral minutes which allocates msi vectors and frees them. These tests have
> been running fine until 2.6.19.
>
> git-bisecting I've established that in between commit
> 04b9267b15206fc902a18de1f78de6c82ca47716 "Eric W. Biederman -- genirq: x86_64
> irq: Remove the msi assumption that irq == vector" and commit
> f29bd1ba68c8c6a0f50bd678bbd5a26674018f7c "Ingo Molnar -- genirq: convert the
> x86_64 architecture to irq-chips" the behaviour broke.
>
> The revisions in between seem to be dependent and give all sorts of other
> issues, so it's rather hard for me to bisect that and give trustworthy results.
>
> the e1000 driver hits the 256-mark cycle (I think - it consistently refuses to
> do 500 msi irq/vector allocations which is my test case) and throws:
>
> e1000: eth4: e1000_request_irq: Unable to allocate MSI interrupt Error: -16
>
> which is caused by a `if ((err = pci_enable_msi(adapter->pdev))) {` call from
> the e1000 driver. It's rather easy to hit this mark with the new 4-port e1000
> adapters :).
>
> as for the e1000 code, I can say that even our oldest msi-enabled e1000 driver
> works fine with 2.6.18 and under. All kernels from 2.6.19 fail consistently.
>
> I mostly suspect commit 7bd007e480672c99d8656c7b7b12ef0549432c37 at the
> moment. Perhaps Eric Biederman can help?
>
> Cheers,
>
> Auke
>

Does this patch fix it for you?  It looks like i386 vector allocate
did not have logic to look through the set of vectors more than once.

The code in this patch is a simplified version of what we have
on x86_64.

Eric


diff --git a/arch/i386/kernel/io_apic.c b/arch/i386/kernel/io_apic.c
index 2424cc9..6a3875f 100644
--- a/arch/i386/kernel/io_apic.c
+++ b/arch/i386/kernel/io_apic.c
@@ -1227,26 +1227,32 @@ static u8 irq_vector[NR_IRQ_VECTORS] __read_mostly = { FIRST_DEVICE_VECTOR , 0 }
 
 static int __assign_irq_vector(int irq)
 {
-	static int current_vector = FIRST_DEVICE_VECTOR, offset = 0;
-	int vector;
+	static int current_vector = FIRST_DEVICE_VECTOR, current_offset = 0;
+	int vector, offset, i;
 
 	BUG_ON((unsigned)irq >= NR_IRQ_VECTORS);
 
 	if (irq_vector[irq] > 0)
 		return irq_vector[irq];
 
-	current_vector += 8;
-	if (current_vector == SYSCALL_VECTOR)
-		current_vector += 8;
-
-	if (current_vector >= FIRST_SYSTEM_VECTOR) {
-		offset++;
-		if (!(offset % 8))
-			return -ENOSPC;
-		current_vector = FIRST_DEVICE_VECTOR + offset;
-	}
-
 	vector = current_vector;
+	offset = current_offset;
+next:
+	vector += 8;
+	if (vector >= FIRST_SYSTEM_VECTOR) {
+		offset = (offset + 1) % 8;
+		vector = FIRST_DEVICE_VECTOR + offset;
+	}
+	if (vector == current_vector)
+		return -ENOSPC;
+	if (vector == SYSCALL_VECTOR)
+		goto next;
+	for (i = 0; i < NR_IRQ_VECTORS; i++)
+		if (irq_vector[i] == vector)
+			goto next;
+
+	current_vector = vector;
+	current_offset = offset;
 	irq_vector[irq] = vector;
 
 	return vector;
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/