Re: [(repost) git Patch 1/1] avoid IRQ0 ioapic pin collision

[ebiederm@xmission.com (Eric W. Biederman)]

Andi Kleen <ak@suse.de> writes:

> On Tuesday 02 May 2006 08:57, Eric W. Biederman wrote:
>> Andi Kleen <ak@suse.de> writes:
>> 
>> No.  At best there is a fixed offset.  They can't be
>> identical because the first 32 vectors are reserved,
>> for processor exceptions.  
>> 
>> Beyond that the kernel would not need the vector_irq and irq_vector
>> arrays if they were always identical, or even if they were one to one.
>
>
> Yes I should have said it's a fixed offset. Sorry for the confusion.
> Just no mapping table needed.

As far as I can see no mapping table is needed simply because
we place a stub with the irq number of the irq we want to service
at the idt entry of the vector we want to service.

For the old ISA irqs it is certainly a fixed offset.

>> If you look at assign_irq_vector you will see that by default we
>> allocate every 8th vector.  Looking at the comment in
>> init_IO_APIC_traps() this seems to be because we want to avoid
>> apic bugs with multiple interrupts of the same priority.
>> Although why we skip 8 instead of 16 is beyond me.
>
> Hmm - i guess that's old APIC bugs. Might be worth revisiting
> on 64bit.

Certainly worth a look.  I just know that is what the code currently
does. 

>> Although now that I think about it, using some assembler macros
>> instead of cpp macros could probably solve the problem more easily
>> than generating the stubs at runtime.  I think the worst case is
>> 256 cpus * 32 irqs per cpu 
>
> 32 irqs? It's (255-32) 

If you look at NR_IRQ_VECTORS.  32 is currently our estimate on the number
of IRQs sources per cpu.  I think a sane formula for NR_IRQS once
the static limits are removed is something like:
#define NR_IRQS (NR_VECTORS + (NR_CPUS*32))

Which starts out a little high to account for systems that are over endowed
with I/O and sticks with our current fairly high estimate of 32 IRQs per cpu.

Although that 32 may actually come from our the skip 8 logic.

>> * 10 bytes per stub = 80K. 
>
> My calculations gave >200k which is definitely too much.

With a per cpu lookup table we get 4 bytes per entry and 256 entries
per cpu = 1K per cpu.  Which ultimately uses 256K, on a 256 cpu
system.  Which is nothing for a machine with 256GB or so of RAM.
And that could really be be 2 bytes per entry.

I really don't care how we generate the stubs, or if we generate
stubs so long as all I need to do is change NR_IRQS recompile and
it works.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/