Re: Ordering between PCI config space writes and MMIO reads?

[John Partridge <johnip@sgi.com>]

Matthew,

So, if I understand correctly, you are saying because we cannot guarantee
the "flush" a config write even by doing a config read of the same register
(because the PPB can re-order) we have to make sure we block or spin on the
config write completion at the lowest level of the config write ?

Thanks
John

Matthew Wilcox wrote:
> On Wed, Nov 01, 2006 at 10:27:19AM -0600, John Partridge wrote:
>
> > Sorry, but I find this change a bit puzzling. The problem is particular to
> > the PPB on the HCA and not Altix.
>
>
> That's not true; it's more likely on Altix, but it's not unique.  *any*
> PCI-PCI bridge can reorder pci config reads and writes.  Apparently the
> normal PCI host bridge implementation avoids this problem by blocking
> until the completion comes back.  If you put a quad-port tulip card into
> an Altix, you could experience the same problem (but it would be
> massively unlikely.  You'd probably have to bring up three interfaces,
> saturate them with traffic, then bring up the fourth to see it.  And
> even then it would be rare).
>
>
> > I can't see anywhere that a PCI Config 
> > Write
> > is required to block until completion, it is the driver and the HCA ,not the
> > Altix hardware that requires the Config Write to have completed before we
> > leave mthca_reset()
>
>
> There's several places in the PCI midlayer that require the config write
> to have completed before we do a config read.  The MWI code relies on
> this to see if the device supports MWI.  If it gets out of order, we'll
> think that the device doesn't support MWI when it thinks it's been told
> to use MWI.  Data corruption could result.
>
>
> > Changing pci_write_config_xxx() will change the behavior
> > for ALL drivers and the possibility of breaking something else. The fix was
> > very low risk in mthca_reset(), changing the PCI code to fix this is much
> > more onerous.
>
>
> I really don't think so.  At worst you'll be changing the timing.


--
John Partridge

Silicon Graphics Inc
Tel:	651-683-3428
Vnet:	233-3428
E-Mail:	johnip@sgi.com
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/