[+jejb to cc]
On Tue, Sep 25, 2007 at 04:58:43PM -0700, [email protected] wrote:
> This is a followup to http://lkml.org/lkml/2007/8/24/280
>
> Despite Grant's desire for a more elegant solution, there's
> not much new here. I moved the API change from pci.h to
> dma-mapping.h and removed the pci_ prefix from the name.
Thanks - but I don't have a better idea either.
I think you are right to just move forward with this until
someone provides a better API.
> Problem Description
> -------------------
> On Altix, DMA may be reordered within the NUMA interconnect.
> This can be a problem with Infiniband, where DMA to Completion Queues
> allocated in user-space can race with data DMA. This patchset allows
> a driver to associate a user-space memory region with a "dmaflush"
> attribute, so that writes to the memory region flush in-flight DMA,
> preventing the CQ/data race.
Can we define this API to provide the same semantics as the memory
that dma_alloc_coherent() returns?
Did I summarize this correctly?
Defining it terms of completion queues won't mean much to most folks.
Better to add a description of completion queues to the DMA-API.txt if
necessary. dma_alloc_coherent() API is pretty well understood.
> There are four patches in this set:
>
> [1/4] dma: add dma_flags_set_dmaflush() to dma interface
Sorry - this feels like a "color of the shed" argument, but isn't
this about DMA ordering attribute?
"dmaflush" is an action and not an attribute to me.
Is dma_flags_set_coherent() better since it's doing the same thing
as dma_alloc_coherent()?
> [2/4] dma: redefine dma_flags_set_dmaflush() for sn-ia64
> [3/4] dma: document dma_flags_set_dmaflush()
This patch updates Documentation/DMA-mapping.txt. But it's a change to
the generic (not PCI specific) API described in DMA-API.txt.
Can you update that as well please?
Upon reading the "2) Platforms that permit DMA reordering", I think I
have been confusing coherency with ordering. I think I have because DMA
is leaving the "PCI domain", crossing an "unordered domain" (NUMA,
interconnect), and then finally hitting the cache coherency "domain"
when it reaches a "far away" memory controller. That's why I've
been thinking of this as a coherency problem.
The description and API uses the word "flush" (which is ok I guess) instead
of describing this in terms of enforcing DMA ordering. Any DMA write to the
"strongly ordered" region will cause _all_ inflight DMA to be visible
to cache coherency, thus preserving the illusion of strong DMA ordering.
Does that sound right/better to you too?
I don't have chipset docs and some of this is just trying to rephrase
what I've heard before from former SGI employees.
> [4/4] mthca: allow setting "dmaflush" attribute on user-allocated memory
Besides calling the parameter "dmaflush", it looks fine to me.
(It's either a DMA ordering or coherency attribute depending on how
you want to look at it.)
thanks,
grant
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
