inconsistent mmap and get_user_pages with hugetlbfs on ppc64

["Sexton, Matt" <sexton@mc.com>]

On a ppc64 platform running 2.6.13-1, the virtual to physical mapping
established by mmap'ing a hugetlbfs file does not seem to match the
mapping described by get_user_pages().

Specifically, I have written a driver for a PCI device that writes data
into a user-allocated memory buffer.  The user passes the virtual
address of the buffer to the driver, which calls get_user_pages() to
lock the pages and to get the page information in order to be able to
build a scatter list of contiguous physical blocks to pass to the DMA
engine on the device. The device then writes a known pattern to the
buffer, which the user space program can verify.

This process works fine on ia32 and ppc64 using malloc'ed memory.  This
process also works fine on ia32 when obtaining the memory by mmap'ing a
file on a hugetlbfs filesystem.  The 2MB pages are used to reduce the
number of entries in the scatter list.

The process doesn't work so well on ppc64 with hugetlbfs (and 16MB
pages).  Often, the data is written to the wrong 16MB pages, from the
perspective of the user space program.  The data is correct within a
16MB page, it's just written to the wrong page. It seems that the
information returned by  get_user_pages() doesn't match the virtual to
physical mapping used by the user process.

Any suggestions on what I could be doing wrong in this specific case?
Any known problems with the kernel in this case?

Please CC me on any replies.

Thanks,
Matt

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/