The problem was that node A's sending thread, which handles sending RDMA
read response data, would write the trigger word, the last packet would
be sent, node B would send a new RDMA read request, node A's interrupt
handler would initialize s_rdma_sge, then node A's sending thread would
update s_rdma_sge. This didn't happen very often naturally but was more
frequent with 1 byte RDMA reads. Rather than adding more locking or
increasing the QP structure size and copying sge data, I modified the
copy routine to update the pointers before writing the trigger word to
avoid the update race.
Signed-off-by: Ralph Campbell <[email protected]>
Signed-off-by: Bryan O'Sullivan <[email protected]>
diff -r 551717ecc3db -r 3d844dee2f61 drivers/infiniband/hw/ipath/ipath_layer.c
--- a/drivers/infiniband/hw/ipath/ipath_layer.c Tue May 23 11:29:16 2006 -0700
+++ b/drivers/infiniband/hw/ipath/ipath_layer.c Tue May 23 11:29:16 2006 -0700
@@ -872,12 +872,13 @@ static void copy_io(u32 __iomem *piobuf,
update_sge(ss, len);
length -= len;
}
+ /* Update address before sending packet. */
+ update_sge(ss, length);
/* must flush early everything before trigger word */
ipath_flush_wc();
__raw_writel(last, piobuf);
/* be sure trigger word is written */
ipath_flush_wc();
- update_sge(ss, length);
}
/**
@@ -943,17 +944,18 @@ int ipath_verbs_send(struct ipath_devdat
if (likely(ss->num_sge == 1 && len <= ss->sge.length &&
!((unsigned long)ss->sge.vaddr & (sizeof(u32) - 1)))) {
u32 w;
-
+ u32 *addr = (u32 *) ss->sge.vaddr;
+
+ /* Update address before sending packet. */
+ update_sge(ss, len);
/* Need to round up for the last dword in the packet. */
w = (len + 3) >> 2;
- __iowrite32_copy(piobuf, ss->sge.vaddr, w - 1);
+ __iowrite32_copy(piobuf, addr, w - 1);
/* must flush early everything before trigger word */
ipath_flush_wc();
- __raw_writel(((u32 *) ss->sge.vaddr)[w - 1],
- piobuf + w - 1);
+ __raw_writel(addr[w - 1], piobuf + w - 1);
/* be sure trigger word is written */
ipath_flush_wc();
- update_sge(ss, len);
ret = 0;
goto bail;
}
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]