Re: tcp_sendpage and page allocation lifetime vs. iscsi

[Avi Kivity <avi@argo.co.il>]

Olivier Galibert wrote:

> I have a problem with the iscsi driver (both 4.x and 5.x) and scsi
> tape I'm not sure how to solve.  It may linked to some specific
> characteristics of the tg3 network driver.
>
> What happens is, from what I can trace:
> 1- st alloc_pages a bunch of pages for buffering
>
> 2- st sends a bunch of them to iscsi for writing (32K is common when
>     labelling a tape for instance)
>
> 3- iscsi sends whatever header is needed followed by the data using
>     tcp_sendpage
>
> 4- tcp_sendpage copies from of the pages but get_page() others,
>     probably depending on the state of the socket buffer.  It returns
>     immediatly anyway, leaving some pages with an elevated count (which, I
>     guess, it will eventually decrement again)
>
> 5- iscsi returns to st
>
> 6- st reuses the buffer immediatly, and/or frees it if the device is
>     closed.  Silent corruption in one case, bad_page in __free_page_ok
>     called from normalize_buffer in the other.
>
> I'm going to complete my traces to be sure that's really what's going
> on (I don't have a log immediatly after sendpage yet).  But in any
> case, what would the solution be?
>
>  
you need a completion to tell you when your buffer has been sent. you 
can use the kiocb parameter to tcp_sendmsg, as it has a completion. 
however, tcp_sendmsg does not appear to use it.

in effect, you need tcp aio, but the mainline kernel does not support it 
yet.

--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/