Re: [PATCH 20/20] iscsi: support for swapping over iSCSI.

[Peter Zijlstra <a.p.zijlstra@chello.nl>]

On Thu, 2006-09-14 at 16:03 -0500, Mike Christie wrote:
> Mike Christie wrote:
> > Peter Zijlstra wrote:
> >> On Thu, 2006-09-14 at 14:22 -0500, Mike Christie wrote:
> >>> Peter Zijlstra wrote:
> >>>> On Wed, 2006-09-13 at 15:50 -0500, Mike Christie wrote:
> >>>>> Peter Zijlstra wrote:
> >>>>>> Implement sht->swapdev() for iSCSI. This method takes care of reserving
> >>>>>> the extra memory needed and marking all relevant sockets with SOCK_VMIO.
> >>>>>>
> >>>>>> When used for swapping, TCP socket creation is done under GFP_MEMALLOC and
> >>>>>> the TCP connect is done with SOCK_VMIO to ensure their success. Also the
> >>>>>> netlink userspace interface is marked SOCK_VMIO, this will ensure that even
> >>>>>> under pressure we can still communicate with the daemon (which runs as
> >>>>>> mlockall() and needs no additional memory to operate).
> >>>>>>
> >>>>>> Netlink requests are handled under the new PF_MEM_NOWAIT when a swapper is
> >>>>>> present. This ensures that the netlink socket will not block. User-space will
> >>>>>> need to retry failed requests.
> >>>>>>
> >>>>>> The TCP receive path is handled under PF_MEMALLOC for SOCK_VMIO sockets.
> >>>>>> This makes sure we do not block the critical socket, and that we do not
> >>>>>> fail to process incomming data.
> >>>>>>
> >>>>>> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> >>>>>> CC: Mike Christie <michaelc@cs.wisc.edu>
> >>>>>> ---
> >>>>>>  drivers/scsi/iscsi_tcp.c            |  103 +++++++++++++++++++++++++++++++-----
> >>>>>>  drivers/scsi/scsi_transport_iscsi.c |   23 +++++++-
> >>>>>>  include/scsi/libiscsi.h             |    1 
> >>>>>>  include/scsi/scsi_transport_iscsi.h |    2 
> >>>>>>  4 files changed, 113 insertions(+), 16 deletions(-)
> >>>>>>
> >>>>>> Index: linux-2.6/drivers/scsi/iscsi_tcp.c
> >>>>>> ===================================================================
> >>>>>> --- linux-2.6.orig/drivers/scsi/iscsi_tcp.c
> >>>>>> +++ linux-2.6/drivers/scsi/iscsi_tcp.c
> >>>>>> @@ -42,6 +42,7 @@
> >>>>>>  #include <scsi/scsi_host.h>
> >>>>>>  #include <scsi/scsi.h>
> >>>>>>  #include <scsi/scsi_transport_iscsi.h>
> >>>>>> +#include <scsi/scsi_device.h>
> >>>>>>  
> >>>>>>  #include "iscsi_tcp.h"
> >>>>>>  
> >>>>>> @@ -845,9 +846,13 @@ iscsi_tcp_data_recv(read_descriptor_t *r
> >>>>>>  	int rc;
> >>>>>>  	struct iscsi_conn *conn = rd_desc->arg.data;
> >>>>>>  	struct iscsi_tcp_conn *tcp_conn = conn->dd_data;
> >>>>>> -	int processed;
> >>>>>> +	int processed = 0;
> >>>>>>  	char pad[ISCSI_PAD_LEN];
> >>>>>>  	struct scatterlist sg;
> >>>>>> +	unsigned long pflags = current->flags;
> >>>>>> +
> >>>>>> +	if (sk_has_vmio(tcp_conn->sock->sk))
> >>>>>> +		current->flags |= PF_MEMALLOC;
> >>>>>>  
> >>>>> Is this too late or not needed or what is it for? This function gets run
> >>>>> from the network layer's softirq and at this point we have a skbuff with
> >>>>> data that we want to process. The iscsi layer also does not allocate
> >>>>> memory for read or write IO in this path.
> >>>> I thought I found allocations in that path, lemme search...
> >>>> found this:
> >>>>
> >>>> iscsi_tcp_data_recv()
> >>>>   iscsi_data_rescv()
> >>>>     iscsi_complete_pdu()
> >>>>       __iscsi_complete_pdu()
> >>>>         iscsi_recv_pdu()
> >>>>           alloc_skb( GFP_ATOMIC);
> >>>>
> >>> You are right that is for the netlink interface. Could we move the
> >>> PF_MEMALLOC setting and clearing to iscsi_recv_pdu and and add it to
> >>> iscsi_conn_error in scsi_transport_iscsi.c so that iscsi_iser and
> >>> qla4xxx will have it set when they need it. I will send a patch for this
> >>> along with a way to have the netlink sock vmio set for all iscsi drivers
> >>> that need it.
> >> I already have such a patch, look at:
> >> http://programming.kicks-ass.net/kernel-patches/vm_deadlock/current/iscsi_vmio.patch
> >>
> > 
> > You are drowning me in patches :) I did not see that one. I was still
> > commenting on this patch :)
> > 
> > The new patch looks ok.
> > 
> 
> Oh, I think you need a sock_put to go with netlink lookup (lookup does a
> hold).

D'0h again, I'd forgotten I'd used it there too.
hit refresh :-)


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/