Re: [PATCH 20/20] iscsi: support for swapping over iSCSI.

[Mike Christie <michaelc@cs.wisc.edu>]

Mike Christie wrote:
> Peter Zijlstra wrote:
>> On Thu, 2006-09-14 at 14:22 -0500, Mike Christie wrote:
>>> Peter Zijlstra wrote:
>>>> On Wed, 2006-09-13 at 15:50 -0500, Mike Christie wrote:
>>>>> Peter Zijlstra wrote:
>>>>>> Implement sht->swapdev() for iSCSI. This method takes care of reserving
>>>>>> the extra memory needed and marking all relevant sockets with SOCK_VMIO.
>>>>>>
>>>>>> When used for swapping, TCP socket creation is done under GFP_MEMALLOC and
>>>>>> the TCP connect is done with SOCK_VMIO to ensure their success. Also the
>>>>>> netlink userspace interface is marked SOCK_VMIO, this will ensure that even
>>>>>> under pressure we can still communicate with the daemon (which runs as
>>>>>> mlockall() and needs no additional memory to operate).
>>>>>>
>>>>>> Netlink requests are handled under the new PF_MEM_NOWAIT when a swapper is
>>>>>> present. This ensures that the netlink socket will not block. User-space will
>>>>>> need to retry failed requests.
>>>>>>
>>>>>> The TCP receive path is handled under PF_MEMALLOC for SOCK_VMIO sockets.
>>>>>> This makes sure we do not block the critical socket, and that we do not
>>>>>> fail to process incomming data.
>>>>>>
>>>>>> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
>>>>>> CC: Mike Christie <michaelc@cs.wisc.edu>
>>>>>> ---
>>>>>>  drivers/scsi/iscsi_tcp.c            |  103 +++++++++++++++++++++++++++++++-----
>>>>>>  drivers/scsi/scsi_transport_iscsi.c |   23 +++++++-
>>>>>>  include/scsi/libiscsi.h             |    1 
>>>>>>  include/scsi/scsi_transport_iscsi.h |    2 
>>>>>>  4 files changed, 113 insertions(+), 16 deletions(-)
>>>>>>
>>>>>> Index: linux-2.6/drivers/scsi/iscsi_tcp.c
>>>>>> ===================================================================
>>>>>> --- linux-2.6.orig/drivers/scsi/iscsi_tcp.c
>>>>>> +++ linux-2.6/drivers/scsi/iscsi_tcp.c
>>>>>> @@ -42,6 +42,7 @@
>>>>>>  #include <scsi/scsi_host.h>
>>>>>>  #include <scsi/scsi.h>
>>>>>>  #include <scsi/scsi_transport_iscsi.h>
>>>>>> +#include <scsi/scsi_device.h>
>>>>>>  
>>>>>>  #include "iscsi_tcp.h"
>>>>>>  
>>>>>> @@ -845,9 +846,13 @@ iscsi_tcp_data_recv(read_descriptor_t *r
>>>>>>  	int rc;
>>>>>>  	struct iscsi_conn *conn = rd_desc->arg.data;
>>>>>>  	struct iscsi_tcp_conn *tcp_conn = conn->dd_data;
>>>>>> -	int processed;
>>>>>> +	int processed = 0;
>>>>>>  	char pad[ISCSI_PAD_LEN];
>>>>>>  	struct scatterlist sg;
>>>>>> +	unsigned long pflags = current->flags;
>>>>>> +
>>>>>> +	if (sk_has_vmio(tcp_conn->sock->sk))
>>>>>> +		current->flags |= PF_MEMALLOC;
>>>>>>  
>>>>> Is this too late or not needed or what is it for? This function gets run
>>>>> from the network layer's softirq and at this point we have a skbuff with
>>>>> data that we want to process. The iscsi layer also does not allocate
>>>>> memory for read or write IO in this path.
>>>> I thought I found allocations in that path, lemme search...
>>>> found this:
>>>>
>>>> iscsi_tcp_data_recv()
>>>>   iscsi_data_rescv()
>>>>     iscsi_complete_pdu()
>>>>       __iscsi_complete_pdu()
>>>>         iscsi_recv_pdu()
>>>>           alloc_skb( GFP_ATOMIC);
>>>>
>>> You are right that is for the netlink interface. Could we move the
>>> PF_MEMALLOC setting and clearing to iscsi_recv_pdu and and add it to
>>> iscsi_conn_error in scsi_transport_iscsi.c so that iscsi_iser and
>>> qla4xxx will have it set when they need it. I will send a patch for this
>>> along with a way to have the netlink sock vmio set for all iscsi drivers
>>> that need it.
>> I already have such a patch, look at:
>> http://programming.kicks-ass.net/kernel-patches/vm_deadlock/current/iscsi_vmio.patch
>>
> 
> You are drowning me in patches :) I did not see that one. I was still
> commenting on this patch :)
> 
> The new patch looks ok.
> 

Oh, I think you need a sock_put to go with netlink lookup (lookup does a
hold).
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/