On Thu, 2007-10-04 at 16:09 -0700, Andrew Morton wrote: > On Fri, 05 Oct 2007 00:39:16 +0200 > Miklos Szeredi <[email protected]> wrote: > > > > throttle_vm_writeout() should be a per-zone thing, I guess. Perhaps fixing > > > that would fix your deadlock. That's doubtful, but I don't know anything > > > about your deadlock so I cannot say. > > > > No, doing the throttling per-zone won't in itself fix the deadlock. > > > > Here's a deadlock example: > > > > Total memory = 32M > > /proc/sys/vm/dirty_ratio = 10 > > dirty_threshold = 3M > > ratelimit_pages = 1M > > > > Some program dirties 4M (dirty_threshold + ratelimit_pages) of mmap on > > a fuse fs. Page balancing is called which turns all these into > > writeback pages. > > > > Then userspace filesystem gets a write request, and tries to allocate > > memory needed to complete the writeout. > > > > That will possibly trigger direct reclaim, and throttle_vm_writeout() > > will be called. That will block until nr_writeback goes below 3.3M > > (dirty_threshold + 10%). But since all 4M of writeback is from the > > fuse fs, that will never happen. > > > > Does that explain it better? > > > > yup, thanks. > > This is a somewhat general problem: a userspace process is in the IO path. > Userspace block drivers, for example - pretty much anything which involves > kernel->userspace upcalls for storage applications. > > I solved it once in the past by marking the userspace process as > PF_MEMALLOC and I beleive that others have implemented the same hack. > > I suspect that what we need is a general solution, and that the solution > will involve explicitly telling the kernel that this process is one which > actually cleans memory and needs special treatment. > > Because I bet there will be other corner-cases where such a process needs > kernel help, and there might be optimisation opportunities as well. > > Problem is, any such mark-me-as-special syscall would need to be > privileged, and FUSE servers presently don't require special perms (do > they?) I think just adding nr_cpus * ratelimit_pages to the dirth_thresh in throttle_vm_writeout() will also solve the problem
Attachment:
signature.asc
Description: This is a digitally signed message part
- Follow-Ups:
- Re: [PATCH] remove throttle_vm_writeout()
- From: Rik van Riel <[email protected]>
- Re: [PATCH] remove throttle_vm_writeout()
- References:
- [PATCH] remove throttle_vm_writeout()
- From: Miklos Szeredi <[email protected]>
- Re: [PATCH] remove throttle_vm_writeout()
- From: Andrew Morton <[email protected]>
- Re: [PATCH] remove throttle_vm_writeout()
- From: Miklos Szeredi <[email protected]>
- Re: [PATCH] remove throttle_vm_writeout()
- From: Andrew Morton <[email protected]>
- [PATCH] remove throttle_vm_writeout()
- Prev by Date: Re: video resume stuff
- Next by Date: Re: [PATCH 5/5] writeback: introduce writeback_control.more_io to indicate more io
- Previous by thread: Re: [PATCH] remove throttle_vm_writeout()
- Next by thread: Re: [PATCH] remove throttle_vm_writeout()
- Index(es):
 |