Re: Swap maximum size documented ?

[Bill Davidsen <davidsen@tmr.com>]

William Lee Irwin III wrote:

> William Lee Irwin III wrote:
>  
>
> > > Without CONFIG_HIGHMEM64G=y you have:
> > > 32 swapfiles, max swapfile size of 64GB.
> > > With CONFIG_HIGHMEM64G=y you have:
> > > 32 swapfiles, max swapfile size of 512GB.
> > >      
>
> On Wed, Jun 01, 2005 at 03:10:59PM -0400, Bill Davidsen wrote:
>  
>
> > Does this apply to mmap as well? I have an application which currently 
> > uses 9TB of data, and one thought to boost performance was to mmap the 
> > data. Unfortunately, I know 16TB isn't going to be enough for more than 
> > a few more years :-(
> >    
>
> This only applies to swapping on ia32/i386.
>
> mmap() is limited only by file offsets, which are fully 32-bit on
> 32-bit systems. remap_file_pages() is limited by PTE_FILE_MAX_BITS,
> which is fully 32-bit with CONFIG_HIGHMEM64G=y on i386 but only 29 bit
> without it on i386. In general checking for PTE_FILE_MAX_BITS on the
> relevant architecture should answer your question for remap_file_pages(),
> and BITS_PER_LONG for mmap(). The swap limits for other architectures
> will also differ and you generally have to look at the swp_entry/pte
> encoding/decoding macros to decipher what the precise limits are
> (though a quick hacky C program can help discover them for you).
> Generally you get the filesizes by PAGE_SIZE << X_FILE_OFFSET_BITS.
>
> It is in principle possible to sweep the kernel to allow larger file
> offsets on 32-bit systems (pgoff_t is something of a preparation for
> that), but I wouldn't advise trying it without rather strong kernel-fu
> and much willingness to debug it by one's self, and that with a common
> failure mode of fs data corruption. Widening swp_entry_t is slightly
> harder as the ptes have limited capacity so you have to somehow allocate
> extra data in a deadlock-free manner, but one also has less disruptive
> failure modes. I suspect you're not itching to implement these things.
>
> One thing to keep in mind is that these are only permissible filesizes.
> Virtualspace must be managed properly for windowing where it's limited
> and to prevent pagetable proliferation where it's not.

Thank you for taking the time to give me such a complete reply (I saved 
it to prevent asking similar in the future). Hopefully I will be able to 
leave the problem to someone else by then.

--
bill davidsen <davidsen@tmr.com>
 CTO TMR Associates, Inc
 Doing interesting things with small computers since 1979

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/