Re: File size limit exceeded

Samuel Flory wrote:

Björn Persson wrote:

Horacio Reyes wrote:

Fews weeks ago, we change our old AIX for a new intel/fedora machine. We
make software in ansi C, we recompile all programs with gcc and works fine
with almost every input file, until we process a program with a 3Gb input
file (AFP file), then one of our programs say "File size limit exceeded"

We re-run this on our old AIX and works fine with this file.

Any idea?

What file system do you use? Ext2 and Ext3 have a limit at about 2 GB. Some other file systems can handle larger files.

Most likely the error message in their program is wrong, and they are hitting a memory addressing limit.

On second thought you may need to modify the program to use LFS, or recompile and add "-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE" to your compile options.

http://www.suse.de/~aj/linux_lfs.html

 Using LFS

For using LFS in user programs, the programs have to use the LFS API. This involves recompilation and changes of programs. The API is documented in the glibc manual (the libc info pages) which can be read with e.g. "info libc".

In a nutshell for using LFS you can choose either of the following:

* Compile your programs with "gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE". This forces all file access calls to use the 64 bit variants. Several types change also, e.g. off_t becomes off64_t. It's therefore important to always use the correct types and to not use e.g. int instead of off_t.
* Define _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE. With these defines you can use the LFS functions like open64 directly.
* Use the O_LARGEFILE flag with open to operate on large files.

A complete documentation of the feature test macros like _FILE_OFFSET_BITS and _LARGEFILE_SOURCE is in the glibc manual (run e.g. "info libc 'Feature Test Macros'").

The LFS API is also documented in the LFS standard which is available at http://ftp.sas.com/standards/large.file/x_open.20Mar96.html.
LFS and Libraries other than Glibc

Be careful when using _FILE_OFFSET_BITS=64 to compile a program that calls a library or a library if any of the interfaces uses off_t. With _FILE_OFFSET_BITS=64 glibc will change the type of off_t to off64_t. You can either change the interface to always use off64_t, use a different function if _FILE_OFFSET_BITS=64 is used (like glibc does). Otherwise take care that both library and program have the same _FILE_OFFSET_BITS setting. Note that glibc is aware of the _FILE_OFFSET_BITS setting, there's no problem with it but there might be problems with other libraries.

--
There is no such thing as obsolete hardware.
Merely hardware that other people don't want.
(The Second Rule of Hardware Acquisition)
Sam Flory  <sflory@xxxxxxxxxxxx>