Hi, > > At 15:20 5/28/2004, Michael Mansour wrote: > > >Not only do I wonder if tar will respond the same > way > > >as rsync without the -S option, but I'm also > wondering > > >if backup software which use rsync to perform > backups > > >would also take this into account... > > > > I also wonder if there is any downside to using -S > or whether it might > > produce an undesirable result in some cases. If > not, perhaps one should > > always use it just in case? > > Sparse files are a boon or a bust. > > If you make a sparse file > > open(); write(byte); lseek(way-out-there); > write(byte); close() > > you have not reserved or allocated the disk blocks > for all the > possible stuff in the middle. If you expect those > blocks to be real > and there is not enough disk space in the future > things will bust. > > In some cases if you compress the file then expand > it you can run > out of disk space on the uncompress pass (compress, > gzip, bzip2, zip). > Watch out for uncompress bombs. > > Some backup and restore tools will run you out of > tape or out of disk > or both with sparse files. Gnu tar has a --sparse > flag. You might not > be able to restore the contents of a file system if > you are not > cautious/ aware. > > Some data base files might be sparse but you could > run out of space > a year later when you do not expect and need the > blocks to be real. > > mkswap (to files) might generate files that look > like they should be > sparse to backup and restore tools but should not > be. i.e. You should > not use swapon on a file with holes. > > Holes on a file system with one size of allocation > block may not be > holes on another file system with a smaller > allocation block size. > Since allocation block size is heuristically > determined one could > be confused when moving files from one FS to another > of a different > size or type. mv does not have a --sparse flag that > I can see. > > The big issue is backup and restore processes > (including cp, mv). > > I guess a utility (holeify and holecount) might be > written. > > > -- > T o m M i t c h e l l > /dev/null the ultimate in secure storage. Thanks Tom, excellent explanation. In regards to where these sparse files came from, I set about to determine this and found out. As mentioned in earlier posts, I use that data volume under Samba, which is shared out to apps/clients to store their files. Azureus is a Java BitTorrent client, and when starting torrent downloads on that data drive, the entire size of the torrent is reserved on the data drive (so a 600Mb ISO shows up on the filesystem even though it is empty). A "du -ks" shows the actual size (a few bytes at the most), although an "ls -l" shows the 600Mb file which the filesystem recognises as space taken. So when I rsynced that data volume to another drive (without the -S option), the rsync naturally copied exactly what it saw, spacewise, but when using -S it recognised the sparse files and therefore didn't take up the space for them. We learn something everyday :) ... I'm wondering now if those sparse files that got "un-sparsed" on the rsync will still be valid if, for example, Azureus was to continue downloading to them. Michael. Find local movie times and trailers on Yahoo! Movies. http://au.movies.yahoo.com
