On Wed, 2008-04-02 at 07:29 +0800, Ed Greshko wrote: > Patrick O'Callaghan wrote: > > On Tue, 2008-04-01 at 20:50 +0800, Ed Greshko wrote: > >> Patrick O'Callaghan wrote: > >>> On Tue, 2008-04-01 at 10:57 +0800, Ed Greshko wrote: > >>>> charles f. zeitler wrote: > >>>>> --- Ed Greshko <Ed.Greshko@xxxxxxxxxxx> wrote: > >>>>> > >>>>>> charles f. zeitler wrote: > >>>>>>> i've been pruning my "downloads" disk, > >>>>>>> rather drastically, and not making a dent. > >>>>>>> > >>>>>>> today some more, less drastic but still > >>>>>>> hefty, same result. > >>>>>>> > >>>>>>> revisited du- checked it twice - three > >>>>>>> times- yup, it reports one directory at > >>>>>>> 800+ gb- on a 400gb disk! > >>>>>>> > >>>>>>> fsck (forced) failed to report any problems, > >>>>>>> there don't seem to be any symlinks, > >>>>>>> and the sub-direcory sizes are sane... > >>>>>>> > >>>>>>> any ideas welcome, and appreciated. > >>>>>> Instead of telling people what you are seeing it would be better to show the > >>>>>> actual commands and output. > >>>>>> > >>>>> good point. > >>>>> > >>>>> > >>>>> [fedora_8@Nyarlethotep ~]$ df > >>>>> Filesystem 1K-blocks Used Available Use% Mounted on > >>>>> /dev/sda8 13250836 11459264 1107608 92% / > >>>>> /dev/sda9 1898468 825572 974904 46% /tmp > >>>>> /dev/sda11 270882768 259964688 5414052 98% /home > >>>>> /dev/sda10 1898468 1156484 643992 65% /var > >>>>> /dev/sdc1 480719056 370452080 105383136 78% /home/fedora_8/music_vids > >>>>> /dev/sda2 101105 17986 77898 19% /boot > >>>>> tmpfs 1037552 248 1037304 1% /dev/shm > >>>>> /dev/sdb1 384578164 330445976 34596748 91% /home/fedora_8/torrents_isos > >>>>> > >>>>> /dev/sdb1 is the drive under discussion. > >>>>> > >>>>> > >>>>> [fedora_8@Nyarlethotep ~]$ du -sb t*s/* > >>>>> 34256010522 torrents_isos/backup > >>>>> 883393808812 torrents_isos/data > >>>>> 58352749159 torrents_isos/finished > >>>>> 75197043648 torrents_isos/finnished > >>>>> 18222558607 torrents_isos/isos > >>>>> 4781438 torrents_isos/logs > >>>>> 16384 torrents_isos/lost+found > >>>>> 4096 torrents_isos/lost_meta > >>>>> 193903286 torrents_isos/meta > >>>>> 1402434610 torrents_isos/new > >>>>> 75585799469 torrents_isos/porn > >>>>> 4096 torrents_isos/rar > >>>>> 1318803 torrents_isos/shas > >>>>> 4096 torrents_isos/tmp > >>>>> 97996487 torrents_isos/total_meta.tar.bz2 > >>>>> 4096 torrents_isos/zip > >>>>> > >>>>> somethings wrong with t*s/data .... > >>>>> > >>>> OK.... I believe I know what the problem is. The torrents_isos/porn > >>>> directory makes things seem larger than what they really are.... > >>>> > >>>> No, just kidding..... > >>>> > >>>> I believe you may have a bunch of non-completed torrent downloads. When you > >>>> start a torrent download the client will reserve the space and it will be > >>>> reflected in the output of "du" but *not* in the output of "df". Thus with > >>>> "du" you can have a situation where it "thinks" more disk space is being > >>>> used than it actually is. FWIW, this is normal. > >>> No. > >> What do you mean "no"? > >> > >> When starting the download of a torrent the output of "du" shows the disk > >> space has been used. But, in reality, it hasn't. df reflects that is > >> hasn't been actually used. > > > > There are two scenarios, according to whether your torrent client > > preallocates space or not. I use Ktorrent, which can be set to do > > either. In the default case, no preallocation is done. After starting a > > torrent an 'ls -l' of the file shows its full size, but a 'du' shows > > only what has actually been downloaded. I checked this before my > > previous post and I just checked it again to be sure. > > > > If preallocation is on then of course 'du' will show the allocated > > space, but 'df' will account for it. I also checked this BTW. The > > numbers add up. > > > > In other words 'df' and 'du' are mutually consistent. > > While it may be the case on your system, it isn't on mine and apparently > isn't on the OP's. And if you use azureus on your system you could prove it > to yourself too. Just installed Azureus and tested it. It behaves exactly the same as Ktorrent, both with and without preallocation. This is hardly surprising as we're talking about how the filesystem works, not about about a specific application. As someone else pointed out, a inconsistency between the results of 'du' and 'df' can occur when a running process has an open file which has been unlinked, meaning the space occupied by the file has not yet been reclaimed even though the file no longer has a name. However, any inconsistency arising from this would be in the *opposite* direction to what the OP is seeing, i.e. less actual free space than accounted for by the filesystem size less the space occupied by (visible) files. In all of this I've been assuming that preallocation means what it has always meant traditionally in Unix/Linux: write garbage into the file to make sure disk space is allocated. It turns out the system can now do this for you, with the fallocate() call. However this is a very recent addition and people were still arguing about its semantics less than a year ago -- see http://lwn.net/Articles/240571. One detail stands out: with the correct parameter, the apparent size of the file (as reported by the stat() call) does not change, even if space is allocated beyond its end. The effect of this on 'df' and 'du' doesn't seem to be documented anywhere. Furthermore, although it's not explicitly stated, one presumes that the unused space is reclaimed when the file is closed, so the OP's question still stands: how can 'du' report more space than is being used, even if no processes have open files? For that matter, how can the system preallocate more space than the size of the filesystem? Doesn't make sense. BTW, it might be worth knowing what filesystem the OP is using. In my case it's ext3. poc
