Re: updatedb

[Bongani Hlope <bhlope@mweb.co.za>]

On Thursday 26 July 2007 05:59:53 Rene Herman wrote:
>
> Problem spot no. 1.
>
> RAM intensive? If I run updatedb here, it never grows itself beyond 2M.
> Yes, two. I'm certainly willing to accept that me and my systems are
> possibly not the reference but assuming I'm _very_ special hasn't done much
> for me either in the past.
>
> The thing updatedb does do, or at least has the potential to do, is fill
> memory with cached inodes/dentries but Linux does not swap to make room for
> caches. So why will updatedb "often cause things to be swapped out"?
>
> [ snip ]
>
> > Swap prefetch, on the other hand, would have kicked in shortly after
> > updatedb finished, leaving the applications in swap for a speedy
> > recovery when the person comes back to their computer.
>
> Problem spot no. 2.
>
> If updatedb filled all of RAM with inodes/dentries, that RAM is now used
> (ie, not free) and swap-prefetch wouldn't have anywhere to prefetch into so
> would _not_ have kicked in.
>
> So what's happening? If you sit down with a copy op "top" in one terminal
> and updatedb in another, what does it show?
>
> Rene.

Just tested that, there's a steady increase in the useage of buff

<start updatedb>
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  1      0 1279412 201160 234720    0    0   193    29  558  657  5  1 89  5
 0  1      0 1276624 203436 234872    0    0  2276     0 1638 2456  4  2 48 48
 1  1      0 1273372 206292 235012    0    0  2852     0 1773 2755  3  3 48 46
 2  1      0 1270128 208376 235360    0    0  2084     0 1545 2168  5  2 47 46

8<

 0  1      0 1228004 237288 237836    0    0  2192     0 1669 2941  6  3 47 44
 1  1      0 1223424 239228 238020    0    0  1932   272 1580 2881  9  4 44 44
 1  1      0 1219692 241600 238208    0    0  2372     0 1719 2881 10  4 45 43
 0  1      0 1217296 243372 238312    0    0  1772     0 1526 2320  4  2 49 46

8<

 0  1      0 1166852 277912 240840    0    0  2244     0 1699 3037  7  2 48 43
 0  1      0 1164016 279528 241016    0    0  1608   824 1512 2364  7  2 47 44
 1  1      0 1161256 281860 241264    0    0  2332     0 1709 2769  7  2 49 43
 1  1      0 1155632 284792 241452    0    0  2932     0 1835 3084  8  4 46 42

8< 

 0  1      0 1104568 324788 243616    0    0  3500     4 1879 3054  5  4 46 46
 1  1      0 1099596 328524 243768    0    0  3736     0 1990 3257  7  4 48 43
 1  1      0 1093976 332516 244060    0    0  3984   572 2013 3348  6  3 48 43
 0  1      0 1090320 335396 244340    0    0  2880     0 1760 2925  5  3 47 46

8<

 1  1      0 1025212 384380 248224    0    0  2940     0 1763 2864  6  3 46 46
 0  1      0 1022196 386444 248328    0    0  2064     8 1527 2543  5  2 45 47
 0  1      0 1018620 389476 248404    0    0  3032     0 1798 2988  6  3 47 45
 0  1      0 1014800 392364 248552    0    0  2888     0 1738 2821  5  2 48 45

8<

 0  1      0 425200 839828 273392    0    0  1744     0 1441 2248  9  2 44 46
 0  1      0 423360 841220 273544    0    0  1384   368 1374 2144  3  1 48 48
 0  1      0 421288 842868 273576    0    0  1648     0 1400 2141  4  2 46 48
 0  1      0 418252 845172 273676    0    0  2300     0 1570 2492  3  1 49 48
 0  0      0 417300 846100 273776    0    0   928     0 1232 1837  3  2 72 24 

<updatedb finished>

 0  0      0 416724 846100 273776    0    0     0     0 1025 1579  5  1 94  0
 0  0      0 417012 846100 273776    0    0     0     0 1002 1474  3  1 97  0
 1  0      0 417220 846100 273776    0    0     0     0 1026 1414  2  0 98  0

So 32 percent of free memory went to the buffers.

5 minutes later it's still not freed

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 2  0      0 409500 846652 277320    0    0   286    31  585  766  6  1 83 10
 1  0      0 409328 846652 277320    0    0     0     0 1003 1442  3  1 97  0

/proc/slabinfo
ext3_inode_cache  176198 176200    816    5    1 : tunables   54   27    8 : 
slabdata  35240  35240      0
dentry            233054 233054    208   19    1 : tunables  120   60    8 : 
slabdata  12266  12266      0
buffer_head       228303 228327    104   37    1 : tunables  120   60    8 : 
slabdata   6171   6171      0

run OpenOffice

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 1  0      0 403664 847056 277460    0    0   235    26  577  766  6  1 85  8
 0  0      0 403656 847056 277460    0    0     0     0 1003 1385  5  0 96  0
 0  0      0 403888 847056 277460    0    0     0     0 1237 1968  3  1 96  0

8<

<starts openoffice>
 0  0      0 400708 847088 277620    0    0     0     0 1058 1259  4  0 95  0
 0  0      0 400584 847088 277620    0    0     0     0 1246 1647  7  1 93  0
 1  1      0 389796 847164 284100    0    0  6528   116 1215 2663 10  4 71 14

8<

 0  0      0 307000 847464 361384    0    0     0     0 1031 1398  5  1 95  0
 0  0      0 307000 847464 361384    0    0     0     0 1003 1369  3  1 95  0
 0  0      0 307124 847464 361384    0    0     0     0 1025 1535  4  1 95  0

8<

 1  1      0 301920 847516 363176    0    0  1780   132 1092 1705 11  2 77 10
 1  1      0 289296 847588 367620    0    0  4608   152 1221 2280 31  4 48 18
 1  0      0 285672 847612 369572    0    0  1936     0 1061 3545 14  3 72 10
<open office loaded>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/