Re: [PATCH][RFC] 4K stacks default, not a debug thing any more...?

[Rene Herman <rene.herman@gmail.com>]

On 07/17/2007 01:27 AM, Matt Mackall wrote:

> Larger soft pages waste tremendous amounts of memory (mostly in page
> cache) for minimal benefit on, say, the typical desktop. While there
> are workloads where it's a win, it's probably on a small percentage of
> machines.
>
> So it's absolutely no help in fixing our order-1 allocation problem
> because we don't want to force large pages on people.

I was just now looking at how much space is in fact wasted in pagecache for 
various pagesizes by running the attached dumb little program from a few 
selected directories (heavy stack recursion, never mind).

Well, hmmm. This is on a (compiled) git tree:

rene@7ixe4:~/src/linux/local$ pageslack
total	: 447350347
 4k	: 67738037 (15%)
 8k	: 147814837 (33%)
16k	: 324614581 (72%)
32k	: 724629941 (161%)
64k	: 1592785333 (356%)

Nicely constant factor 2.2 instead of the 2 one would expect but oh well. On 
a collection of larger files the percentages obviously drop. This is on a 
directory of ogg vorbis files:

root@7ixe4:/mnt/ogg/.../... # pageslack
total	: 70817974
 4k	: 26442 (0%)
 8k	: 67402 (0%)
16k	: 124746 (0%)
32k	: 288586 (0%)
64k	: 419658 (0%)

The "typical desktop" is presented by neither I guess but does involve audio 
and (much larger still) video and bloody huge browser apps.

Not too sure then that 8K wouldn't be something I'd want, given fewer 
pagefaults and all that...

Rene.

/* gcc -W -Wall -o pageslack pageslack.c */

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <dirent.h>

#define PAGE_SIZE (1UL << PAGE_SHIFT)
#define PAGE_MASK (~(PAGE_SIZE - 1))

unsigned long long total;
unsigned long long slack[5];

void do_dir(const char *name)
{
	DIR *dir; 
	struct dirent *ent;

	dir = opendir(name);
	if (!dir) {
		perror("opendir");
		exit(EXIT_FAILURE);
	}
	while ((ent = readdir(dir))) {
		struct stat buf;
		char path[PATH_MAX];

		if (!strcmp(ent->d_name, "."))
			continue;
		if (!strcmp(ent->d_name, ".."))
			continue;

		sprintf(path, "%s/%s", name, ent->d_name);
		if (stat(path, &buf)) {
			perror("stat");
			exit(EXIT_FAILURE);
		}
		if (S_ISDIR(buf.st_mode)) {
			do_dir(path);
			continue;
		}
		if (S_ISREG(buf.st_mode)) {
			int i;

			for (i = 0; i < 5; i++) {
				unsigned long PAGE_SHIFT = 12 + i;
				slack[i] += (PAGE_SIZE - (buf.st_size % PAGE_SIZE)) % PAGE_SIZE;
			}
			total += buf.st_size;
		}
	}
	if (closedir(dir)) {
		perror("closedir");
		exit(EXIT_FAILURE);
	}
}

int main(void)
{
	do_dir(".");
	printf("total\t: %llu\n", total);
	printf(" 4k\t: %llu (%llu%%)\n", slack[0], (100 * slack[0]) / total);
	printf(" 8k\t: %llu (%llu%%)\n", slack[1], (100 * slack[1]) / total);
	printf("16k\t: %llu (%llu%%)\n", slack[2], (100 * slack[2]) / total);
	printf("32k\t: %llu (%llu%%)\n", slack[3], (100 * slack[3]) / total);
	printf("64k\t: %llu (%llu%%)\n", slack[4], (100 * slack[4]) / total);
	return EXIT_SUCCESS;
}