Re: question about contest benchmark

[Con Kolivas <kernel@kolivas.org>]

On Wed, 4 May 2005 04:11, Haoqiang Zheng wrote:
> I am wondering how we should interpret the CONTEST benchmark results.
> I tried CONTEST with process_load on 2.6.12-rc3 (single CPU, P4 2.8G,
> 1G RAM). The CPU usage of kernel compiling is 28.9%, the load consumes
> 70.1% and the ratio is 3.98.  Based on what Con says, the result is
> bad since the ratio is high. I did some tracing and found the
> background load (contest) runs at a dynamic priority of 115-120, which
> is often higher than the dynamic priority of the kernel compiling
> processes. This explains why the process_load consumes so much CPU.
>
>  My question is why is the result bad at all? One could certainly
> argue that contest processes shouldn't consume so much CPU time since
> they are considered to be background jobs. But why is kernel compiling
> considered foreground jobs? Why making kernel compiling faster is
> better? Actually, I am wondering if CONTEST is an appropriate
> benchmark to report system responsiveness at all?

I don't think in my readme do I say anywhere what is the ideal balance. 
Process_load is a uniquely different load to the other loads which are 
various combinations of cpu and i/o. It spawns processes that wake up, hand 
their data off to another process and go to sleep. Thus the processes behave 
like interactive one with their frequent waiting, but share their effective 
group cpu usage amongst all the process_child processes running so none of 
them is actually seen as cpu bound. Furthermore there are massive numbers of 
context switches between them meaning there is a large in-kernel "system" 
load that is done on behalf of the process_child ren. The purpose of the 
process_load in contest is to ensure that an interactive design is not 
DoS'able by processes behaving like this. Process_load spawns 4 times as many 
processes as the timed 'make' in contest so theoretically ideal cpu balance 
between them should show process_load having 4x as much cpu as the make. 
Because their cpu binding is so intermittent it's hard to balance them 
perfectly. Anyway the balance in your output seems pretty good. When the 
interactive design goes horribly wrong process_load consumes 100 times as 
much cpu as the 'make'.

>
>  Any comments?
>
>  BTW, what benchmark do you guys use to test system responsiveness?

Note that interactivity is not responsiveness which some people try to measure 
with contest, and there is still no interactivity benchmark. Responsiveness 
is the ability of the system to continue performing tasks at a reasonable 
pace under various system loads. Interactivity is having low scheduling 
latency and jitter in those tasks where human interaction would notice the 
latency and jitter - and what constitutes and interactive tasks has not been 
quantified although we all know what they are when using the pc.

Cheers,
Con

Attachment: pgp98KBy9TUrd.pgp
Description: PGP signature