Jay Lan wrote:
> Shailabh Nagar wrote:
>
>> Balbir Singh wrote:
>>
>>> Andrew Morton wrote:
>>>
>>>
>>>> per-task-delay-accounting-setup.patch
>>>> per-task-delay-accounting-setup-fix-1.patch
>>>> per-task-delay-accounting-setup-fix-2.patch
>>>> per-task-delay-accounting-sync-block-i-o-and-swapin-delay-collection.patch
>>>>
>>>>
>>>> per-task-delay-accounting-sync-block-i-o-and-swapin-delay-collection-fix-1.patch
>>>>
>>>>
>>>> per-task-delay-accounting-cpu-delay-collection-via-schedstats.patch
>>>> per-task-delay-accounting-cpu-delay-collection-via-schedstats-fix-1.patch
>>>>
>>>> per-task-delay-accounting-utilities-for-genetlink-usage.patch
>>>> per-task-delay-accounting-taskstats-interface.patch
>>>> per-task-delay-accounting-taskstats-interface-fix-1.patch
>>>> per-task-delay-accounting-taskstats-interface-fix-2.patch
>>>> per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface.patch
>>>>
>>>>
>>>> per-task-delay-accounting-delay-accounting-usage-of-taskstats-interface-use-portable-cputime-api-in-__delayacct_add_tsk.patch
>>>>
>>>>
>>>> per-task-delay-accounting-documentation.patch
>>>> per-task-delay-accounting-proc-export-of-aggregated-block-i-o-delays.patch
>>>>
>>>>
>>>> per-task-delay-accounting-proc-export-of-aggregated-block-i-o-delays-warning-fix.patch
>>>>
>>>>
>>>>
>>>> I just don't know. There are a number of groups who pop up with
>>>> various
>>>> enhanced accounting requirements and patches (all quite different)
>>>> but I
>>>> haven't heard a lot of enthusiasm from any of them over this work,
>>>> which
>>>> attempts to provide an extensible framework for accumulation and
>>>> querying
>>>> of per-task metrics.
>>>>
>>>> But then again, we cannot just sit there and wait for everyone to be
>>>> 100%
>>>> happy. So I'm 51% inclined to push this along.
>>>>
>>>> Anyone else who has an interest in this sort of thing needs to be aware
>>>> that there will be an expectation that any future statistics
>>>> submissions
>>>> should use these interfaces. So the time to pay attention is right
>>>> now.
>>>>
>>>
>>> Hi, Andrew,
>>>
>>> Here is a brief summary of the status of the response we have
>>> received from
>>> the stakeholders (some of it has been duplicated in previous postings)
>>>
>>> Project Response
>>>
>>> 1. CSA accounting/PAGG/JOB: Has agreed to use
>>> taskstats
>>> Jay Lan <[email protected]> interface
>>>
>>> 2. per-process IO statistics: None
>>> Levent Serinol <[email protected]> Needs are subset of CSA
>>>
>>> 3. per-cpu time statistics: None (email bounced)
>>> Erich Focht <[email protected]> Needs can be met by
>>> taskstats
>>> Statistics not yet
>>> submitted
>>>
>>> 4. Microstate accounting: None
>>> Peter Chubb <[email protected]> overlap with delay
>>> accounting
>>> prefers /proc due to
>>> convenience
>>> taskstats can meet the
>>> needs
>>>
>>>
>>> 5. ELSA: Guillaume Thouvenin None
>>> <[email protected]> ELSA is not a direct user
>>> of new kernel statistics
>>> Consumer of CSA/BSD
>>> accounting
>>> statistics
>>>
>>> 6. pnotify: Jes Sorensen <[email protected]> None
>>> (taken over pnotify from Erik Jacobson) Informed over private
>>> email
>>> that pnotify replacement is
>>> being worked on. pnotify
>>> or its replacement will
>>> not be concerned with
>>> exporting data to user
>>> space
>>> or collecting any
>>> statistics.
>>>
>>>
>>> 7. Scalable statistics counters with /proc Not working on it
>>> reporting: anymore
>>> Ravikiran G Thirumalai,
>>> Dipankar Sarma <[email protected]>
>>>
>>> Studying the responses from all stake holders, Jay Lan's was the most
>>> encouraging. Peter Chubb prefers the /proc interface due to the text
>>> interface
>>> and ease of parsing. (in our opinion, taskstats can meet the needs
>>> easily
>>> and the getdelays utility can provide the same ease for parsing).
>>> The others did not respond.
>>> Some performance numbers of taskstats were posted at
>>> http://lkml.org/lkml/2006/3/23/141. The result highlights are included
>>> below
>>>
>>> Results highlights
>>>
>>> - Configuring delay accounting adds < 0.5%
>>> overhead in most cases and even reduces overhead
>>> in some cases
>>>
>>> - Enabling delay accounting has similar results
>>> with a maximum overhead of 1.2% for hackbench,
>>> most other overheads < 1% and reduction in
>>> overhead in some cases
>>>
>>> These statistics are _per task_ and can be extended easily by anyone
>>> who wishes to obtain per task data. An example of per task improved
>>> scheduler statistics was mentioned in http://lkml.org/lkml/2006/6/1/381
>>> (I am not sure if the email refers to our per-task statistics). If not,
>>> the new statistics could easily use the taskstats interface.
>>>
>>> These statistics can be used by software product stacks to monitor
>>> usage information about the various tasks they create and control.
>>> I also informally spoke to a group of students (verbally), who were
>>> excited at the possibility of using the per-task statistics to do
>>> dynamic deadline based power management. They want to use the delay data
>>> (CPU and IO) to predict deadlines for a task and then use these results
>>> for dynamically scaling CPU frequency.
>>>
>>>
>>> The ability to monitor the CPU run and delay data and IO delay data is
>>> useful.
>>>
>>> I would request you to consider the inclusion per-task delay accounting
>>> into
>>> 2.6.18.
>>>
>>
>>
>>
>> Andrew,
>>
>> The only other new set of patches to be discussed in this context are the
>> statistics-infrastructure patches from Martin Peschke.
>>
>> That infrastructure cannot meet the needs of delay accounting, CSA
>> etc. because
>> - it only provides "user pull" model of getting stats whereas "kernel
>> push" is
>> needed for delay accounting
>
>
> Doesn't taskstats interface provide "user pull" request-reply model
> also? Serious accounting needs to push accounting data as soon as
> possible.
Yes, I meant to say "kernel push" is also needed for delay accounting.
So taskstats provides both pull and push whereas statistics infrastructure, on
account of use of fs-based interface, provides only user-pull.
>
>> - it uses a relatively slow interface unsuitable for high volumes of
>> data. Each
>> statistic has its own definition, needs to be read separately using
>> ASCII,
>> reading data continuously means open/read/close each time.....all of
>> which is not very conducive to large structures being sent to userspace.
>
>
> Yes, i second the point. It won't be able to catch up the traffic.
>
>> - its oriented towards sampled data whereas taskstats isn't.
>>
>> So, we have a good consensus from existing/potential users of
>> taskstats and would
>> very much appreciate it being included in 2.6.18.
>
>
> Andrew, it has become clear that the community wants to see accounting
> data processing being moved to userspace. Thus there is a need for a
> common accounting interface to provide minimal works at kernel (via
> hooks at fork and exit) and deliver data to userspace.
>
> The delayacct patchset provides a good framework and example that
> i believe CSA/Job can follow and build upon to move most of our work
> to userspace and thus cut off dependency of PAGG. We will submit CSA
> patch soon based on the taskstats interface.
>
> Thanks,
> - jay
>
> P.S. Balbir and Shailabh, Chris Sturtivant will continue the CSA work
> at SGI. Please also cc Chris <[email protected]> in the future.
> Thanks!
Sure.
Thanks,
Shailabh
>
>
>>
>> --Shailabh
>>
>>
>>
>>
>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[Index of Archives]
[Kernel Newbies]
[Netfilter]
[Bugtraq]
[Photo]
[Stuff]
[Gimp]
[Yosemite News]
[MIPS Linux]
[ARM Linux]
[Linux Security]
[Linux RAID]
[Video 4 Linux]
[Linux for the blind]
[Linux Resources]
