Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

[Davide Libenzi <davidel@xmailserver.org>]

On Fri, 23 Feb 2007, Evgeniy Polyakov wrote:

> I was not clear - I meant why do we need to do that when we can run the
> same code in userspace? And better if we can have non-blocking dataflows
> and number of threads equal to number of processors...

I've a userspace library that does exactly that (GUASI - GPL code avail if 
you want, but no man page written yet). It uses a pool of threads and 
queue requests. I've a bench that crawl through a directory a read files. 
The sync against async performance sucks. You can't do the cachehit 
optimization in userspace. With network stuff could prolly do better 
(since network is more heavily towards async), but still. 




> I started a week of writing without russian-english dictionary, so
> expect some troubles in communications with me :)
> 
> I said that about kernel design - when we have thousand(s) of threads,
> which do the work - if number of context switches is small (i.e. when
> operations mostly do not block), then it is ok (although 'ps' output 
> with threads can scary a grandma).
> It is also ok to say - 'hey, Linux has so easy AIO model, so that
> everyone should switch and start using it and do not care about problems 
> associated with multi-threaded programming with high concurrency', 
> but, in my opinion, both that cases can not cover all (and most of) 
> the usage cases.
> 
> To eat my hat (or force others to do the same) I'm preparing a tree for 
> threadlet test - I plan to write a trivial web server
> (accept/recv/send/sendfile in one threadlet function) and give it a try
> soon.

Funny, I lazily started doing the same thing last weekend (than I had to 
stop, since real job kicked in ;). I wanted to compare a fully MT trivial 
HTTP server:

http://www.xmailserver.org/thrhttp.c

with one that is event-driven (epoll) and coroutine based. This one will 
only be compared for memory-content delivery, since it has no async vfs 
capabilities. They both support the special "/mem-XXXX" url, that allows 
an HTTP loader to request a given content size.
I also have a epoll+coroutine HTTP loader (that works around httperf 
limitations).
Then, I wanted to compare the above, with one that is epoll+GUASI+coroutine 
based (basically a userpace-only thingy).
I've the code for all the above.
Finally, with one that is epoll+syslet+coroutine based (no code for this 
yet - but it should be a easy port from the GUASI one).
Keep in mind though, that a threadlet solution doing accept/recv/send/sendfile
is becoming blazingly similar to a full MT solution.
I can only immagine the thunders and flames that Larry would throw at us 
for using all those threads :D




- Davide


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/