Coroutines, sockets and multiplexing

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Coroutines, sockets and multiplexing

Adrian Sietsma
There has been a fair bit of discussion lately about async socket io and
coroutines.
In an attempt to clarify the issues (and my thoughts) here is a rough
summary, with comments.

Note : for the purpose of this discussion, real-time (<n ms) response is not
required. This avoids issues such as interrupting the gc., etc.

1. Non-blocking versions of blocking api/os calls.
- Difficult, os-specific, and requires background thread(s), with all the
design issues involved (pooling, thread-per-request, etc). Ignored for now.

2. Non blocking file io.
- Not too bad if supported by the os, but still very os-specific.

3. Event-driven callbacks.
see mike pall's event api documentation
<http://lua-users.org/files/wiki_insecure/users/MikePall/event.html>
for a more detailed discussion

   There are 2 fundamental design choices here
  - are the callbacks to share a common Lua universe ?
  - is the main processing loop in Lua or c ?

   The simplest (and fastest ?) model here would be to have the main loop in
c(++), using os threading and events to run each lua callback in it's own
lua state (vm). We found this model perfect for a specific large commercial
task.
nb. the ability to clone a lua state would be lovely in this case, as we
could load all libraries and perform initialisation of one "master" state,
which is then cloned for each child thread.

If we want the main loop to be in Lua, we need an efficient way for Lua to
sleep between events. We can use select() for socket io, but other events,
signals, etc become os-specific.
A single shared vm also increases granularity (response time) : events can
only be serviced between lua instructions. This will suffice for most
applications.
If we want the event handlers to be interruptible, we need to check for
events between lua instructions, and yield to the scheduler as required.
This can be simulated using the debug linehook, but that imposes a fair
amount of overhead - the event test should be in the core lua vm.
The callback handlers otherwise need to be written with multiplexing in
mind, and yield regularly to the scheduler.


4. Multiplexed socket/file io.
- See #2 re file io.

A simple way to handle socket multiplexing is to have a main scheduler based
around select(), and to override the read() and write() functions to yield
back to the scheduler if they would block. This allows re-use of existing
blocking socket code unchanged.

I have built a library which emulates/wraps the luasocket interface, using
this approach; i can thus re-use Diego's http libraries (thanks Diego). This
lib was first developed for a http proxy prototype, and is fine for
light-duty web serving / prototyping. I have tested it with 50 simultaneous
http requests on windows without a problem : it runs into winsock issues
when i try too many requests.

I found it a very nice model for simple multiplexing :

-- sample code ----------------------------
-- a, b are connected sockets
-- this fn is called from a couroutine
function simplecopy(a,b)
   while true do
     local s, err = a:read("*l") -- yields if insufficient data
     if err then return nil, err
     if not s then return true end
     local ret, err = b:write(s.."\n") -- yields until done
     if err then return nil, err
   end
end -- sample ------------------------------

The main drawback with this approach is response time : the scheduler is
only invoked when a coro explicitly yields, or calls one of the yielding io
functions.
It is also difficult to fully exploit native async. io (IOCP in windows).

ps. A wonderful side-effect of using coroutines is the error behaviour - you
effectively get a free "try", and errors can be propgated / handled in a
very flexible manner.

pps For high performance web serving, Apache + FastCGI Lua ?

ppps These comments represent my limited problem set; actual mileage may vary.

Adrian
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines, sockets and multiplexing

Jon Smirl
You want to use epoll on Linux not select. Check out the performance
charts here:
http://www.monkey.org/~provos/libevent/

kqueue is for BSD

--
Jon Smirl
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines, sockets and multiplexing

Javier Guerra Giraldez
In reply to this post by Adrian Sietsma
On Tuesday 31 January 2006 1:12 am, Adrian Sietsma wrote:
>    The simplest (and fastest ?) model here would be to have the main loop
> in c(++), using os threading and events to run each lua callback in it's
> own lua state (vm). We found this model perfect for a specific large
> commercial task.

since there's been a lot of previous work on this subject in c(++); it's
relatively easy for a specific project to choose one model and go with it.

but to make it usefull for everybody, the Lua way would be to get a nice set
of primitives and let the user build his own dispatcher.

i'm trying to write something (currently in the third rewrite since the thread
started) that would provide a single Lua interface to build the dispatcher,
and at the same time, a simple set of C routines to build IO libraries.

at the moment, my model is a bit like this:

the library writer splits each potentially blocking operation in three C
functions:
int (*begin)(lua_State *L, void **userdata);
void (*work)(void *userdata);
void (*end)(lua_State *L, void *userdata);

when the Lua developer calls the function, only the begin C function is
called; it sets up it's userdata, and the framework returns a "future" to the
Lua code.
all futures are contained in a queue, and a set of helper threads call the
work functions asynchronously. this work functions can't modify the Lua
State.  when finished, the framework puts the future in the 'done' queue.
the dispatcher (written in Lua) calls a function that returns the done futures
from this queue (might block if there's nothing there).

with the 'done' future, the Lua code calls a 'finish' method, the end function
is executed, and any data from the operation is returned.

this is (i think) enough to write a simple dispatcher: just wrap the new
nonblocking operations so that they yield out of the coroutine 'thread' with
the future; the dispatcher associates the unfinished future with the thread,
and resumes it when this same future comes out of the done queue.  the
wrapper calls the finish method and returns the result to the original
caller.

i'm still mangled in code; mostly about who gets the responsibility to dealloc
the futures, and if they should be reused for sequential operations.

any opinions?

--
Javier

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines, sockets and multiplexing

Jon Smirl
On 1/31/06, Javier Guerra <[hidden email]> wrote:
> since there's been a lot of previous work on this subject in c(++); it's
> relatively easy for a specific project to choose one model and go with it.

I just noticed this paper on state threads. It is very similar to what
you are doing.
http://state-threads.sourceforge.net/docs/st.html

--
Jon Smirl
[hidden email]