Nonblocking stdio

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Nonblocking stdio

Jim Mellander
I have a lua application on unix which reads from stdio - there is some significant work it could do if there is no input. I thought about kludging up a shim which would read from stdio, and write to a socket, from which my app could read from in non-blocking mode using the luasocket facilities.

It would be nice if the stdio facilities could use the luasocket 'select' interface, etc. Any ideas?


--
Jim Mellander
Incident Response Manager
Computer Protection Program
Lawrence Berkeley National Laboratory
(510) 486-7204

Your fortune for today is:

QOTD:
	"I'll listen to reason when it comes out on CD."


Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Diego Nehab-3
Hi,

I have a lua application on unix which reads from stdio - there is some significant work it could do if there is no input. I thought about kludging up a shim which would read from stdio, and write to a socket, from which my app could read from in non-blocking mode using the luasocket facilities.

It would be nice if the stdio facilities could use the luasocket 'select' interface, etc. Any ideas?

It would be nice, indeed.

Unfortunately, on Windows, select only supports sockets.
Even on Unix, passing the file descriptor used by a stdio FILE * to select would be problematic due to input/output buffering.
LuaSocket does its own buffering, so I know exactly what is
going on.

[]s,
Diego.

Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Luiz Henrique de Figueiredo
> Unfortunately, on Windows, select only supports sockets.

Does libevent work on Windows?

Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Diego Nehab-3
Hi,

Unfortunately, on Windows, select only supports sockets.

Does libevent work on Windows?

I don't know about libevent.

But there are other ways to wait on sockets and files
on Windows that do not involve calling the select function
and are *very* efficient. What I am saying is that the select function on Windows, which is provided by WinSock, only supports sockets.

People went through a lot of trouble trying to provide a
compatible implementation of select in cygwin. Their
approach, if I remember correctly, involves sorting all
"descriptors" into groups (sockets, files etc), then
spawning one auxiliar thread to wait on each group, then waiting on these threads with a timeout, and finally consolidating the results.

I am sure whoever implemented this had no other choice and
needed a long shower when he/she was done.

Regards,
Diego.

Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Luiz Henrique de Figueiredo
> I don't know about libevent.

http://monkey.org/~provos/libevent/

It seems to cater for Windows, though I have no experience with either libevent
or Windows (thank heavens for that!)
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Lisa Parratt
In reply to this post by Diego Nehab-3
Diego Nehab wrote:

Unfortunately, on Windows, select only supports sockets.

This is highly misleading. Yes, select() only supports sockets on Windows.

However, that's simply because you're using the wrong function - you should be using WaitForObjects(). When it comes to blocking on events, Windows makes the Unix API look like the utter toy it is.

--
Lisa


Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Adrian Sietsma
Lisa Parratt wrote:

This is highly misleading. Yes, select() only supports sockets on Windows.

However, that's simply because you're using the wrong function - you should be using WaitForObjects(). When it comes to blocking on events, Windows makes the Unix API look like the utter toy it is.

A clarification - the windows WaitForMultipleObjects call you refer to still has a limit of MAXIMUM_WAIT_OBJECTS handles : that number is, surprise surprise, 64.

Real high-performance i/o on windows requires io completion ports : my xp box runs out of buffers for o/lapped io at 3,976 open sockets x 2 (client+server apps).

The real design issue anyway is how to structure Lua apps around nonblocking io / events : how to schedule coroutines, associate them with events, add extension libraries etc.

Adrian

Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdiou

Diego Nehab-3
In reply to this post by Lisa Parratt
Hi,

Unfortunately, on Windows, select only supports sockets.

This is highly misleading. Yes, select() only supports sockets on Windows.

Hey! Don't you think it would have been even more misleading
if I answered the original post by saying it is all a
"simple" matter of changing Lua and LuaSocket to use
WaitForMultipleObjects, and that whoever wrote Lua and
LuaSocket should have done that in the first place, what
were they thinking. :)

If you really want to do a good job on Windows, you have to
use I/O completion ports. Unfortunately, this forces the
users to changes the way they write code. The code becomes
event based. We are discussing this for LuaSocket 3.0,
but I am not sure yet if we will come up with a Simple
Unified Solution that will for for everyone.

Regards,
Diego.