Nonblocking stdio

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Nonblocking stdio

Jim Mellander
I have a lua application on unix which reads from stdio - there is some
significant work it could do if there is no input.  I thought about
kludging up a shim which would read from stdio, and write to a socket,
from which my app could read from in non-blocking mode using the
luasocket facilities.

It would be nice if the stdio facilities could use the luasocket
'select' interface, etc.  Any ideas?


--
Jim Mellander
Incident Response Manager
Computer Protection Program
Lawrence Berkeley National Laboratory
(510) 486-7204

Your fortune for today is:

QOTD:
        "I'll listen to reason when it comes out on CD."

Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Diego Nehab-3
Hi,

> I have a lua application on unix which reads from stdio - there is some
> significant work it could do if there is no input.  I thought about kludging
> up a shim which would read from stdio, and write to a socket, from which my
> app could read from in non-blocking mode using the luasocket facilities.
>
> It would be nice if the stdio facilities could use the luasocket 'select'
> interface, etc.  Any ideas?

It would be nice, indeed.

Unfortunately, on Windows, select only supports sockets.
Even on Unix, passing the file descriptor used by a stdio
FILE * to select would be problematic due to input/output buffering.
LuaSocket does its own buffering, so I know exactly what is
going on.

[]s,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Luiz Henrique de Figueiredo
> Unfortunately, on Windows, select only supports sockets.

Does libevent work on Windows?
Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Diego Nehab-3
Hi,

>> Unfortunately, on Windows, select only supports sockets.
>
> Does libevent work on Windows?

I don't know about libevent.

But there are other ways to wait on sockets and files
on Windows that do not involve calling the select function
and are *very* efficient. What I am saying is that the select
function on Windows, which is provided by WinSock, only supports
sockets.

People went through a lot of trouble trying to provide a
compatible implementation of select in cygwin. Their
approach, if I remember correctly, involves sorting all
"descriptors" into groups (sockets, files etc), then
spawning one auxiliar thread to wait on each group, then
waiting on these threads with a timeout, and finally
consolidating the results.

I am sure whoever implemented this had no other choice and
needed a long shower when he/she was done.

Regards,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Luiz Henrique de Figueiredo
> I don't know about libevent.

http://monkey.org/~provos/libevent/

It seems to cater for Windows, though I have no experience with either libevent
or Windows (thank heavens for that!)
--lhf
Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Lisa Parratt
In reply to this post by Diego Nehab-3
Diego Nehab wrote:
>
> Unfortunately, on Windows, select only supports sockets.

This is highly misleading. Yes, select() only supports sockets on Windows.

However, that's simply because you're using the wrong function - you
should be using WaitForObjects(). When it comes to blocking on events,
Windows makes the Unix API look like the utter toy it is.

--
Lisa

Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdio

Adrian Sietsma
Lisa Parratt wrote:

> This is highly misleading. Yes, select() only supports sockets on Windows.
>
> However, that's simply because you're using the wrong function - you
> should be using WaitForObjects(). When it comes to blocking on events,
> Windows makes the Unix API look like the utter toy it is.
>
A clarification - the windows WaitForMultipleObjects call you refer to still
has a limit of MAXIMUM_WAIT_OBJECTS handles : that number is, surprise
surprise, 64.

Real high-performance i/o on windows requires io completion ports : my xp
box runs out of buffers for o/lapped io at 3,976 open sockets  x 2
(client+server apps).

The real design issue anyway is how to structure Lua apps around nonblocking
io / events : how to schedule coroutines, associate them with events, add
extension libraries etc.

Adrian
Reply | Threaded
Open this post in threaded view
|

Re: Nonblocking stdiou

Diego Nehab-3
In reply to this post by Lisa Parratt
Hi,

>> Unfortunately, on Windows, select only supports sockets.
>
> This is highly misleading. Yes, select() only supports sockets on Windows.

Hey! Don't you think it would have been even more misleading
if I answered the original post by saying it is all a
"simple" matter of changing Lua and LuaSocket to use
WaitForMultipleObjects, and that whoever wrote Lua and
LuaSocket should have done that in the first place, what
were they thinking. :)

If you really want to do a good job on Windows, you have to
use I/O completion ports. Unfortunately, this forces the
users to changes the way they write code. The code becomes
event based. We are discussing this for LuaSocket 3.0,
but I am not sure yet if we will come up with a Simple
Unified Solution that will for for everyone.

Regards,
Diego.