Sharing data between LuaTask tasks...

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Sharing data between LuaTask tasks...

duck
It occurred to me that LuaTask (in which each LuaTask gets its own Lua 
state in a new thread) might be a handy way of writing simple threaded 
servers. However, only number and string objects can be passed in messages 
between tasks.

I was wondering:

1. Is there any reliable way to pass an accept()ed LuaSocket object from 
one LuaTask to another? Can userdata be serialised?

2. If so (and assuming that socket is only ever manipulated in the task to 
which it's passed), is it safe to pass this sort of data between tasks? Is 
LuaSocket itself thread-safe?

Has anyone used LuaTask for handling multiple threads which communicate 
using multiple blocking sockets?


Reply | Threaded
Open this post in threaded view
|

Re: Sharing data between LuaTask tasks...

Javier Guerra Giraldez
On Sunday 13 May 2007, duck wrote:
> Has anyone used LuaTask for handling multiple threads which communicate
> using multiple blocking sockets?

(a bit of self-promotion...)

i haven't used LuaTask precisely because of that limitation on message 
passing; my solution is HTT (Helper Threads Toolkit), it makes easy to write 
non-blocking C modules, so that you can easily poll with a coroutine-based 
Lua scheduler.

as examples, i've written file and TCP I/O modules, as well as a simple (but 
efficient) scheduler.

check it in http://luaforge.net/projects/helper-threads/  but don't waste time 
with the packaged r-4, go directly to the CVS head.


-- 
Javier

Attachment: pgp32o6QdZ8Gk.pgp
Description: PGP signature

Reply | Threaded
Open this post in threaded view
|

Re: Sharing data between LuaTask tasks...

Daniel Quintela
In reply to this post by duck
duck escribió:
It occurred to me that LuaTask (in which each LuaTask gets its own Lua state in a new thread) might be a handy way of writing simple threaded servers. However, only number and string objects can be passed in messages between tasks.

I was wondering:

1. Is there any reliable way to pass an accept()ed LuaSocket object from one LuaTask to another? Can userdata be serialised?

2. If so (and assuming that socket is only ever manipulated in the task to which it's passed), is it safe to pass this sort of data between tasks? Is LuaSocket itself thread-safe?

Has anyone used LuaTask for handling multiple threads which communicate using multiple blocking sockets?

There is no general answer about passing userdata.

But in the LuaSocket case... these object has two methods to manipulate the underlying native machinery: setfd and getfd.
You can get the native socket with getfd and pass it as an integer value.
If you use skt:setfd( -1 ) the native socket is not closed when skt is garbage collected.

You can download a little old examples using this trick at: http://www.soongsoft.com/lua/LuaTaskExamples.tar.gz

Best regards,
Daniel


Reply | Threaded
Open this post in threaded view
|

Re: Sharing data between LuaTask tasks...

duck
In reply to this post by duck
> But in the LuaSocket case... these object has two methods to manipulate 
> the underlying native machinery: setfd and getfd.
> You can get the native socket with getfd and pass it as an integer value.
> If you use skt:setfd( -1 ) the native socket is not closed when skt is 
> garbage collected.
> 
> You can download a little old examples using this trick at: 
> http://www.soongsoft.com/lua/LuaTaskExamples.tar.gz

This is rather cool. I see that you do a bind-listen-close, then switch 
the socket's fd, then call accept to get a new "tcp{client}" class socket
with which you can communicate.

I presume the initial bind-listen-close is only there to switch the 
cloned socket's class ID to "tcp{server}" so that it will be legal to 
LuaSocket when passed to accept()?

If I wanted to call accept() outside the child task and to clone the already-
accept()ed socket inside the child, I guess I'd have to trick LuaSocket into
providing me with a "tcp{client}" class socket into which I could stuff
the accept()ed socket's fd? And the only way I can see to do that is via
a successful connect() or accept()...in other words, it's not really possible.

In your examples, you mention:

    --  I hacked LuaSocket to simplify this but,
    --  in order to use a stock distribution of LuaSocket,
    --  I'm going to use code based in a kind suggestion
    --  from Diego Nehab.

Any chance of posting the patches you wrote here so I can have a look at 
them? I don't mind the bind-listen-close trick but it would be nice to 
avoid having to open listneing sockets unnecessarily. Wiould help to avoid
endpoint firewall nuisances and complaints from admins about "large amounts
of socket activity on your PC" :-)

I thought of simply adding a function to tcp.c called, say, "setclass()"
which would take a master socket and blindly change its class. I can't see
any way to tweak the metatable of a userdata from inside Lua -- and I 
suspect that there isn't one (and that this is deliberate to prevent 
disasters).


Reply | Threaded
Open this post in threaded view
|

Re: Sharing data between LuaTask tasks...

Daniel Quintela
duck escribió:
In your examples, you mention:

    --  I hacked LuaSocket to simplify this but,
    --  in order to use a stock distribution of LuaSocket,
    --  I'm going to use code based in a kind suggestion
    --  from Diego Nehab.

Any chance of posting the patches you wrote here so I can have a look at them? I don't mind the bind-listen-close trick but it would be nice to avoid having to open listneing sockets unnecessarily. Wiould help to avoid
endpoint firewall nuisances and complaints from admins about "large amounts
of socket activity on your PC" :-)

Use the attached patch with care because I had to update it from "tcp.c,v 1.31" to "tcp.c,v 1.41".
I had no time to test it with the current LuaSocket release.

Best regards,
Daniel

--- tcp.c	2006-04-27 00:23:21.000000000 -0300
+++ hack-tcp.c	2007-05-19 13:41:42.000000000 -0300
@@ -3,6 +3,8 @@
 * LuaSocket toolkit
 *
 * RCS ID: $Id: tcp.c,v 1.41 2005/10/07 04:40:59 diego Exp $
+*
+* attach/detach hack by Daniel Quintela
 \*=========================================================================*/
 #include <string.h> 
 
@@ -19,6 +21,8 @@
 * Internal function prototypes
 \*=========================================================================*/
 static int global_create(lua_State *L);
+static int global_attach(lua_State *L);	// attach/detach hack
+static int global_attachserver(lua_State *L);	// attach/detach hack
 static int meth_connect(lua_State *L);
 static int meth_listen(lua_State *L);
 static int meth_bind(lua_State *L);
@@ -31,6 +35,7 @@
 static int meth_receive(lua_State *L);
 static int meth_accept(lua_State *L);
 static int meth_close(lua_State *L);
+static int meth_detach(lua_State *L);	// attach/detach hack
 static int meth_setoption(lua_State *L);
 static int meth_settimeout(lua_State *L);
 static int meth_getfd(lua_State *L);
@@ -60,6 +65,7 @@
     {"setsockname", meth_bind},
     {"settimeout",  meth_settimeout},
     {"shutdown",    meth_shutdown},
+    {"detach",      meth_detach},	// attach/detach hack
     {NULL,          NULL}
 };
 
@@ -75,6 +81,8 @@
 /* functions in library namespace */
 static luaL_reg func[] = {
     {"tcp", global_create},
+    {"attach", global_attach},	// attach/detach hack
+    {"attachserver", global_attachserver},	// attach/detach hack
     {NULL, NULL}
 };
 
@@ -237,6 +245,17 @@
 }
 
 /*-------------------------------------------------------------------------*\
+* Detaches socket used by object - attach/detach hack
+\*-------------------------------------------------------------------------*/
+static int meth_detach(lua_State *L)
+{
+    p_tcp tcp = (p_tcp) auxiliar_checkgroup(L, "tcp{any}", 1);
+    tcp->sock = SOCK_INVALID;
+    lua_pushnumber(L, 1);
+    return 1;
+}
+
+/*-------------------------------------------------------------------------*\
 * Puts the sockt in listen mode
 \*-------------------------------------------------------------------------*/
 static int meth_listen(lua_State *L)
@@ -337,3 +356,38 @@
         return 2;
     }
 }
+
+/*-------------------------------------------------------------------------*\
+* Attaches a client tcp object - attach/detach hack
+\*-------------------------------------------------------------------------*/
+int global_attach(lua_State *L)
+{
+	t_socket skt = ( t_socket) luaL_checknumber(L, 1);
+    p_tcp client = lua_newuserdata(L, sizeof(t_tcp));
+    auxiliar_setclass(L, "tcp{client}", -1);
+    client->sock = skt;
+    /* initialize remaining structure fields */
+    io_init(&client->io, (p_send) socket_send, (p_recv) socket_recv,
+		(p_error) socket_ioerror, &client->sock);
+    timeout_init(&client->tm, -1, -1);
+    buffer_init(&client->buf, &client->io, &client->tm);
+    return 1;
+}
+
+/*-------------------------------------------------------------------------*\
+* Attaches a server tcp object - attach/detach hack
+\*-------------------------------------------------------------------------*/
+int global_attachserver(lua_State *L)
+{
+	t_socket skt = ( t_socket) luaL_checknumber(L, 1);
+    p_tcp server = lua_newuserdata(L, sizeof(t_tcp));
+    auxiliar_setclass(L, "tcp{server}", -1);
+    server->sock = skt;
+    /* initialize remaining structure fields */
+    io_init(&server->io, (p_send) socket_send, (p_recv) socket_recv,
+		(p_error) socket_ioerror, &server->sock);
+    timeout_init(&server->tm, -1, -1);
+    buffer_init(&server->buf, &server->io, &server->tm);
+    return 1;
+}
+
Reply | Threaded
Open this post in threaded view
|

Re: Sharing data between LuaTask tasks...

Asko Kauppi

I'd be willing to help anyone following this thread to combine LuaSocket with Lua Lanes.

More info: http://luaforge.net/docman/index.php? group_id=265&selected_doc_group_id=292&language_id=1

A new version, having wait timeouts implemented, is currently in LuaForge:
http://luaforge.net/frs/shownotes.php?release_id=732

Currently, Lanes supports passing userdata as arguments and return values (copied stack-to-stack), but not as message values (serialized). Lanes is intended to be used with stock Lua, and stock modules s.a. LuaSocket.

-asko


Daniel Quintela kirjoitti 19.5.2007 kello 20:00:

duck escribió:
In your examples, you mention:

    --  I hacked LuaSocket to simplify this but,
    --  in order to use a stock distribution of LuaSocket,
    --  I'm going to use code based in a kind suggestion
    --  from Diego Nehab.

Any chance of posting the patches you wrote here so I can have a look at them? I don't mind the bind-listen-close trick but it would be nice to avoid having to open listneing sockets unnecessarily. Wiould help to avoid endpoint firewall nuisances and complaints from admins about "large amounts
of socket activity on your PC" :-)


Use the attached patch with care because I had to update it from "tcp.c,v 1.31" to "tcp.c,v 1.41".
I had no time to test it with the current LuaSocket release.

Best regards,
Daniel

--- tcp.c	2006-04-27 00:23:21.000000000 -0300
+++ hack-tcp.c	2007-05-19 13:41:42.000000000 -0300
@@ -3,6 +3,8 @@
 * LuaSocket toolkit
 *
 * RCS ID: $Id: tcp.c,v 1.41 2005/10/07 04:40:59 diego Exp $
+*
+* attach/detach hack by Daniel Quintela
\*==================================================================== =====*/
 #include <string.h>

@@ -19,6 +21,8 @@
 * Internal function prototypes
\*==================================================================== =====*/
 static int global_create(lua_State *L);
+static int global_attach(lua_State *L);	// attach/detach hack
+static int global_attachserver(lua_State *L);	// attach/detach hack
 static int meth_connect(lua_State *L);
 static int meth_listen(lua_State *L);
 static int meth_bind(lua_State *L);
@@ -31,6 +35,7 @@
 static int meth_receive(lua_State *L);
 static int meth_accept(lua_State *L);
 static int meth_close(lua_State *L);
+static int meth_detach(lua_State *L);	// attach/detach hack
 static int meth_setoption(lua_State *L);
 static int meth_settimeout(lua_State *L);
 static int meth_getfd(lua_State *L);
@@ -60,6 +65,7 @@
     {"setsockname", meth_bind},
     {"settimeout",  meth_settimeout},
     {"shutdown",    meth_shutdown},
+    {"detach",      meth_detach},	// attach/detach hack
     {NULL,          NULL}
 };

@@ -75,6 +81,8 @@
 /* functions in library namespace */
 static luaL_reg func[] = {
     {"tcp", global_create},
+    {"attach", global_attach},	// attach/detach hack
+    {"attachserver", global_attachserver},	// attach/detach hack
     {NULL, NULL}
 };

@@ -237,6 +245,17 @@
 }

/ *--------------------------------------------------------------------- ----*\
+* Detaches socket used by object - attach/detach hack
+ \*-------------------------------------------------------------------- -----*/
+static int meth_detach(lua_State *L)
+{
+    p_tcp tcp = (p_tcp) auxiliar_checkgroup(L, "tcp{any}", 1);
+    tcp->sock = SOCK_INVALID;
+    lua_pushnumber(L, 1);
+    return 1;
+}
+
+/ *--------------------------------------------------------------------- ----*\
 * Puts the sockt in listen mode
\*-------------------------------------------------------------------- -----*/
 static int meth_listen(lua_State *L)
@@ -337,3 +356,38 @@
         return 2;
     }
 }
+
+/ *--------------------------------------------------------------------- ----*\
+* Attaches a client tcp object - attach/detach hack
+ \*-------------------------------------------------------------------- -----*/
+int global_attach(lua_State *L)
+{
+	t_socket skt = ( t_socket) luaL_checknumber(L, 1);
+    p_tcp client = lua_newuserdata(L, sizeof(t_tcp));
+    auxiliar_setclass(L, "tcp{client}", -1);
+    client->sock = skt;
+    /* initialize remaining structure fields */
+    io_init(&client->io, (p_send) socket_send, (p_recv) socket_recv,
+		(p_error) socket_ioerror, &client->sock);
+    timeout_init(&client->tm, -1, -1);
+    buffer_init(&client->buf, &client->io, &client->tm);
+    return 1;
+}
+
+/ *--------------------------------------------------------------------- ----*\
+* Attaches a server tcp object - attach/detach hack
+ \*-------------------------------------------------------------------- -----*/
+int global_attachserver(lua_State *L)
+{
+	t_socket skt = ( t_socket) luaL_checknumber(L, 1);
+    p_tcp server = lua_newuserdata(L, sizeof(t_tcp));
+    auxiliar_setclass(L, "tcp{server}", -1);
+    server->sock = skt;
+    /* initialize remaining structure fields */
+    io_init(&server->io, (p_send) socket_send, (p_recv) socket_recv,
+		(p_error) socket_ioerror, &server->sock);
+    timeout_init(&server->tm, -1, -1);
+    buffer_init(&server->buf, &server->io, &server->tm);
+    return 1;
+}
+