simple and efficient way to have bi-directional communication between lua process

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

simple and efficient way to have bi-directional communication between lua process

gary ng
Hi,

Since lua don't have thread support out of the box,
the alternative would be spawning multiple small lua
process if I want some form of multi tasking(I know
there is coroutine but let's say that coroutine
doesn't fit the scenario).

How would I have a simple efficient way of
bi-directional communication between two lua process ?
On, unix platform, using domain socket seems to be
easiest(though I am not sure if luasocket supports it
and I was told that there is no such a thing on
Windows). popen() only creates a one way pipe.

Or must I fallback to the 127.0.0.1 interface ? I see
two problems with that, security and unnecessary
payload(the network layer). Though it has the
advantage that it is network ready by design.

What I want is a relatively portable way(mainly *nix
and Windows) for this need.

regards,


 
____________________________________________________________________________________
The fish are biting. 
Get more visitors on your site using Yahoo! Search Marketing.
http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php

Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

Sam Roberts-2
On Wed, Feb 07, 2007 at 11:25:36PM -0800, gary ng wrote:
> Or must I fallback to the 127.0.0.1 interface ? I see
> two problems with that, security and unnecessary
> payload(the network layer). Though it has the
> advantage that it is network ready by design.
> 
> What I want is a relatively portable way(mainly *nix
> and Windows) for this need.

IPC isn't portable between unix and Windows, only tcp is common. You'll
have to use different methods, and write code that makes them look the
same, I think.

Sam


Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

Asko Kauppi
In reply to this post by gary ng

You don't need to dive into OS specific issues, at all. Threads share a common address space, so all you have to do is serialize Lua data structures into a stream (allocated char array), deserialize it in the other thread. I would definately see a need for this + basic thread creation (portable) + lazy evaluation, all together in an easy to use package.

DBUS, sockets etc. can be done using similar approach, but they'd be useful only if
- communicating with existing non-Lua services
- communicating across process boundaries (another CPU)


gary ng kirjoitti 8.2.2007 kello 9.25:

Hi,

Since lua don't have thread support out of the box,
the alternative would be spawning multiple small lua
process if I want some form of multi tasking(I know
there is coroutine but let's say that coroutine
doesn't fit the scenario).

How would I have a simple efficient way of
bi-directional communication between two lua process ?
On, unix platform, using domain socket seems to be
easiest(though I am not sure if luasocket supports it
and I was told that there is no such a thing on
Windows). popen() only creates a one way pipe.

Or must I fallback to the 127.0.0.1 interface ? I see
two problems with that, security and unnecessary
payload(the network layer). Though it has the
advantage that it is network ready by design.

What I want is a relatively portable way(mainly *nix
and Windows) for this need.

regards,



______________________________________________________________________ ______________
The fish are biting.
Get more visitors on your site using Yahoo! Search Marketing.
http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php


Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

Javier Guerra Giraldez
On Thursday 08 February 2007, Asko Kauppi wrote:
> You don't need to dive into OS specific issues, at all. Threads share
> a common address space, so all you have to do is serialize Lua data
> structures into a stream (allocated char array), deserialize it in
> the other thread. I would definately see a need for this + basic
> thread creation (portable) + lazy evaluation, all together in an easy
> to use package.

what would be easier: writing a thin portable layer over pthreads/winthreads, 
or discovering the hard way that you shouldn't modify the same structure on 
two threads and reinventing semaphores?

-- 
Javier

Attachment: pgp9UhM_55zzP.pgp
Description: PGP signature

Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

gary ng
In reply to this post by Asko Kauppi
--- Asko Kauppi <[hidden email]> wrote:

> 
> You don't need to dive into OS specific issues, at
> all. Threads share a common address space, so all 
> you have to do is serialize Lua data structures 
> into a stream (allocated char array), deserialize 
> it in the other thread. 

Could you elab ? Since I am thinking about using also
lua as the main program, I don't quite understand how
thread would play a role here assume I am using
standard lua.

Though I like the idea of serializing and would like
to get more info/pointer of how to do it.


 
____________________________________________________________________________________
Finding fabulous fares is fun.  
Let Yahoo! FareChase search your favorite travel sites to find flight and hotel bargains.
http://farechase.yahoo.com/promo-generic-14795097

Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

Asko Kauppi
In reply to this post by Javier Guerra Giraldez
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


There's not necessarily any locking involved:

- - Lua data structures are sent serialized; they are not modified by separate threads

- - Control of the thread execution (percentage, done, cancelled) can be done in a way that the two threads don't write on same fields. If only one thread writes, and the other reads, there is no need for locking. Or locking is kind of built into the data usage.

I've done similar earlier and I am aware of semaphores etc. in the traditional way (where they rock). Walking on thin ice here, but thanks for the warning! ;)

- -asko


There's a minimal critical section

Javier Guerra kirjoitti 8.2.2007 kello 21.41:

On Thursday 08 February 2007, Asko Kauppi wrote:
You don't need to dive into OS specific issues, at all. Threads share
a common address space, so all you have to do is serialize Lua data
structures into a stream (allocated char array), deserialize it in
the other thread. I would definately see a need for this + basic
thread creation (portable) + lazy evaluation, all together in an easy
to use package.

what would be easier: writing a thin portable layer over pthreads/ winthreads, or discovering the hard way that you shouldn't modify the same structure on
two threads and reinventing semaphores?

--
Javier

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFy/gyGJtHlJZfjQoRAk4JAKCQalQ/Q1W+vF3puoytr4KoIhPJwQCgiD5J
EMHSK98T3Huedjon8EC+76g=
=sSxV
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

Javier Guerra Giraldez
On Thursday 08 February 2007, Asko Kauppi wrote:
> - Control of the thread execution (percentage, done, cancelled) can
> be done in a way that the two threads don't write on same fields. If
> only one thread writes, and the other reads, there is no need for
> locking. Or locking is kind of built into the data usage.

do you mean a thread-safe FIFO? sure they're great... and usually implemented 
using semaphores.

> There's a minimal critical section

you know critical sections are also a locking operation, right? it's easy to 
implement them using semaphores; or the other way around, if you like.

in all, my point is that if you use threads (and i think the'll be more and 
more inescapable with multiple-core chips getting so comon), then you have to 
decide on a few primitives (a FIFO might be the easiest, i used some on my 
helper threads toolkit), and build your functionality on top of that.

but unfortunately, there are very few multiplatform options.  in LuaThreads, 
Diego wrote a very thin adaptor library to get a subset of a pthreads-like 
API on windows.  it's very minimal, i think only 200 or so lines of C.  once 
you have that, it's easy to build everything else.


-- 
Javier

Attachment: pgptssnnl8Wjc.pgp
Description: PGP signature

Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

Hakki Dogusan
In reply to this post by Asko Kauppi
Hi,

(Sorry, I lost the original message)

Asko Kauppi wrote:
[snipped Asko's reply]
gary ng kirjoitti 8.2.2007 kello 9.25:

Hi,

Since lua don't have thread support out of the box,
the alternative would be spawning multiple small lua
process if I want some form of multi tasking(I know
there is coroutine but let's say that coroutine
doesn't fit the scenario).

How would I have a simple efficient way of
bi-directional communication between two lua process ?
On, unix platform, using domain socket seems to be
easiest(though I am not sure if luasocket supports it
and I was told that there is no such a thing on
Windows). popen() only creates a one way pipe.

Or must I fallback to the 127.0.0.1 interface ? I see
two problems with that, security and unnecessary
payload(the network layer). Though it has the
advantage that it is network ready by design.

What I want is a relatively portable way(mainly *nix
and Windows) for this need.

regards,


If you will choose socket way, then I suggest you to look
at YAMI library (http://www.msobczak.com/prog/yami/).

I'm using it in my dsas library.



--
Regards,
Hakki Dogusan

Reply | Threaded
Open this post in threaded view
|

Re: simple and efficient way to have bi-directional communication between lua process

Daniel Quintela
In reply to this post by gary ng
On miÃ, 2007-02-07 at 23:25 -0800, gary ng wrote:
> Hi,
> 
> Since lua don't have thread support out of the box,
> the alternative would be spawning multiple small lua
> process if I want some form of multi tasking(I know
> there is coroutine but let's say that coroutine
> doesn't fit the scenario).
> 
> How would I have a simple efficient way of
> bi-directional communication between two lua process ?
> On, unix platform, using domain socket seems to be
> easiest(though I am not sure if luasocket supports it
> and I was told that there is no such a thing on
> Windows). popen() only creates a one way pipe.
> 
> Or must I fallback to the 127.0.0.1 interface ? I see
> two problems with that, security and unnecessary
> payload(the network layer). Though it has the
> advantage that it is network ready by design.
> 
> What I want is a relatively portable way(mainly *nix
> and Windows) for this need.
> 
> regards,
> 
> 
>  
> ____________________________________________________________________________________
> The fish are biting. 
> Get more visitors on your site using Yahoo! Search Marketing.
> http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php


You are talking about multiple processes, then you are considering
multiple Lua universes.
LuaTask ( http://luatask.luaforge.net ) is a multithreading addon with
multiple Lua universes and memory queues for inter-universe
communication.

Regards,
DQ