communicating sequential processes

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

communicating sequential processes

Phoenix Sol
Olá lua list,

I feel like I am standing on a precipice here, contemplating lua, and running out of reasons not to dive off...

Is it safe to assume that it would be relatively easy to implement 'message-passing-style-scheduling' in lua, where one coroutine yields to another by passing a message to it? (without accumulating stack frames?!)

Can this be done in lua, without changes to the interpreter?

What I am really after, is a lua implementation of Stackless Python's 'Channels'...

Obrigado!

Phoenix Sol
Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Peter Cawley
On Sat, Apr 25, 2009 at 4:24 PM, Phoenix Sol <[hidden email]> wrote:
> Is it safe to assume that it would be relatively easy to implement
> 'message-passing-style-scheduling' in lua, where one coroutine yields to
> another by passing a message to it? (without accumulating stack frames?!)

coroutine.yield can be passed whatever arguments you want, which
become additional return values of the call to coroutine.resume which
caused the coroutine to run. This allows for a coroutine to pass
values as it yields. Also note that extra arguments to
coroutine.resume will become the return values of coroutine.yield in
the resumed coroutine, allowing you to pass values as you resume.
Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Phoenix Sol
Oh, cool Peter. I know I need to 'rtfm' before I get carried away with these questions (hope I haven't annoyed anyone yet)... but just one more before I do that, please:

    Can the args to coroutine.yield be used to determine which coro should be run next?

Thanks!

On Sat, Apr 25, 2009 at 8:42 AM, Peter Cawley <[hidden email]> wrote:
On Sat, Apr 25, 2009 at 4:24 PM, Phoenix Sol <[hidden email]> wrote:
> Is it safe to assume that it would be relatively easy to implement
> 'message-passing-style-scheduling' in lua, where one coroutine yields to
> another by passing a message to it? (without accumulating stack frames?!)

coroutine.yield can be passed whatever arguments you want, which
become additional return values of the call to coroutine.resume which
caused the coroutine to run. This allows for a coroutine to pass
values as it yields. Also note that extra arguments to
coroutine.resume will become the return values of coroutine.yield in
the resumed coroutine, allowing you to pass values as you resume.

Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Asko Kauppi
In reply to this post by Phoenix Sol

Have you looked at Lanes?

http://luaforge.net/projects/lanes/
http://kotisivu.dnainternet.net/askok/bin/lanes/

It is a loadable module that provides Lua state separation, message  
passing, shared table storage etc. It is not using coroutines for  
this, but individual Lua states run on OS threads.

-asko


Phoenix Sol kirjoitti 25.4.2009 kello 18:24:

> Olá lua list,
>
> I feel like I am standing on a precipice here, contemplating lua,  
> and running out of reasons not to dive off...
>
> Is it safe to assume that it would be relatively easy to implement  
> 'message-passing-style-scheduling' in lua, where one coroutine  
> yields to another by passing a message to it? (without accumulating  
> stack frames?!)
>
> Can this be done in lua, without changes to the interpreter?
>
> What I am really after, is a lua implementation of Stackless  
> Python's 'Channels'...
>
> Obrigado!
>
> Phoenix Sol

Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Phoenix Sol
No, but I am now; thanks :)

It's really refreshing to 'rethink things'... I've become accustomed to thinking in terms of processes and coroutines, because of Python's 'global interpreter lock' (present in Stackless, btw); I'm excited by the idea of using threads and coroutines instead... among other things ;-)


On Sat, Apr 25, 2009 at 9:38 AM, Asko Kauppi <[hidden email]> wrote:

Have you looked at Lanes?

http://luaforge.net/projects/lanes/
http://kotisivu.dnainternet.net/askok/bin/lanes/

It is a loadable module that provides Lua state separation, message passing, shared table storage etc. It is not using coroutines for this, but individual Lua states run on OS threads.

-asko


Phoenix Sol kirjoitti 25.4.2009 kello 18:24:


Olá lua list,

I feel like I am standing on a precipice here, contemplating lua, and running out of reasons not to dive off...

Is it safe to assume that it would be relatively easy to implement 'message-passing-style-scheduling' in lua, where one coroutine yields to another by passing a message to it? (without accumulating stack frames?!)

Can this be done in lua, without changes to the interpreter?

What I am really after, is a lua implementation of Stackless Python's 'Channels'...

Obrigado!

Phoenix Sol


Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Mark Hamburg
In reply to this post by Phoenix Sol
The communication channel is limited in what it can pass, but you might want to read the paper on LuaProc:


LuaForge release at:


That said, it's fairly straightforward to implement this sort of support using Lua coroutines. The chief issue that comes into play in an implementation is that if you want to allow other uses of coroutines beyond yielding back to the scheduler, you need to patch the implementation of yield and resume so that they recognize when a yield needs to work its way back to the scheduler and when they just represent a localized coroutine interaction. I don't know whether there are any publicly available sources that demonstrate that. The basic idea is to extend yield by passing a target as the first item and have resume look at the results and continue yielding if it wasn't meant to receive the value (and then pass the resumption from that yield back down).

Mark

Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Phoenix Sol
Alright, thanks Mark :)

I'll try to stop bugging you guys for a while; I'm totally sold on learning lua ;-)


On Sat, Apr 25, 2009 at 10:04 AM, Mark Hamburg <[hidden email]> wrote:
The communication channel is limited in what it can pass, but you might want to read the paper on LuaProc:


LuaForge release at:


That said, it's fairly straightforward to implement this sort of support using Lua coroutines. The chief issue that comes into play in an implementation is that if you want to allow other uses of coroutines beyond yielding back to the scheduler, you need to patch the implementation of yield and resume so that they recognize when a yield needs to work its way back to the scheduler and when they just represent a localized coroutine interaction. I don't know whether there are any publicly available sources that demonstrate that. The basic idea is to extend yield by passing a target as the first item and have resume look at the results and continue yielding if it wasn't meant to receive the value (and then pass the resumption from that yield back down).

Mark


Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Asko Kauppi
In reply to this post by Mark Hamburg

I'm a fan of LuaProc's approach, as well.

The basic difference to Lanes that I can recall was not having a 1:1  
relationship to OS threads. That makes LuaProc way more scalable.

It does not matter if one only wants 10+ threads (Lua states) but it  
starts to matter in 100's and 1000's.

Someone could merge the two efforts, s.a. teaching Lanes to use  
coroutines so that the N:M mapping becomes possible. Unfortunately, I  
won't be having much time in the future on Lua any more.

-asko


Mark Hamburg kirjoitti 25.4.2009 kello 20:04:

> The communication channel is limited in what it can pass, but you  
> might want to read the paper on LuaProc:
>
> Exploring Lua for Concurrent Programming
>
> LuaForge release at:
>
> LuaForge: luaproc: Project Info
>
> That said, it's fairly straightforward to implement this sort of  
> support using Lua coroutines. The chief issue that comes into play  
> in an implementation is that if you want to allow other uses of  
> coroutines beyond yielding back to the scheduler, you need to patch  
> the implementation of yield and resume so that they recognize when a  
> yield needs to work its way back to the scheduler and when they just  
> represent a localized coroutine interaction. I don't know whether  
> there are any publicly available sources that demonstrate that. The  
> basic idea is to extend yield by passing a target as the first item  
> and have resume look at the results and continue yielding if it  
> wasn't meant to receive the value (and then pass the resumption from  
> that yield back down).
>
> Mark
>

Reply | Threaded
Open this post in threaded view
|

Re: communicating sequential processes

Javier Guerra Giraldez
Asko Kauppi wrote:

>
> I'm a fan of LuaProc's approach, as well.
>
> The basic difference to Lanes that I can recall was not having a 1:1  
> relationship to OS threads. That makes LuaProc way more scalable.
>
> It does not matter if one only wants 10+ threads (Lua states) but it  
> starts to matter in 100's and 1000's.
>
> Someone could merge the two efforts, s.a. teaching Lanes to use  
> coroutines so that the N:M mapping becomes possible. Unfortunately, I  
> won't be having much time in the future on Lua any more.

that's what my sample scheduler on Helper Threads Toolkit does, it manages a number of C helper threads, and shuffles another number of Lua 'threads' (coroutine based).

--
Javier