Cooperative timeslicing?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Cooperative timeslicing?

Are Leistad
As we know it's possible to use lua_resume() with lua_sethook() to get cooperative timeslicing of sorts for a Lua thread, and even implement multiple threads with a Lua script. Of course it's not particularly efficient, and the execution of the global state can't be timesliced in this way.
 
Having an explicit mechanism for cooperative timeslicing of the VM without the overhead of the debug hook, and for all cases of execution, would be very useful. This might be achieved by pcall/resume functions that set an internal flag to enable a polled mode for the VM, e.g. lua_ppcall(), lua_presume() etc.. These would then return immediately after having prepared the Lua state for polled execution, and then there might be a function to drive the VM execution a few instructions at a time by polling. So something like:
 
int status = lua_ppcall( L, ... );
while( status == LUA_CONTINUE )
  {
    status = lua_pollvm( L, number_of_instructions );
  }
 
Such a cooperative timeslicing method would allow any number of Lua scripts/threads to run within a single host thread, allowing easy and inexpensive multithreading, synchronisation, aborting or timing out scripts etc.. Library functions called from Lua would still block of course, unlike when running a Lua state in a dedicated preemptive thread, but that might be acceptable. It could hopefully be quite efficient and easy to implement and could open up for new ways of using Lua. Any thoughts?
 



Denne e-posten er fri for virus og uønsket programvare fordi avast! Antivirus beskyttelsen er aktiv.


Reply | Threaded
Open this post in threaded view
|

Re: Cooperative timeslicing?

steve donovan
On Tue, Feb 3, 2015 at 7:05 PM, Are Leistad <[hidden email]> wrote:
As we know it's possible to use lua_resume() with lua_sethook() to get cooperative timeslicing of sorts for a Lua thread, and even implement multiple threads with a Lua script. Of course it's not particularly efficient, and the execution of the global state can't be timesliced in this way.

But how this is different from Lua coroutines with an appropriate scheduler?
 

Reply | Threaded
Open this post in threaded view
|

Re: Cooperative timeslicing?

aryajur
In reply to this post by Are Leistad
I had written a small C module to allow running multiple Lua scripts by executing each one for a certain number of steps, also allowing data passing from the parent script to the child scripts. But it uses hooks in the C API. It is here http://www.neukleus.com/LuaStepper.html

Milind


On Tue, Feb 3, 2015 at 9:05 AM, Are Leistad <[hidden email]> wrote:
As we know it's possible to use lua_resume() with lua_sethook() to get cooperative timeslicing of sorts for a Lua thread, and even implement multiple threads with a Lua script. Of course it's not particularly efficient, and the execution of the global state can't be timesliced in this way.
 
Having an explicit mechanism for cooperative timeslicing of the VM without the overhead of the debug hook, and for all cases of execution, would be very useful. This might be achieved by pcall/resume functions that set an internal flag to enable a polled mode for the VM, e.g. lua_ppcall(), lua_presume() etc.. These would then return immediately after having prepared the Lua state for polled execution, and then there might be a function to drive the VM execution a few instructions at a time by polling. So something like:
 
int status = lua_ppcall( L, ... );
while( status == LUA_CONTINUE )
  {
    status = lua_pollvm( L, number_of_instructions );
  }
 
Such a cooperative timeslicing method would allow any number of Lua scripts/threads to run within a single host thread, allowing easy and inexpensive multithreading, synchronisation, aborting or timing out scripts etc.. Library functions called from Lua would still block of course, unlike when running a Lua state in a dedicated preemptive thread, but that might be acceptable. It could hopefully be quite efficient and easy to implement and could open up for new ways of using Lua. Any thoughts?
 



Denne e-posten er fri for virus og uønsket programvare fordi avast! Antivirus beskyttelsen er aktiv.



Reply | Threaded
Open this post in threaded view
|

Re: Cooperative timeslicing?

Are Leistad
In reply to this post by steve donovan
From: steve donovan
Sent: Tuesday, February 03, 2015 7:27 PM
>> On Tue, Feb 3, 2015 at 7:05 PM, Are Leistad <[hidden email]> wrote:
>> As we know it's possible to use lua_resume() with lua_sethook() to get
>> cooperative timeslicing of sorts for a Lua thread, and even implement
>> multiple threads with a Lua script. Of course it's not particularly
>> efficient, and the execution of the global state can't be timesliced
>> in this way.
>
> But how this is different from Lua coroutines with an appropriate scheduler?
 
The first paragraph is talking about that, if I understand you correctly(?) I even use this to great effect in an application where performance isn't critical.
 
As the posting goes on to talk about, I'm thinking about a general mechanism to cooperatively timelice the VM itself from the host environemt, potentially by “simply” calling luaV_execute() for N instructions at a time by using something like the example I gave ( lua_pollvm(...) ).



Denne e-posten er fri for virus og uønsket programvare fordi avast! Antivirus beskyttelsen er aktiv.


Reply | Threaded
Open this post in threaded view
|

Re: Cooperative timeslicing?

Are Leistad
In reply to this post by aryajur
Milind Gupta wrote:
> I had written a small C module to allow running multiple Lua scripts by
> executing each one for a certain number of steps, also allowing data passing
> from the parent script to the child scripts. But it uses hooks in the C API.
>
> Milind
 
I'm doing something similar, only with a simpler implementation, using lua_newthread() and yielding from the debug hook. It works fine, executing any Lua function as a timesliced thread and allowing multiple such threads to run with the script. There are some snags; the global thread isn't timesliced, and I can't currently actual Lua coroutines don't cooperate fully. The reason I posted about this is that I think it would be nice if Lua supported cooperative timeslicing of the VM in a general and efficient manner.



Denne e-posten er fri for virus og uønsket programvare fordi avast! Antivirus beskyttelsen er aktiv.


Reply | Threaded
Open this post in threaded view
|

Re: Cooperative timeslicing?

aryajur

On Tue, Feb 3, 2015 at 11:30 AM, Are Leistad <[hidden email]> wrote:
Milind Gupta wrote:
> I had written a small C module to allow running multiple Lua scripts by
> executing each one for a certain number of steps, also allowing data passing
> from the parent script to the child scripts. But it uses hooks in the C API.
>
> Milind
 
I'm doing something similar, only with a simpler implementation, using lua_newthread() and yielding from the debug hook. It works fine, executing any Lua function as a timesliced thread and allowing multiple such threads to run with the script. There are some snags; the global thread isn't timesliced, and I can't currently actual Lua coroutines don't cooperate fully. The reason I posted about this is that I think it would be nice if Lua supported cooperative timeslicing of the VM in a general and efficient manner.

Yes coroutines was the tricky part but I managed to get them working by wrapping the coroutine functions inside each new thread. Every thread is a new lua state and is 'instruction' sliced controlled by the main lua script that started the thread.  



Denne e-posten er fri for virus og uønsket programvare fordi avast! Antivirus beskyttelsen er aktiv.