running LUA code step by step

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

running LUA code step by step

Gabriel Peyré
sorry if this question has already been asked ...

I am trying to run multiple LUA chunks of code at the same time 
(to make a kind of codeware with robots scripted in LUA).
To do that I need to process each LUA code one intruction by one
instruction (or at least line by line), but I didn't manage to do it.
I've try using the line hook, but without great success ...

thanks a lot for your attention.

- gabriel -


Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Russ Webb
I've thought about writing something similar.  The way I was going to do it
was to wrap all the robot operations in calls that decremented a count
yielded to the other robots when the count was negative.  That way you can
set the counter to say 1000000 and have different operations have different
costs, maybe sin(x) cost 1 point of time but shoot_laser() cost 1000 points.

Basically move the time counting up into the routines that do work and not
in the byte code.  You loose granularity but you gain weighting control of
each operation.  I guess you could do both if you had a the VM decrement the
count for each bytecode, then the next call to a wrapped function would
include to overhead of the bytecodes before it --- if there's no simulation
'advantage' to completing t.x = 4 then that will work.

Of course, your needs might be vastly different.

Russ

> From: Gabriel Peyré <[hidden email]>
> Reply-To: [hidden email]
> Date: Fri, 15 Mar 2002 20:26:41 +0100
> To: Multiple recipients of list <[hidden email]>
> Subject: running LUA code step by step
> 
> sorry if this question has already been asked ...
> 
> I am trying to run multiple LUA chunks of code at the same time
> (to make a kind of codeware with robots scripted in LUA).
> To do that I need to process each LUA code one intruction by one
> instruction (or at least line by line), but I didn't manage to do it.
> I've try using the line hook, but without great success ...
> 
> thanks a lot for your attention.
> 
> - gabriel -
> 


Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Sean Middleditch
In reply to this post by Gabriel Peyré
On Fri, 2002-03-15 at 14:26, Gabriel Peyré wrote:
> sorry if this question has already been asked ...
> 
> I am trying to run multiple LUA chunks of code at the same time 
> (to make a kind of codeware with robots scripted in LUA).
> To do that I need to process each LUA code one intruction by one
> instruction (or at least line by line), but I didn't manage to do it.
> I've try using the line hook, but without great success ...
> 
> thanks a lot for your attention.

Yes, I'm also looking for a similar solution.  I want Lua to be able to
rune multiple chunks/states/threads/whatever, but be interuptable (to
give control back to the controlling thread).

For example, one project I'm using that has Lua, there are a constant
stream of network events to respond to (which might include triggering a
Lua script).  However, any of these scripts might perform certain user
tasks (like ask the user for some input) - those scripts need to "pause"
and let the other scripts runs on the incoming data.  Also, there may be
several long-running scripts.  These need to pause after, say, 1000
bytecode executions (or something similar) and return control the
hosting C program, so it can handle the network events/ user input/
etc.  Then, of course, when the main loop is all set, it can inform the
Lua interpreter to "wake-up" all the threads and run them again, until
they use up their time slot/byte-code count, and are paused again.

Basically, I need pre-emptive multitasking in a language, where the
interpeter(s) are running in a single OS thread.

I believe on the Lua site there is a patch to do this - it is rather
buggy, in places, however.  Perhaps someone more experienced with the
Lua internals could fix it up?

> 
> - gabriel -
> 



Reply | Threaded
Open this post in threaded view
|

RE: running LUA code step by step

Curt Carpenter
In reply to this post by Gabriel Peyré
> I want Lua to be able to rune multiple chunks/states/threads/whatever,
but be interruptible (to give control back to the controlling thread).

My solution, which requires 4.1:

In Windows, we have the concept of fibers--user scheduled execution
contexts within a thread, each with their own stack. In my thread, I
have a primary fiber, and a bunch of worker fibers. Fibers are extremely
lightweight to switch, compared to threads, and you don't have to worry
about making stuff thread safe. The main loop on the primary fiber looks
like this:

1) Fire up new fibers (and call lua_newthread) for new objects which
need their own script execution/stack/lua_state
2) Keep a list of objects that are ready to execute. Loop over all
objects and switch to their fiber.
3) In each object, figure out when they want to yield, either by a
line/call hook, or by a call into a registered C function that indicates
they need to wait (like a call that blocks on something out of its
control).
4) When an object is ready to yield, switch back to the primary fiber.
5) Depending on your needs, have some plan to add objects back to the
ready list. If you want all objects to execute code on every cycle, then
maybe you don't need a ready list, but I use it to mark scripts that are
ready to continue after having some condition met that they are blocked
on.

Curt

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Luiz Henrique de Figueiredo
Lua 4.1w4 already contains native coroutines, which will help to run
things "in parallel". Check it out. --lhf

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Steve Dekorte-4
In reply to this post by Sean Middleditch

On Friday, March 15, 2002, at 12:30  PM, Sean Middleditch wrote:
Yes, I'm also looking for a similar solution.  I want Lua to be able to
rune multiple chunks/states/threads/whatever, but be interuptable (to
give control back to the controlling thread).

I think the yield patch can accomplish this in a cooperative way.

Steve



Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Luiz Henrique de Figueiredo
In reply to this post by Gabriel Peyré
>I think the yield patch can accomplish this in a cooperative way.

This has been added officially to 4.1w4. --lhf

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Sean Middleditch
In reply to this post by Steve Dekorte-4
On Fri, 2002-03-15 at 16:33, Steve Dekorte wrote:
> 
> On Friday, March 15, 2002, at 12:30  PM, Sean Middleditch wrote:
> > Yes, I'm also looking for a similar solution.  I want Lua to be able to
> > rune multiple chunks/states/threads/whatever, but be interuptable (to
> > give control back to the controlling thread).
> 
> I think the yield patch can accomplish this in a cooperative way.

yield... doesn't ring a bell in Lua terms (and am about to leave).  You
don't mean yield as in how Ruby handles it, do you?  I don't want to
have the user pass around callbacks.  that's not fun.  something more
like

myvar = ask_user ( "Are you sure?" )

I'm not putting this stuff in front of programmers, I'm putting it in
front of users to do their macro scripting.  I want to avoid callbacks
(on their end) as much as possible.

> 
> Steve
> 
> 



Reply | Threaded
Open this post in threaded view
|

RE: running LUA code step by step

Curt Carpenter
In reply to this post by Gabriel Peyré
> Lua 4.1w4 already contains native coroutines, which will help to run
things "in parallel". Check it out. --lhf

I think what Sean is talking about is what I was talking about, namely
scripts that have blocking calls (into C). I don't understand how this
would be accomplished with Lua's coroutines, but my previous email
explains how I did it.

Curt

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Roberto Ierusalimschy
In reply to this post by Curt Carpenter
> 1) Fire up new fibers (and call lua_newthread) for new objects which
> need their own script execution/stack/lua_state
> 2) Keep a list of objects that are ready to execute. Loop over all
> objects and switch to their fiber.
> 3) In each object, figure out when they want to yield, either by a
> line/call hook, or by a call into a registered C function that indicates
> they need to wait (like a call that blocks on something out of its
> control).
> 4) When an object is ready to yield, switch back to the primary fiber.
> 5) Depending on your needs, have some plan to add objects back to the
> ready list. If you want all objects to execute code on every cycle, then
> maybe you don't need a ready list, but I use it to mark scripts that are
> ready to continue after having some condition met that they are blocked
> on.

You should be able to do exactly the same thing with coroutines in
Luaw4. (The only drawback is that currently you cannot yield inside a
linehook; we will correct that. But you can yield inside a registered C
function that indicates they need to wait.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Sean Middleditch
On Sat, 2002-03-16 at 07:14, Roberto Ierusalimschy wrote:
> > 1) Fire up new fibers (and call lua_newthread) for new objects which
> > need their own script execution/stack/lua_state
> > 2) Keep a list of objects that are ready to execute. Loop over all
> > objects and switch to their fiber.
> > 3) In each object, figure out when they want to yield, either by a
> > line/call hook, or by a call into a registered C function that indicates
> > they need to wait (like a call that blocks on something out of its
> > control).
> > 4) When an object is ready to yield, switch back to the primary fiber.
> > 5) Depending on your needs, have some plan to add objects back to the
> > ready list. If you want all objects to execute code on every cycle, then
> > maybe you don't need a ready list, but I use it to mark scripts that are
> > ready to continue after having some condition met that they are blocked
> > on.
> 
> You should be able to do exactly the same thing with coroutines in
> Luaw4. (The only drawback is that currently you cannot yield inside a
> linehook; we will correct that. But you can yield inside a registered C
> function that indicates they need to wait.)

OK, where can I find Lua Work 4?  All I can find on the website is 4.0,
and my Debian packages are Alpha 4 (I can't find anything regarding
yield in their headers).

What am I missing?

> 
> -- Roberto



Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Gabriel Peyré
In reply to this post by Roberto Ierusalimschy
Thanks a lot for all these answers !
I've just read the documentation of lua-w4, and I found no explaination
about the couroutines functions, wich seems to be:
--
LUA_API void lua_cobegin (lua_State *L, int nargs);
LUA_API int lua_yield (lua_State *L, int nresults);
LUA_API int lua_resume (lua_State *L, lua_State *co);
--
do I just need to call 'lua_yield' during the execution of the script of one
of my robot [I take the example
of a codeware] (for instance during a line hook), then execute the of the
other codes, and when the last code
has been executed, call lua_resume ?
I thought the limitations of the line hook will not permit that kind of
manipulations ...


- gabriel -


----- Original Message -----
From: "Roberto Ierusalimschy" <[hidden email]>
To: "Multiple recipients of list" <[hidden email]>
Sent: Saturday, March 16, 2002 1:14 PM
Subject: Re: running LUA code step by step


> > 1) Fire up new fibers (and call lua_newthread) for new objects which
> > need their own script execution/stack/lua_state
> > 2) Keep a list of objects that are ready to execute. Loop over all
> > objects and switch to their fiber.
> > 3) In each object, figure out when they want to yield, either by a
> > line/call hook, or by a call into a registered C function that indicates
> > they need to wait (like a call that blocks on something out of its
> > control).
> > 4) When an object is ready to yield, switch back to the primary fiber.
> > 5) Depending on your needs, have some plan to add objects back to the
> > ready list. If you want all objects to execute code on every cycle, then
> > maybe you don't need a ready list, but I use it to mark scripts that are
> > ready to continue after having some condition met that they are blocked
> > on.
>
> You should be able to do exactly the same thing with coroutines in
> Luaw4. (The only drawback is that currently you cannot yield inside a
> linehook; we will correct that. But you can yield inside a registered C
> function that indicates they need to wait.)
>
> -- Roberto


Reply | Threaded
Open this post in threaded view
|

RE: running LUA code step by step

Curt Carpenter
In reply to this post by Gabriel Peyré
Can you explain how that works? Does the yield call inside a registered
C function not return until some other state yields back to the first
state? 
If so, I don't understand how that's possible (without blowing the stack
out, for example). 
If not, it's not exactly the same thing.

Thanks,

Curt

-----Original Message-----
From: Roberto Ierusalimschy [[hidden email]] 
Sent: Saturday, March 16, 2002 4:14 AM
To: Multiple recipients of list
Subject: Re: running LUA code step by step 


> 1) Fire up new fibers (and call lua_newthread) for new objects which 
> need their own script execution/stack/lua_state
> 2) Keep a list of objects that are ready to execute. Loop over all 
> objects and switch to their fiber.
> 3) In each object, figure out when they want to yield, either by a 
> line/call hook, or by a call into a registered C function that 
> indicates they need to wait (like a call that blocks on something out 
> of its control).
> 4) When an object is ready to yield, switch back to the primary fiber.
> 5) Depending on your needs, have some plan to add objects back to the 
> ready list. If you want all objects to execute code on every cycle, 
> then maybe you don't need a ready list, but I use it to mark scripts 
> that are ready to continue after having some condition met that they 
> are blocked on.

You should be able to do exactly the same thing with coroutines in
Luaw4. (The only drawback is that currently you cannot yield inside a
linehook; we will correct that. But you can yield inside a registered C
function that indicates they need to wait.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Guy English
In reply to this post by Gabriel Peyré
On Saturday, March 16, 2002, at 03:53  PM, Gabriel Peyré wrote:
Thanks a lot for all these answers !
I've just read the documentation of lua-w4, and I found no explaination
about the couroutines functions, wich seems to be:
--
LUA_API void lua_cobegin (lua_State *L, int nargs);
LUA_API int lua_yield (lua_State *L, int nresults);
LUA_API int lua_resume (lua_State *L, lua_State *co);
--

I've been playing around with this and the following approach seems to work:

lua_getglobal( L, "main" );
lua_cobegin( L, 0 );
lua_resume( L, 0, 0 );
... lua code runs until a "yield" is issued.

at this point you continue with the rest of your program. The next time you'd like the Lua code to execute just call lua_resume again. I'm not clear on the argument handling stuff that's why it's 0 for now. I also don't need it for my purposes. Another problem I face is that lua_resume will throw a fit is the Lua script has finished ( or wasn't actually yielded ). There seems to be no public function in the API to determine if a script has yielded or stopped naturally. I ended up putting detection code in lua_resume and returning a status code of my own to know when my scripts had terminated. I'm pretty sure there's a much better way.

Hope that helps a little,
Guy


Reply | Threaded
Open this post in threaded view
|

RE: running LUA code step by step

Sean Middleditch
In reply to this post by Gabriel Peyré
>From reading the C source, it looks something like this (not sure on it,
only looked at it very briefly):

A function can call lua_yield() to setup the lua state/stack, then the C
function would return.  When the Lua coroutine ran again, it would
re-run the C function, but the state (that it previously ran, plus any
arguments) would be based into it again.

The functionality looks rather convoluted, and not even what I was
looking for - it is not Lua threading, which is all that I want or
need.  I can't even see the point to this coroutine implementation.

On Sat, 2002-03-16 at 17:39, Curt Carpenter wrote:
> Can you explain how that works? Does the yield call inside a registered
> C function not return until some other state yields back to the first
> state? 
> If so, I don't understand how that's possible (without blowing the stack
> out, for example). 
> If not, it's not exactly the same thing.
> 
> Thanks,
> 
> Curt
> 
> -----Original Message-----
> From: Roberto Ierusalimschy [[hidden email]] 
> Sent: Saturday, March 16, 2002 4:14 AM
> To: Multiple recipients of list
> Subject: Re: running LUA code step by step 
> 
> 
> > 1) Fire up new fibers (and call lua_newthread) for new objects which 
> > need their own script execution/stack/lua_state
> > 2) Keep a list of objects that are ready to execute. Loop over all 
> > objects and switch to their fiber.
> > 3) In each object, figure out when they want to yield, either by a 
> > line/call hook, or by a call into a registered C function that 
> > indicates they need to wait (like a call that blocks on something out 
> > of its control).
> > 4) When an object is ready to yield, switch back to the primary fiber.
> > 5) Depending on your needs, have some plan to add objects back to the 
> > ready list. If you want all objects to execute code on every cycle, 
> > then maybe you don't need a ready list, but I use it to mark scripts 
> > that are ready to continue after having some condition met that they 
> > are blocked on.
> 
> You should be able to do exactly the same thing with coroutines in
> Luaw4. (The only drawback is that currently you cannot yield inside a
> linehook; we will correct that. But you can yield inside a registered C
> function that indicates they need to wait.)
> 
> -- Roberto



Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Roberto Ierusalimschy
In reply to this post by Gabriel Peyré
> do I just need to call 'lua_yield' during the execution of the script of one
> of my robot [I take the example
> of a codeware] (for instance during a line hook), then execute the of the
> other codes, and when the last code
> has been executed, call lua_resume ?

Yes.


> I thought the limitations of the line hook will not permit that kind of
> manipulations ...

Yes... Currently (w4) you cannot call lua_yield inside a hook. But I
guess we will fix that.


-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Roberto Ierusalimschy
In reply to this post by Curt Carpenter
> Can you explain how that works? Does the yield call inside a registered
> C function not return until some other state yields back to the first
> state? 

No. You must call lua_yield as "return lua_yield(...)". So your
registered C function ends when it calls yield. We "yield" Lua code, not
C code.

> If so, I don't understand how that's possible (without blowing the stack
> out, for example). 
> If not, it's not exactly the same thing.

It is now exactly the same thing from C, but it is from Lua. So, you
must write your "blocking C functions" with care. For instance, one
solution for a blocking read could be something like this:

  int aux_read (lua_State *L) {
    if (cannot_read()) return lua_yield();
    else {
      ...
    }
  }

and wrap that in Lua:

  function read ()
    while 1 do
      local res = aux_read()
      if res then return res end
    end
  end

A more sofisticated solution could avoid the loop, by making sure that the
scheduler does not return to that thread until its stream is ready...

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: running LUA code step by step

Roberto Ierusalimschy
In reply to this post by Guy English
> There seems to be no public function in the API to determine 
> if a script has yielded or stopped naturally.

No, but lua_resume "returns" the arguments you passed to "yield", or the
return value of the thread (when it stops naturally). So, as long as you
follow some convention, it is easy to set them apart.


> I'm not clear on the argument handling stuff that's why it's 0 for now.

  LUA_API void lua_cobegin (lua_State *L, int nargs);

  nargs is the number of arguments you are passing to the start function
  (like "nargs" in lua_call).

  LUA_API int  lua_resume (lua_State *L, lua_State *co);
  'L' is the calling thread, 'co' is the co-routine you are activating.

  LUA_API int  lua_yield (lua_State *L, int nresults);
  'nresults' is the number of results (in the stack) that will be passed
  back to lua_resume.

-- Roberto