Chunk lifetime, coroutines and threads.

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Chunk lifetime, coroutines and threads.

Adam D. Moss
Hello!

I'm a new lua user and I've just joined the list.

I'm using lua 5.0alpha as the scripting core for a
game engine, after also evaluating Small and lua4.
Small is smaller than lua and also seems faster, but
lua won for:
* being an overall nicer language to write in
* userdata -- great
* a more game-oriented user community
* not requiring function declaration (though already I'm
  starting to suspect that this is not an advantage...)

I chose lua 5 over lua 4 mostly because of the lightweight
userdata, and secondarily because coroutines sound so seductive.

I've been through the official documentation, the Wiki and
a few months of mailing lists.  Nonetheless I have some
comments and questions I was wondering if someone could help
with.  :)

At the moment I register a subset of the game's API
with every context that I lua_open(), then run a script
with that context.

My understanding is that lua_close() will automatically happen
when the script finishes running.  In that case am I forbidden
from running it explicitly after script completion?  As I see it,
lua_open() creates a context using dynamically-allocated
memory and lua_close() frees that memory again, so any
pointer which the application keeps to its lua_State
after the script has run is now pointing at free memory.
The manual is misleading on that matter, implying (to me)
that you can lua_close() a state if you like but you
generally do not need to, when in truth you must NOT lua_close()
a lua_State after running a script to completion with that
context.

That brings me to my real questions.  I have various
precompiled chunks in memory.  I do my script-running with
lua_pcall(L, 0, LUA_MULTRET, 0).  So, every time I run
a script its context gets implicitly destroyed upon
completion; this means that for every script invokation I
have to re-allocate a context and re-register my game's C API
functions with it.

Is there any way to re-use a context?  I think (have not
tried, yet) that I could simply do the equivilent of a
while(1){body of script; coroutine.yield();} in my LUA
scripts and the script would thus never end (with the
implicit destruction) until I want it to, and I still
get control back to the calling C code after every
run without resorting to the debugging API.  But that
seems a bit hacky.

Can an object created by a script survive beyond that
script's own lifetime?  I am confused by the manual's
concept of a 'global environment'.  The manual says
that lua uses no global variables and the whole state of
the interpreter is in the lua_State.  Fine.  But in
this case how can it be useful that this state is
implicitly destroyed upon executing a chunk?  I must
simply be missing the obvious mechanism to run multiple
chunks on a single existing lua_State; as it stands, I
can't perform multiple identical lua_pcall()s in a row.
Please help!  I must be so stupid.  :)

My final questions are hopefully just asking for 
clarification to the lua5 manual, really.  The manual
says 'If you have a C library that offers multi-threading,
then Lua can cooperate with it to implement the equivalent
facility in Lua.  Also, Lua implements its own coroutine
system on top of threads'.

Well, this surprises me a bit because my understanding of
coroutines is something like 'threadless threads', just
a really convenient way of managing state-machines with
the dirtiness hidden away, but essentially still very much
single-threaded in reality.

This would be academic except that experience has made me
afraid of system-thread-using libraries.  I'm curious
as to whether lua's coroutines really require threads as
the manual implies; it looks like my lua500 build is not
linked to any thread libraries but the tests which use
coroutine.yield() still work fine.

Thanks a lot!

--Adam
-- 
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3
'The difference between BSD and GPL is similar to the difference
 between sex and rape.' -- anon

Reply | Threaded
Open this post in threaded view
|

Re: Chunk lifetime, coroutines and threads.

Thomas Lavergne-3
My understanding is that lua_close() will automatically happen
when the script finishes running.  In that case am I forbidden
from running it explicitly after script completion?  As I see it,
lua_open() creates a context using dynamically-allocated
memory and lua_close() frees that memory again, so any
pointer which the application keeps to its lua_State
after the script has run is now pointing at free memory.
The manual is misleading on that matter, implying (to me)
that you can lua_close() a state if you like but you
generally do not need to, when in truth you must NOT lua_close()
a lua_State after running a script to completion with that
context.

No the lua free the lua_state only when you call lua_close on it or when the app finish, so you can use it for all evaluation you want :

My final questions are hopefully just asking for clarification to the lua5 manual, really. The manual
says 'If you have a C library that offers multi-threading,
then Lua can cooperate with it to implement the equivalent
facility in Lua.  Also, Lua implements its own coroutine
system on top of threads'.

Well, this surprises me a bit because my understanding of
coroutines is something like 'threadless threads', just
a really convenient way of managing state-machines with
the dirtiness hidden away, but essentially still very much
single-threaded in reality.

This would be academic except that experience has made me
afraid of system-thread-using libraries.  I'm curious
as to whether lua's coroutines really require threads as
the manual implies; it looks like my lua500 build is not
linked to any thread libraries but the tests which use
coroutine.yield() still work fine.

The lua coroutine don't use any thread lib, the manual would say that you could have a real multi thread system in lua if you provide an external multi thread lib and you can use an independent implementation in lua of corountine, so if you don't have lib is like if you have only one thread and you run coroutine on it.

--
Thomas Lavergne                       "Le vrai rêveur est celui qui rêve
                                       de l'impossible."  (Elsa Triolet)
[hidden email]
[hidden email]    ICQ:#137121910     http://assoc.wanadoo.fr/thallium/


Reply | Threaded
Open this post in threaded view
|

Re: Chunk lifetime, coroutines and threads.

Adam D. Moss
Lavergne Thomas wrote:
> No the lua free the lua_state only when you call lua_close on it or when
> the app finish, so you can use it for all evaluation you want :

Thanks!  In that case I was confused by a mixture of mild
documentation ambiguity and the fact that I just can't seem to
lua_pcall() the same chunk twice, which should work if I understand
you correctly, but does not.

This is what I do (lua 5.00alpha):

L = lua_open();       // succeeds
register_apis(L);     // succeeds
lua_load (L, ...);    // succeeds
lua_pcall(L, 0, LUA_MULTRET, 0); // succeeds, works, returns 0
lua_pcall(L, 0, LUA_MULTRET, 0); // does nothing, returns LUA_ERRRUN
lua_pcall(L, 0, LUA_MULTRET, 0); // does nothing, returns LUA_ERRRUN
lua_pcall(L, 0, LUA_MULTRET, 0); // does nothing, returns LUA_ERRRUN

I believe that ALL of those lua_pcall()s should work -- right?

The script is simply fib.lua from the test directory of
the lua5 distribution.  If I cut the script down to just
a single print("HELLO!") then the problem still persists.

..
> The lua coroutine don't use any thread lib, the manual would say that
> you could have a real multi thread system in lua if you provide an
> external multi thread lib and you can use an independent implementation
> in lua of corountine, so if you don't have lib is like if you have only
> one thread and you run coroutine on it.

That's good news.  Thank you.

--Adam
-- 
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3

Reply | Threaded
Open this post in threaded view
|

Re: Chunk lifetime, coroutines and threads.

Peter Loveday-2
> This is what I do (lua 5.00alpha):
>
> L = lua_open();       // succeeds
> register_apis(L);     // succeeds
> lua_load (L, ...);    // succeeds
> lua_pcall(L, 0, LUA_MULTRET, 0); // succeeds, works, returns 0
> lua_pcall(L, 0, LUA_MULTRET, 0); // does nothing, returns LUA_ERRRUN
> lua_pcall(L, 0, LUA_MULTRET, 0); // does nothing, returns LUA_ERRRUN
> lua_pcall(L, 0, LUA_MULTRET, 0); // does nothing, returns LUA_ERRRUN
>
> I believe that ALL of those lua_pcall()s should work -- right?

lua_pcall(...) will 'pop' all the arguments and function (or chunk) from the
stack, so after the call you'll find your stack is now empty.  Try either:

lua_load (L, ...);
lua_pcall(L, 0, LUA_MULTRET, 0);
lua_load (L, ...);
lua_pcall(L, 0, LUA_MULTRET, 0);
...

or, perhaps more efficiently:

lua_load (L, ...);

lua_pushvalue(L, -1); // Duplicate top item
lua_pcall(L, 0, LUA_MULTRET, 0);
lua_pushvalue(L, -1); // Duplicate top item
lua_pcall(L, 0, LUA_MULTRET, 0);
...
...
lua_pop(L,1);


Love, Light and Peace,
- Peter Loveday
Director of Development, eyeon Software



Reply | Threaded
Open this post in threaded view
|

Re: Chunk lifetime, coroutines and threads.

Adam D. Moss
Peter Loveday wrote:
> lua_pcall(...) will 'pop' all the arguments and function (or chunk) from the
> stack, so after the call you'll find your stack is now empty.
...
> lua_load (L, ...);
> 
> lua_pushvalue(L, -1); // Duplicate top item
> lua_pcall(L, 0, LUA_MULTRET, 0);
> lua_pushvalue(L, -1); // Duplicate top item
> lua_pcall(L, 0, LUA_MULTRET, 0);

Thanks!  That does work!

What exactly does lua_load() push onto the stack for popping
at execution?  ie. Since I'm passing 0 'nargs' to lua_pcall()
I presume that I don't have to push any arguments onto the
stack, so what is lua_pcall() requiring and how can I manually
derive this magic value myself so that I may push it onto the
stack every time I wish to invoke a particular chunk?

(I've looked at the lua_load() source but I can't find the
point at which it pushes this vital value; I wouldn't know
it if I saw it.)

Thanks,
--Adam
-- 
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3

Reply | Threaded
Open this post in threaded view
|

Re: Chunk lifetime, coroutines and threads.

Peter Loveday-2
> What exactly does lua_load() push onto the stack for popping
> at execution?  ie. Since I'm passing 0 'nargs' to lua_pcall()
> I presume that I don't have to push any arguments onto the
> stack, so what is lua_pcall() requiring and how can I manually
> derive this magic value myself so that I may push it onto the
> stack every time I wish to invoke a particular chunk?


Basically lua_load() pushes a function object representing the chunk of code
that is loaded from disk.

Probably the best thing is to store this in either a global variable, or
possibly better still in the Lua Registry (that way it is a lot less likely
any code can kill your chunk by setting a global variable).  You can then
retrieve it and call it whenever you like.

Something like:

    lua_pushstring(L, "MyChunkFunc");
    lua_load (L, ...);
    lua_settable(L, LUA_REGISTRYINDEX);

Then when you wish to call it:

    lua_pushstring(L, "MyChunkFunc");
    lua_gettable(L, LUA_REGISTRYINDEX);
    lua_pcall(L, 0, LUA_MULTRET, 0);


To use a global, simply store and retrieve it using LUA_GLOBALSINDEX
instead.


Note that specifying LUA_MULTRET means the stack will contain any number of
values returned from the chunk; if you do not need this, and don't want to
deal with cleaning up the stack afterwards, you can pass 0 for this arg.


Love, Light and Peace,
- Peter Loveday
Director of Development, eyeon Software



Reply | Threaded
Open this post in threaded view
|

Re: Chunk lifetime, coroutines and threads.

Adam D. Moss
Peter Loveday wrote:
> Something like:
>     lua_pushstring(L, "MyChunkFunc");
>     lua_load (L, ...);
>     lua_settable(L, LUA_REGISTRYINDEX);

Woo!  You people are the best.  :)

Well, I have things chugging along quite nicely now (I ended
up putting the function objects into fields of a single global
table like the standard lua packages, since I wanted scripts
to be able to call each other; I even tried letting scripts
auto-load each other by using the __index metafunction
of this table, which is fun but has some issues I'll revisit
when I think that this is a good idea rather than just being
cute).

I'm enjoying lua so far.  Thanks for the help, everyone.

Regards,
--Adam
-- 
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3
"Beware the leader who bangs the drums of war in order to whip the
citizenry into a patriotic fervor, for patriotism ... both emboldens
the blood, just as it narrows the mind." -- Julius Caesar