RE: About lua_newthread

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Tom Miles-2
Hi,
 
Sorry about the the thread necromancy, but I have just encountered this situation in the project I am currently working on.  I am running several threads from my main thread, all of which are just loaded with functions to call in an event based system.  At various intervals I want to "unload" these functions and load a new set, so I want to shut down the thread.  I therefore need to remove references to my thread from the stack as per the suggestion, however, when I iterate through the stack all that is on there is a userdata of indeterminate origin, not a thread.  (Having said that, I've just discovered that if I do lua_touserdata() and compare it to the address of the state returned with lua_newthread(), then they are the same, so I have found my thread - but there seems to be a little bug there as it isn't of type thread?).
 
On a similar note, is there a way of removing a non integer index from a table?  I use the thread as a key in a table to match it up to a game object, and when that object is destroyed I want to remove the entry in the table as well.
 
Thanks for any help.
 
Tom
 
 -----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Raymond Jacobs
Sent: 18 May 2006 16:20
To: Lua list
Subject: Re: About lua_newthread

When you are done with a thread (when lua_resume returns and it's not a yield), you should remove the thread directly from the stack, this will remove the reference to it and it can then be garbage collected (its a pain I know =/).
 
another method which I've not yet done but should work, is to create a new thread, create a lua reference to it (using lua_ref) then pop it off the stack immediately (so it doesn't get buried), and later when you are done with it, destroy the reference (lua_unref) and that should allow it to be GC'd
 
hope that helps,
 
Raymond Jacobs
Owner,
Ethereal Darkness Interactive

 
On 5/18/06, mos <[hidden email]> wrote:

Hi

in lua5.1 manual:

lua_newthread
         lua_State *lua_newthread (lua_State *L);
There is no explicit function to close or to destroy a thread. Threads are
subject to garbage collection, like any Lua object

       I don't understand this , for example like Npc in a game

       function AI()
               while(true) do
                       walkto(A)
                       walkto(B)
                       ...
                       walkto(Z)
               end
       end

       I bind each npc with a lua_State by lua_newthread

       but the npc will die and i should close the lua_State , but I can
not.
       what can I do ? or some better suggestion ?

Best Regards
mos



Reply | Threaded
Open this post in threaded view
|

Re: About lua_newthread

Julien Hamaide-2
Tom Miles a écrit :

Hi,
Sorry about the the thread necromancy, but I have just encountered this situation in the project I am currently working on. I am running several threads from my main thread, all of which are just loaded with functions to call in an event based system. At various intervals I want to "unload" these functions and load a new set, so I want to shut down the thread. I therefore need to remove references to my thread from the stack as per the suggestion, however, when I iterate through the stack all that is on there is a userdata of indeterminate origin, not a thread. (Having said that, I've just discovered that if I do lua_touserdata() and compare it to the address of the state returned with lua_newthread(), then they are the same, so I have found my thread - but there seems to be a little bug there as it isn't of type thread?).


Can you show us how you initialize your thread??

On a similar note, is there a way of removing a non integer index from a table? I use the thread as a key in a table to match it up to a game object, and when that object is destroyed I want to remove the entry in the table as well.


If you use a weak value table, the entry will be deleted when the object
is collected.

Just put "v" in field __mode of the metatable of your table. If you want
weak key, just put "k".

Julien Hamaide

Lead Programmer

10Tacle Studios Belgium

Thanks for any help. Tom -----Original Message----- *From:* [hidden email] [[hidden email] Behalf Of* Raymond Jacobs
*Sent:* 18 May 2006 16:20
*To:* Lua list
*Subject:* Re: About lua_newthread

    When you are done with a thread (when lua_resume returns and it's
    not a yield), you should remove the thread directly from the
    stack, this will remove the reference to it and it can then be
    garbage collected (its a pain I know =/).
another method which I've not yet done but should work, is to
    create a new thread, create a lua reference to it (using lua_ref)
    then pop it off the stack immediately (so it doesn't get buried),
    and later when you are done with it, destroy the reference
    (lua_unref) and that should allow it to be GC'd
hope that helps, Raymond Jacobs
    Owner,
    Ethereal Darkness Interactive

On 5/18/06, *mos* <[hidden email] <[hidden email]>>
    wrote:


        Hi

        in lua5.1 manual:

        lua_newthread
                 lua_State *lua_newthread (lua_State *L);
        There is no explicit function to close or to destroy a thread.
        Threads are
        subject to garbage collection, like any Lua object

               I don't understand this , for example like Npc in a game

               function AI()
                       while(true) do
                               walkto(A)
                               walkto(B)
                               ...
                               walkto(Z)
                       end
               end

               I bind each npc with a lua_State by lua_newthread

               but the npc will die and i should close the lua_State ,
        but I can
        not.
               what can I do ? or some better suggestion ?

        Best Regards
        mos






Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Tom Miles-2
In reply to this post by Tom Miles-2
Title: RE: About lua_newthread

Thanks for the speedy response. 



>> Can you show us how you initialize your thread??

I am using an approach similar to the one in Game Programming Gems 5, where I have a manager object that controls the main state and dishes out threads when requested.  I.e.

// 

static void CreateThread(State & state)
{
        if (m_MainState == NULL)
        {
                m_MainState = luaL_newstate();
                luaL_openlibs(m_MainState);

                luaL_openlib(m_MainState, "Script", scriptLib, 0);
        }

        state.m_state = lua_newthread(m_MainState);

        // save a pointer to the thread manager object in the global table
        // using the new thread's vm pointer as a key
        lua_pushlightuserdata(m_MainState, state.m_state);              <--- Is this causing a problem?
        lua_pushlightuserdata(m_MainState, &state);
        lua_settable(m_MainState, LUA_GLOBALSINDEX );
}

static void ReleaseThread(State & state)
{
        int stackSize = lua_gettop(m_MainState);
        for (int n=0; n<stackSize; ++n)
        {
                // This check should be: lua_isthread() && lua_tothread() == state.m_state
                if (lua_touserdata(m_MainState, n) == state.m_state)
                {
                        lua_remove(m_MainState, n);
                        break;
                }
        }
        // For the time being, set the value of the key (the state we are removing) in our map to be nil
        lua_pushlightuserdata(m_MainState, state.m_state);
        lua_pushnil(m_MainState);
        lua_settable(m_MainState, LUA_GLOBALSINDEX);

        // There is also code to close down lua if we have released the last thread here
}

>> If you use a weak value table, the entry will be deleted when the object
>> is collected.

>> Just put "v" in field __mode of the metatable of your table. If you want
>> weak key, just put "k".

As you can see, I am using the globals table, but I suppose I could create a weak table especially for the job instead.

> Tom

>  -----Original Message-----
> *From:* [hidden email]
> [[hidden email]]*On Behalf Of* Raymond Jacobs
> *Sent:* 18 May 2006 16:20
> *To:* Lua list
> *Subject:* Re: About lua_newthread
>
>     When you are done with a thread (when lua_resume returns and it's
>     not a yield), you should remove the thread directly from the
>     stack, this will remove the reference to it and it can then be
>     garbage collected (its a pain I know =/).
>     
>     another method which I've not yet done but should work, is to
>     create a new thread, create a lua reference to it (using lua_ref)
>     then pop it off the stack immediately (so it doesn't get buried),
>     and later when you are done with it, destroy the reference
>     (lua_unref) and that should allow it to be GC'd
>     
>     hope that helps,
>     
>     Raymond Jacobs
>     Owner,
>     Ethereal Darkness Interactive
>
>     
>     On 5/18/06, *mos* <[hidden email] <[hidden email]>>
>     wrote:
>
>
>         Hi
>
>         in lua5.1 manual:
>
>         lua_newthread
>                  lua_State *lua_newthread (lua_State *L);
>         There is no explicit function to close or to destroy a thread.
>         Threads are
>         subject to garbage collection, like any Lua object
>
>                I don't understand this , for example like Npc in a game
>
>                function AI()
>                        while(true) do
>                                walkto(A)
>                                walkto(B)
>                                ...
>                                walkto(Z)
>                        end
>                end
>
>                I bind each npc with a lua_State by lua_newthread
>
>                but the npc will die and i should close the lua_State ,
>         but I can
>         not.
>                what can I do ? or some better suggestion ?
>
>         Best Regards
>         mos
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: About lua_newthread

Julien Hamaide-2
The problem with this approach is that the thread will never be garbage collected as lua_newthread push a thread handle on the main thread stack.

So at the end of this call, you still have the thread handle on the stack.

I also see that you use the global table to store thread function. Pay attention that if 2 thread have functions with the same name, they will collide.

I use same kind of thing for a game, this is my approach for both liveness of thread and local space :

In my Thread constructor :
// master state is my main thread
   ThreadState = lua_newthread( master_state );

   // prevent thread from being garbage collected
   lua_pushvalue( master_state, -1 );
   ThreadReferenceIdentifier = luaL_ref( master_state, LUA_REGISTRYINDEX );

//Create a local environment with a link to global environment via __index metamethod
   lua_newtable( master_state );
   lua_pushvalue( master_state, -1 );
   lua_setmetatable( master_state, -2 ); //Set itself as metatable
   lua_pushvalue( master_state, LUA_GLOBALSINDEX );
   lua_setfield( master_state, -2, "__index" );
   lua_setfenv( master_state, -2 );
   lua_pop( master_state, 1 );

and in the destructor :

luaL_unref( master_state, LUA_REGISTRYINDEX, ThreadReferenceIdentifier );

If you have any questions about it, don't hesitate

Julien Hamaide
Lead Programmer
10Tacle Studios Belgium


Tom Miles a écrit :

Thanks for the speedy response.


>> Can you show us how you initialize your thread??

I am using an approach similar to the one in Game Programming Gems 5, where I have a manager object that controls the main state and dishes out threads when requested. I.e.

//
static void CreateThread(State & state)
{
        if (m_MainState == NULL)
        {
                m_MainState = luaL_newstate();
                luaL_openlibs(m_MainState);

                luaL_openlib(m_MainState, "Script", scriptLib, 0);
        }

        state.m_state = lua_newthread(m_MainState);

// save a pointer to the thread manager object in the global table
        // using the new thread's vm pointer as a key
lua_pushlightuserdata(m_MainState, state.m_state); <--- Is this causing a problem?
        lua_pushlightuserdata(m_MainState, &state);
        lua_settable(m_MainState, LUA_GLOBALSINDEX );
}

static void ReleaseThread(State & state)
{
        int stackSize = lua_gettop(m_MainState);
        for (int n=0; n<stackSize; ++n)
        {
// This check should be: lua_isthread() && lua_tothread() == state.m_state
                if (lua_touserdata(m_MainState, n) == state.m_state)
                {
                        lua_remove(m_MainState, n);
                        break;
                }
        }
// For the time being, set the value of the key (the state we are removing) in our map to be nil
        lua_pushlightuserdata(m_MainState, state.m_state);
        lua_pushnil(m_MainState);
        lua_settable(m_MainState, LUA_GLOBALSINDEX);

// There is also code to close down lua if we have released the last thread here
}

>> If you use a weak value table, the entry will be deleted when the object
>> is collected.

>> Just put "v" in field __mode of the metatable of your table. If you want
>> weak key, just put "k".

As you can see, I am using the globals table, but I suppose I could create a weak table especially for the job instead.

> Tom
> > -----Original Message-----
> *From:* [hidden email]
> [[hidden email] Behalf Of* Raymond Jacobs
> *Sent:* 18 May 2006 16:20
> *To:* Lua list
> *Subject:* Re: About lua_newthread
>
>     When you are done with a thread (when lua_resume returns and it's
>     not a yield), you should remove the thread directly from the
>     stack, this will remove the reference to it and it can then be
>     garbage collected (its a pain I know =/).
> > another method which I've not yet done but should work, is to
>     create a new thread, create a lua reference to it (using lua_ref)
>     then pop it off the stack immediately (so it doesn't get buried),
>     and later when you are done with it, destroy the reference
>     (lua_unref) and that should allow it to be GC'd
> > hope that helps, > > Raymond Jacobs
>     Owner,
>     Ethereal Darkness Interactive
>
> > On 5/18/06, *mos* <[hidden email] <[hidden email]>>
>     wrote:
>
>
>         Hi
>
>         in lua5.1 manual:
>
>         lua_newthread
>                  lua_State *lua_newthread (lua_State *L);
>         There is no explicit function to close or to destroy a thread.
>         Threads are
>         subject to garbage collection, like any Lua object
>
>                I don't understand this , for example like Npc in a game
>
>                function AI()
>                        while(true) do
>                                walkto(A)
>                                walkto(B)
>                                ...
>                                walkto(Z)
>                        end
>                end
>
>                I bind each npc with a lua_State by lua_newthread
>
>                but the npc will die and i should close the lua_State ,
>         but I can
>         not.
>                what can I do ? or some better suggestion ?
>
>         Best Regards
>         mos
>
>
>




Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Jerome Vuarand-2
In reply to this post by Tom Miles-2
Title: RE: About lua_newthread
I think the line you point is the right one. Values are not typed in C API, stack entries are. So if you push a void* as a light user data, it is a light user data for Lua. lua_newthread already pushes the lua_State on the stack as a thread, so you don't need to explicitly do it.


De : [hidden email] [mailto:[hidden email]] De la part de Tom Miles
Envoyé : 7 juin 2006 12:18
À : Lua list
Objet : RE: About lua_newthread

Thanks for the speedy response. 



>> Can you show us how you initialize your thread??

I am using an approach similar to the one in Game Programming Gems 5, where I have a manager object that controls the main state and dishes out threads when requested.  I.e.

// 

static void CreateThread(State & state)
{
        if (m_MainState == NULL)
        {
                m_MainState = luaL_newstate();
                luaL_openlibs(m_MainState);

                luaL_openlib(m_MainState, "Script", scriptLib, 0);
        }

        state.m_state = lua_newthread(m_MainState);

        // save a pointer to the thread manager object in the global table
        // using the new thread's vm pointer as a key
        lua_pushlightuserdata(m_MainState, state.m_state);              <--- Is this causing a problem?
        lua_pushlightuserdata(m_MainState, &state);
        lua_settable(m_MainState, LUA_GLOBALSINDEX );
}

static void ReleaseThread(State & state)
{
        int stackSize = lua_gettop(m_MainState);
        for (int n=0; n<stackSize; ++n)
        {
                // This check should be: lua_isthread() && lua_tothread() == state.m_state
                if (lua_touserdata(m_MainState, n) == state.m_state)
                {
                        lua_remove(m_MainState, n);
                        break;
                }
        }
        // For the time being, set the value of the key (the state we are removing) in our map to be nil
        lua_pushlightuserdata(m_MainState, state.m_state);
        lua_pushnil(m_MainState);
        lua_settable(m_MainState, LUA_GLOBALSINDEX);

        // There is also code to close down lua if we have released the last thread here
}

>> If you use a weak value table, the entry will be deleted when the object
>> is collected.

>> Just put "v" in field __mode of the metatable of your table. If you want
>> weak key, just put "k".

As you can see, I am using the globals table, but I suppose I could create a weak table especially for the job instead.

> Tom

>  -----Original Message-----
> *From:* [hidden email]
> [[hidden email]]*On Behalf Of* Raymond Jacobs
> *Sent:* 18 May 2006 16:20
> *To:* Lua list
> *Subject:* Re: About lua_newthread
>
>     When you are done with a thread (when lua_resume returns and it's
>     not a yield), you should remove the thread directly from the
>     stack, this will remove the reference to it and it can then be
>     garbage collected (its a pain I know =/).
>     
>     another method which I've not yet done but should work, is to
>     create a new thread, create a lua reference to it (using lua_ref)
>     then pop it off the stack immediately (so it doesn't get buried),
>     and later when you are done with it, destroy the reference
>     (lua_unref) and that should allow it to be GC'd
>     
>     hope that helps,
>     
>     Raymond Jacobs
>     Owner,
>     Ethereal Darkness Interactive
>
>     
>     On 5/18/06, *mos* <[hidden email] <[hidden email]>>
>     wrote:
>
>
>         Hi
>
>         in lua5.1 manual:
>
>         lua_newthread
>                  lua_State *lua_newthread (lua_State *L);
>         There is no explicit function to close or to destroy a thread.
>         Threads are
>         subject to garbage collection, like any Lua object
>
>                I don't understand this , for example like Npc in a game
>
>                function AI()
>                        while(true) do
>                                walkto(A)
>                                walkto(B)
>                                ...
>                                walkto(Z)
>                        end
>                end
>
>                I bind each npc with a lua_State by lua_newthread
>
>                but the npc will die and i should close the lua_State ,
>         but I can
>         not.
>                what can I do ? or some better suggestion ?
>
>         Best Regards
>         mos
>
>
>


Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Tom Miles-2
Title: RE: About lua_newthread
My point though was that there is no item of type "thread" on the stack after I create the thread.  Lua seems to be saving it on the stack as type userdata.   What is even more bizarre (to me), is that if I omit the part where I store the thread as a key in the globals table, then the single item at the point of a deletion comes out as type 'string' and the removal check fails.  It seems I have no understanding of the mysteries of the Lua engine :)
 
It is at this point that I think I would like to support Mos' call for a lua_deletethread() function in the next version :)
 
Tom
-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Jerome Vuarand
Sent: 07 June 2006 21:29
To: Lua list
Subject: RE: About lua_newthread

I think the line you point is the right one. Values are not typed in C API, stack entries are. So if you push a void* as a light user data, it is a light user data for Lua. lua_newthread already pushes the lua_State on the stack as a thread, so you don't need to explicitly do it.


De : [hidden email] [mailto:[hidden email]] De la part de Tom Miles
Envoyé : 7 juin 2006 12:18
À : Lua list
Objet : RE: About lua_newthread

Thanks for the speedy response. 



>> Can you show us how you initialize your thread??

I am using an approach similar to the one in Game Programming Gems 5, where I have a manager object that controls the main state and dishes out threads when requested.  I.e.

// 

static void CreateThread(State & state)
{
        if (m_MainState == NULL)
        {
                m_MainState = luaL_newstate();
                luaL_openlibs(m_MainState);

                luaL_openlib(m_MainState, "Script", scriptLib, 0);
        }

        state.m_state = lua_newthread(m_MainState);

        // save a pointer to the thread manager object in the global table
        // using the new thread's vm pointer as a key
        lua_pushlightuserdata(m_MainState, state.m_state);              <--- Is this causing a problem?
        lua_pushlightuserdata(m_MainState, &state);
        lua_settable(m_MainState, LUA_GLOBALSINDEX );
}

static void ReleaseThread(State & state)
{
        int stackSize = lua_gettop(m_MainState);
        for (int n=0; n<stackSize; ++n)
        {
                // This check should be: lua_isthread() && lua_tothread() == state.m_state
                if (lua_touserdata(m_MainState, n) == state.m_state)
                {
                        lua_remove(m_MainState, n);
                        break;
                }
        }
        // For the time being, set the value of the key (the state we are removing) in our map to be nil
        lua_pushlightuserdata(m_MainState, state.m_state);
        lua_pushnil(m_MainState);
        lua_settable(m_MainState, LUA_GLOBALSINDEX);

        // There is also code to close down lua if we have released the last thread here
}

>> If you use a weak value table, the entry will be deleted when the object
>> is collected.

>> Just put "v" in field __mode of the metatable of your table. If you want
>> weak key, just put "k".

As you can see, I am using the globals table, but I suppose I could create a weak table especially for the job instead.

> Tom

>  -----Original Message-----
> *From:* [hidden email]
> [[hidden email]]*On Behalf Of* Raymond Jacobs
> *Sent:* 18 May 2006 16:20
> *To:* Lua list
> *Subject:* Re: About lua_newthread
>
>     When you are done with a thread (when lua_resume returns and it's
>     not a yield), you should remove the thread directly from the
>     stack, this will remove the reference to it and it can then be
>     garbage collected (its a pain I know =/).
>     
>     another method which I've not yet done but should work, is to
>     create a new thread, create a lua reference to it (using lua_ref)
>     then pop it off the stack immediately (so it doesn't get buried),
>     and later when you are done with it, destroy the reference
>     (lua_unref) and that should allow it to be GC'd
>     
>     hope that helps,
>     
>     Raymond Jacobs
>     Owner,
>     Ethereal Darkness Interactive
>
>     
>     On 5/18/06, *mos* <[hidden email] <[hidden email]>>
>     wrote:
>
>
>         Hi
>
>         in lua5.1 manual:
>
>         lua_newthread
>                  lua_State *lua_newthread (lua_State *L);
>         There is no explicit function to close or to destroy a thread.
>         Threads are
>         subject to garbage collection, like any Lua object
>
>                I don't understand this , for example like Npc in a game
>
>                function AI()
>                        while(true) do
>                                walkto(A)
>                                walkto(B)
>                                ....
>                                walkto(Z)
>                        end
>                end
>
>                I bind each npc with a lua_State by lua_newthread
>
>                but the npc will die and i should close the lua_State ,
>         but I can
>         not.
>                what can I do ? or some better suggestion ?
>
>         Best Regards
>         mos
>
>
>


Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Tom Miles-2
Title: RE: About lua_newthread

Hmmm.  Previous posts on this sort of topic have stated that you can just remove the thread handle from the stack, hence my previous posts.  To be honest, I'm not sure what the creation of the local environment acheives?  Could you explain that part for me?

Thanks,

Tom

-----Original Message-----
From: [hidden email]
[[hidden email]]On Behalf Of Julien Hamaide
Sent: 07 June 2006 17:34
To: Lua list
Subject: Re: About lua_newthread


The problem with this approach is that the thread will never be garbage
collected as lua_newthread push a thread handle on the main thread stack.

So at the end of this call, you still have the thread handle on the stack.

I also see that you use the global table to store thread function. Pay
attention that if 2 thread have functions with the same name, they will
collide.

I use same kind of thing for a game, this is my approach for both
liveness of thread and local space :

In my Thread constructor :
  
    // master state is my main thread
    ThreadState = lua_newthread( master_state );

    // prevent thread from being garbage collected
    lua_pushvalue( master_state, -1 );
    ThreadReferenceIdentifier = luaL_ref( master_state, LUA_REGISTRYINDEX );

    //Create a local environment with a link to global environment via
__index metamethod
    lua_newtable( master_state );
    lua_pushvalue( master_state, -1 );
    lua_setmetatable( master_state, -2 ); //Set itself as metatable
    lua_pushvalue( master_state, LUA_GLOBALSINDEX );
    lua_setfield( master_state, -2, "__index" );
    lua_setfenv( master_state, -2 );
    lua_pop( master_state, 1 );

and in the destructor :

    luaL_unref( master_state, LUA_REGISTRYINDEX,
ThreadReferenceIdentifier );

If you have any questions about it, don't hesitate

Julien Hamaide
Lead Programmer
10Tacle Studios Belgium


Tom Miles a écrit :

> Thanks for the speedy response.
>
>
>
> >> Can you show us how you initialize your thread??
>
> I am using an approach similar to the one in Game Programming Gems 5,
> where I have a manager object that controls the main state and dishes
> out threads when requested.  I.e.
>
> //
>
> static void CreateThread(State & state)
> {
>         if (m_MainState == NULL)
>         {
>                 m_MainState = luaL_newstate();
>                 luaL_openlibs(m_MainState);
>
>                 luaL_openlib(m_MainState, "Script", scriptLib, 0);
>         }
>
>         state.m_state = lua_newthread(m_MainState);
>
>         // save a pointer to the thread manager object in the global
> table
>         // using the new thread's vm pointer as a key
>         lua_pushlightuserdata(m_MainState, state.m_state);     
>         <--- Is this causing a problem?
>         lua_pushlightuserdata(m_MainState, &state);
>         lua_settable(m_MainState, LUA_GLOBALSINDEX );
> }
>
> static void ReleaseThread(State & state)
> {
>         int stackSize = lua_gettop(m_MainState);
>         for (int n=0; n<stackSize; ++n)
>         {
>                 // This check should be: lua_isthread() &&
> lua_tothread() == state.m_state
>                 if (lua_touserdata(m_MainState, n) == state.m_state)
>                 {
>                         lua_remove(m_MainState, n);
>                         break;
>                 }
>         }
>         // For the time being, set the value of the key (the state we
> are removing) in our map to be nil
>         lua_pushlightuserdata(m_MainState, state.m_state);
>         lua_pushnil(m_MainState);
>         lua_settable(m_MainState, LUA_GLOBALSINDEX);
>
>         // There is also code to close down lua if we have released
> the last thread here
> }
>
> >> If you use a weak value table, the entry will be deleted when the
> object
> >> is collected.
>
> >> Just put "v" in field __mode of the metatable of your table. If you
> want
> >> weak key, just put "k".
>
> As you can see, I am using the globals table, but I suppose I could
> create a weak table especially for the job instead.
>
> > Tom
> >
> >  -----Original Message-----
> > *From:* [hidden email]
> > [[hidden email]]*On Behalf Of* Raymond
> Jacobs
> > *Sent:* 18 May 2006 16:20
> > *To:* Lua list
> > *Subject:* Re: About lua_newthread
> >
> >     When you are done with a thread (when lua_resume returns and it's
> >     not a yield), you should remove the thread directly from the
> >     stack, this will remove the reference to it and it can then be
> >     garbage collected (its a pain I know =/).
> >    
> >     another method which I've not yet done but should work, is to
> >     create a new thread, create a lua reference to it (using lua_ref)
> >     then pop it off the stack immediately (so it doesn't get buried),
> >     and later when you are done with it, destroy the reference
> >     (lua_unref) and that should allow it to be GC'd
> >    
> >     hope that helps,
> >    
> >     Raymond Jacobs
> >     Owner,
> >     Ethereal Darkness Interactive
> >
> >    
> >     On 5/18/06, *mos* <[hidden email] <[hidden email]>>
> >     wrote:
> >
> >
> >         Hi
> >
> >         in lua5.1 manual:
> >
> >         lua_newthread
> >                  lua_State *lua_newthread (lua_State *L);
> >         There is no explicit function to close or to destroy a thread.
> >         Threads are
> >         subject to garbage collection, like any Lua object
> >
> >                I don't understand this , for example like Npc in a game
> >
> >                function AI()
> >                        while(true) do
> >                                walkto(A)
> >                                walkto(B)
> >                                ...
> >                                walkto(Z)
> >                        end
> >                end
> >
> >                I bind each npc with a lua_State by lua_newthread
> >
> >                but the npc will die and i should close the lua_State ,
> >         but I can
> >         not.
> >                what can I do ? or some better suggestion ?
> >
> >         Best Regards
> >         mos
> >
> >
> >
>
>

Reply | Threaded
Open this post in threaded view
|

Re: About lua_newthread

Javier Arevalo
In reply to this post by Tom Miles-2
Tom Miles wrote:
It is at this point that I think I would like to support Mos' call for a lua_deletethread() function in the next version :)

There are no delete functions in a garbage-collection environment... it may be hard to wrap your head around this, but it is central to the Lua philosophy.

--
Javier Arévalo
Pyro Studios

Reply | Threaded
Open this post in threaded view
|

Re: About lua_newthread

Julien Hamaide-2
In reply to this post by Tom Miles-2
As you know, function in lua is just a variable. When you create a thread, all global variable and functions from it are put inside the global environment, which by default is the global environment of the main state. So, for example, if a threads declare a global function Update, a variable Update is create in LUA_GLOBALINDEX table and is then available to all threads. The problem is if another thread create also a function Update, it overwrites the previous one, as the value of variable Update is just replace.

So by creating a local environment, each thread global are not copied inside main state global. The __index tricks still allow threads to access all global field that had been registered ( such as library ). __index is triggered when a field is not found in current table.

If you want, just make a test, create 2 script with same name function, you'll see that the function is overwritten

Julien

Tom Miles a écrit :

Hmmm. Previous posts on this sort of topic have stated that you can just remove the thread handle from the stack, hence my previous posts. To be honest, I'm not sure what the creation of the local environment acheives? Could you explain that part for me?

Thanks,

Tom

-----Original Message-----
From: [hidden email]
[[hidden email] Behalf Of Julien Hamaide
Sent: 07 June 2006 17:34
To: Lua list
Subject: Re: About lua_newthread


The problem with this approach is that the thread will never be garbage
collected as lua_newthread push a thread handle on the main thread stack.

So at the end of this call, you still have the thread handle on the stack.

I also see that you use the global table to store thread function. Pay
attention that if 2 thread have functions with the same name, they will
collide.

I use same kind of thing for a game, this is my approach for both
liveness of thread and local space :

In my Thread constructor :
// master state is my main thread
    ThreadState = lua_newthread( master_state );

    // prevent thread from being garbage collected
    lua_pushvalue( master_state, -1 );
ThreadReferenceIdentifier = luaL_ref( master_state, LUA_REGISTRYINDEX );

    //Create a local environment with a link to global environment via
__index metamethod
    lua_newtable( master_state );
    lua_pushvalue( master_state, -1 );
    lua_setmetatable( master_state, -2 ); //Set itself as metatable
    lua_pushvalue( master_state, LUA_GLOBALSINDEX );
    lua_setfield( master_state, -2, "__index" );
    lua_setfenv( master_state, -2 );
    lua_pop( master_state, 1 );

and in the destructor :

    luaL_unref( master_state, LUA_REGISTRYINDEX,
ThreadReferenceIdentifier );

If you have any questions about it, don't hesitate

Julien Hamaide
Lead Programmer
10Tacle Studios Belgium


Tom Miles a écrit :

> Thanks for the speedy response.
>
>
>
> >> Can you show us how you initialize your thread??
>
> I am using an approach similar to the one in Game Programming Gems 5,
> where I have a manager object that controls the main state and dishes
> out threads when requested.  I.e.
>
> //
>
> static void CreateThread(State & state)
> {
>         if (m_MainState == NULL)
>         {
>                 m_MainState = luaL_newstate();
>                 luaL_openlibs(m_MainState);
>
>                 luaL_openlib(m_MainState, "Script", scriptLib, 0);
>         }
>
>         state.m_state = lua_newthread(m_MainState);
>
>         // save a pointer to the thread manager object in the global
> table
>         // using the new thread's vm pointer as a key
> lua_pushlightuserdata(m_MainState, state.m_state); > <--- Is this causing a problem?
>         lua_pushlightuserdata(m_MainState, &state);
>         lua_settable(m_MainState, LUA_GLOBALSINDEX );
> }
>
> static void ReleaseThread(State & state)
> {
>         int stackSize = lua_gettop(m_MainState);
>         for (int n=0; n<stackSize; ++n)
>         {
>                 // This check should be: lua_isthread() &&
> lua_tothread() == state.m_state
>                 if (lua_touserdata(m_MainState, n) == state.m_state)
>                 {
>                         lua_remove(m_MainState, n);
>                         break;
>                 }
>         }
>         // For the time being, set the value of the key (the state we
> are removing) in our map to be nil
>         lua_pushlightuserdata(m_MainState, state.m_state);
>         lua_pushnil(m_MainState);
>         lua_settable(m_MainState, LUA_GLOBALSINDEX);
>
>         // There is also code to close down lua if we have released
> the last thread here
> }
>
> >> If you use a weak value table, the entry will be deleted when the
> object
> >> is collected.
>
> >> Just put "v" in field __mode of the metatable of your table. If you
> want
> >> weak key, just put "k".
>
> As you can see, I am using the globals table, but I suppose I could
> create a weak table especially for the job instead.
>
> > Tom
> >
> >  -----Original Message-----
> > *From:* [hidden email]
> > [[hidden email] Behalf Of* Raymond
> Jacobs
> > *Sent:* 18 May 2006 16:20
> > *To:* Lua list
> > *Subject:* Re: About lua_newthread
> >
> >     When you are done with a thread (when lua_resume returns and it's
> >     not a yield), you should remove the thread directly from the
> >     stack, this will remove the reference to it and it can then be
> >     garbage collected (its a pain I know =/).
> > > > another method which I've not yet done but should work, is to
> >     create a new thread, create a lua reference to it (using lua_ref)
> >     then pop it off the stack immediately (so it doesn't get buried),
> >     and later when you are done with it, destroy the reference
> >     (lua_unref) and that should allow it to be GC'd
> > > > hope that helps, > > > > Raymond Jacobs
> >     Owner,
> >     Ethereal Darkness Interactive
> >
> > > > On 5/18/06, *mos* <[hidden email] <[hidden email]>>
> >     wrote:
> >
> >
> >         Hi
> >
> >         in lua5.1 manual:
> >
> >         lua_newthread
> >                  lua_State *lua_newthread (lua_State *L);
> > There is no explicit function to close or to destroy a thread.
> >         Threads are
> >         subject to garbage collection, like any Lua object
> >
> > I don't understand this , for example like Npc in a game
> >
> >                function AI()
> >                        while(true) do
> >                                walkto(A)
> >                                walkto(B)
> >                                ...
> >                                walkto(Z)
> >                        end
> >                end
> >
> >                I bind each npc with a lua_State by lua_newthread
> >
> > but the npc will die and i should close the lua_State ,
> >         but I can
> >         not.
> >                what can I do ? or some better suggestion ?
> >
> >         Best Regards
> >         mos
> >
> >
> >
>
>



Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Tom Miles-2
In reply to this post by Tom Miles-2
Title: RE: About lua_newthread

Fair point I guess :)

-----Original Message-----
From: [hidden email]
[[hidden email]]On Behalf Of Javier Arevalo
Sent: 08 June 2006 10:22
To: Lua list
Subject: Re: About lua_newthread


Tom Miles wrote:
> It is at this point that I think I would like to support Mos' call for a
> lua_deletethread() function in the next version :)

There are no delete functions in a garbage-collection environment... it
may be hard to wrap your head around this, but it is central to the Lua
philosophy.

--
Javier Arévalo
Pyro Studios

Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Richard Ranft
In reply to this post by Javier Arevalo
> There are no delete functions in a garbage-collection environment... it
> may be hard to wrap your head around this, but it is central to the Lua
> philosophy.

Perhaps not "delete" per se, but C# has a Dispose() which I'm sure forces a
GC flag on the object....  Ok, I'm not "sure," but I think we can make a
strong assumption there.  Something like this could not only mark the object
for deletion but trace back through the reference stack and notify other
constructs that have references to this object that it is about to go away
so that they can request replacement or countermand the deletion.

Sorry - rambling.

Rich


Reply | Threaded
Open this post in threaded view
|

Re: About lua_newthread

Javier Arevalo
Richard Ranft wrote:
C# has a Dispose() which I'm sure forces a
GC flag on the object....  Ok, I'm not "sure," but I think we can make a
strong assumption there.

No no, .NET's Dispose() is just their standardization (sp?) of the interface for releasing resources owned by classes. It isn't directly related to the lifetime of the object.

Something like this could not only mark the object
for deletion but trace back through the reference stack and notify other
constructs that have references to this object that it is about to go away
so that they can request replacement or countermand the deletion.

It is very common for language users to create their own sub-languages, coding conventions, protocols, however you call it. You are free to implement your vision of deletable garbage-collected objects, but I don't think it should be part of the standard Lua system; certainly not with the overhead of what you describe.

A fairly common approach in games and GUI systems is to mark objects as "dead" (i.e. wants to be deleted). Everyone who references them must check for this during regular operations, and eliminate their own reference if the object is flagged as dead. Simple and lightweight.

If Lua shipped with a number of standard classes and libraries, it would probably need to define a larger number of protocols like these.

--
Javier Arévalo
Pyro Studios

Reply | Threaded
Open this post in threaded view
|

Re: About lua_newthread

Lisa Parratt
In reply to this post by Tom Miles-2
Tom Miles wrote:
Hi,
Sorry about the the thread necromancy, but I have just encountered this situation in the project I am currently working on. I am running several threads from my main thread, all of which are just loaded with functions to call in an event based system. At various intervals I want to "unload" these functions and load a new set, so I want to shut down the thread. I therefore need to remove references to my thread from the stack as per the suggestion, however, when I iterate through the stack all that is on there is a userdata of indeterminate origin, not a thread. (Having said that, I've just discovered that if I do lua_touserdata() and compare it to the address of the state returned with lua_newthread(), then they are the same, so I have found my thread - but there seems to be a little bug there as it isn't of type thread?).
It strikes me the crux of this problem isn't that there's not a function to delete a thread, but simply that you're using the stack as a persistent store for thread objects. Take them off the stack, stash them some where more sensible - a member of the table representing the actor the thread represents, or a schedule table, or other similar options.
On a similar note, is there a way of removing a non integer index from a table? I use the thread as a key in a table to match it up to a game object, and when that object is destroyed I want to remove the entry in the table as well.
As others have said - weak tables.

--
Lisa

Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Tom Miles-2
In reply to this post by Tom Miles-2
Title: RE: About lua_newthread

> It strikes me the crux of this problem isn't that there's not a function
> to delete a thread, but simply that you're using the stack as a
> persistent store for thread objects. Take them off the stack, stash them
> some where more sensible - a member of the table representing the actor
> the thread represents, or a schedule table, or other similar options.

That's kind of true.  I'm not explicitly using the stack for storage, but seeing as I'm not doing anything with it either, I guess I am.  It was a very naive approach that I've had lots of guidance with now :)

> > On a similar note, is there a way of removing a non integer index from
> > a table?  I use the thread as a key in a table to match it up to a
> > game object, and when that object is destroyed I want to remove the
> > entry in the table as well.

> As others have said - weak tables.

Yup. :)

Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Tom Miles-2
In reply to this post by Tom Miles-2
Title: RE: About lua_newthread

Thanks for all that.  I didn't realise that functions added to one thread are added to the global environment.  Seems a bit funny, but my brain has been trained into thinking "C++" for 10 years :)

> -----Original Message-----
> From: [hidden email]
> [[hidden email]]On Behalf Of
> Julien Hamaide
> Sent: 08 June 2006 10:26
> To: Lua list
> Subject: Re: About lua_newthread
>
>
> As you know, function in lua is just a variable. When you create a
> thread, all global variable and functions from it are put inside the
> global environment, which by default is the global environment of the
> main state. So, for example, if a threads declare a global function
> Update, a variable Update is create in LUA_GLOBALINDEX table
> and is then
> available to all threads. The problem is if another thread
> create also a
> function Update, it overwrites the previous one, as the value of
> variable Update is just replace.
>
> So by creating a local environment, each thread global are not copied
> inside main state global.
> The __index tricks still allow threads to access all global
> field that
> had been registered ( such as library ). __index is triggered when a
> field is not found in current table.
>
> If you want, just make a test, create 2 script with same name
> function,
> you'll see that the function is overwritten
>
> Julien
>
> Tom Miles a écrit :
>
> > Hmmm.  Previous posts on this sort of topic have stated
> that you can
> > just remove the thread handle from the stack, hence my previous
> > posts.  To be honest, I'm not sure what the creation of the local
> > environment acheives?  Could you explain that part for me?
> >
> > Thanks,
> >
> > Tom
> >
> > -----Original Message-----
> > From: [hidden email]
> > [[hidden email]]On Behalf Of
> Julien Hamaide
> > Sent: 07 June 2006 17:34
> > To: Lua list
> > Subject: Re: About lua_newthread
> >
> >
> > The problem with this approach is that the thread will
> never be garbage
> > collected as lua_newthread push a thread handle on the main
> thread stack.
> >
> > So at the end of this call, you still have the thread handle on the
> > stack.
> >
> > I also see that you use the global table to store thread
> function. Pay
> > attention that if 2 thread have functions with the same
> name, they will
> > collide.
> >
> > I use same kind of thing for a game, this is my approach for both
> > liveness of thread and local space :
> >
> > In my Thread constructor :
> >  
> >     // master state is my main thread
> >     ThreadState = lua_newthread( master_state );
> >
> >     // prevent thread from being garbage collected
> >     lua_pushvalue( master_state, -1 );
> >     ThreadReferenceIdentifier = luaL_ref( master_state,
> > LUA_REGISTRYINDEX );
> >
> >     //Create a local environment with a link to global
> environment via
> > __index metamethod
> >     lua_newtable( master_state );
> >     lua_pushvalue( master_state, -1 );
> >     lua_setmetatable( master_state, -2 ); //Set itself as metatable
> >     lua_pushvalue( master_state, LUA_GLOBALSINDEX );
> >     lua_setfield( master_state, -2, "__index" );
> >     lua_setfenv( master_state, -2 );
> >     lua_pop( master_state, 1 );
> >
> > and in the destructor :
> >
> >     luaL_unref( master_state, LUA_REGISTRYINDEX,
> > ThreadReferenceIdentifier );
> >
> > If you have any questions about it, don't hesitate
> >
> > Julien Hamaide
> > Lead Programmer
> > 10Tacle Studios Belgium
> >
> >
> > Tom Miles a écrit :
> >
> > > Thanks for the speedy response.
> > >
> > >
> > >
> > > >> Can you show us how you initialize your thread??
> > >
> > > I am using an approach similar to the one in Game
> Programming Gems 5,
> > > where I have a manager object that controls the main
> state and dishes
> > > out threads when requested.  I.e.
> > >
> > > //
> > >
> > > static void CreateThread(State & state)
> > > {
> > >         if (m_MainState == NULL)
> > >         {
> > >                 m_MainState = luaL_newstate();
> > >                 luaL_openlibs(m_MainState);
> > >
> > >                 luaL_openlib(m_MainState, "Script", scriptLib, 0);
> > >         }
> > >
> > >         state.m_state = lua_newthread(m_MainState);
> > >
> > >         // save a pointer to the thread manager object in
> the global
> > > table
> > >         // using the new thread's vm pointer as a key
> > >         lua_pushlightuserdata(m_MainState, state.m_state);    
> > >         <--- Is this causing a problem?
> > >         lua_pushlightuserdata(m_MainState, &state);
> > >         lua_settable(m_MainState, LUA_GLOBALSINDEX );
> > > }
> > >
> > > static void ReleaseThread(State & state)
> > > {
> > >         int stackSize = lua_gettop(m_MainState);
> > >         for (int n=0; n<stackSize; ++n)
> > >         {
> > >                 // This check should be: lua_isthread() &&
> > > lua_tothread() == state.m_state
> > >                 if (lua_touserdata(m_MainState, n) ==
> state.m_state)
> > >                 {
> > >                         lua_remove(m_MainState, n);
> > >                         break;
> > >                 }
> > >         }
> > >         // For the time being, set the value of the key
> (the state we
> > > are removing) in our map to be nil
> > >         lua_pushlightuserdata(m_MainState, state.m_state);
> > >         lua_pushnil(m_MainState);
> > >         lua_settable(m_MainState, LUA_GLOBALSINDEX);
> > >
> > >         // There is also code to close down lua if we
> have released
> > > the last thread here
> > > }
> > >
> > > >> If you use a weak value table, the entry will be
> deleted when the
> > > object
> > > >> is collected.
> > >
> > > >> Just put "v" in field __mode of the metatable of your
> table. If you
> > > want
> > > >> weak key, just put "k".
> > >
> > > As you can see, I am using the globals table, but I
> suppose I could
> > > create a weak table especially for the job instead.
> > >
> > > > Tom
> > > >
> > > >  -----Original Message-----
> > > > *From:* [hidden email]
> > > > [[hidden email]]*On Behalf
> Of* Raymond
> > > Jacobs
> > > > *Sent:* 18 May 2006 16:20
> > > > *To:* Lua list
> > > > *Subject:* Re: About lua_newthread
> > > >
> > > >     When you are done with a thread (when lua_resume
> returns and it's
> > > >     not a yield), you should remove the thread directly from the
> > > >     stack, this will remove the reference to it and it
> can then be
> > > >     garbage collected (its a pain I know =/).
> > > >   
> > > >     another method which I've not yet done but should
> work, is to
> > > >     create a new thread, create a lua reference to it
> (using lua_ref)
> > > >     then pop it off the stack immediately (so it
> doesn't get buried),
> > > >     and later when you are done with it, destroy the reference
> > > >     (lua_unref) and that should allow it to be GC'd
> > > >   
> > > >     hope that helps,
> > > >   
> > > >     Raymond Jacobs
> > > >     Owner,
> > > >     Ethereal Darkness Interactive
> > > >
> > > >   
> > > >     On 5/18/06, *mos* <[hidden email]
<[hidden email]>>
> > >     wrote:
> > >
> > >
> > >         Hi
> > >
> > >         in lua5.1 manual:
> > >
> > >         lua_newthread
> > >                  lua_State *lua_newthread (lua_State *L);
> > >         There is no explicit function to close or to destroy a
> thread.
> > >         Threads are
> > >         subject to garbage collection, like any Lua object
> > >
> > >                I don't understand this , for example like Npc in a
> game
> > >
> > >                function AI()
> > >                        while(true) do
> > >                                walkto(A)
> > >                                walkto(B)
> > >                                ...
> > >                                walkto(Z)
> > >                        end
> > >                end
> > >
> > >                I bind each npc with a lua_State by lua_newthread
> > >
> > >                but the npc will die and i should close the
> lua_State ,
> > >         but I can
> > >         not.
> > >                what can I do ? or some better suggestion ?
> > >
> > >         Best Regards
> > >         mos
> > >
> > >
> > >
> >
> >
>

Reply | Threaded
Open this post in threaded view
|

RE: About lua_newthread

Richard Ranft
In reply to this post by Javier Arevalo
> No no, .NET's Dispose() is just their standardization (sp?) of the
> interface for releasing resources owned by classes. It isn't directly
> related to the lifetime of the object.

Thanks for clearing that up for me Javier - pretty good to know.  I know
that .NET is supposed to clean up after itself, but I figured that this was
the programmer's way of explicitly telling the GC that he was done with the
object.

> It is very common for language users to create their own sub-languages,
> coding conventions, protocols, however you call it. You are free to
> implement your vision of deletable garbage-collected objects, but I
> don't think it should be part of the standard Lua system; certainly not
> with the overhead of what you describe.

I wasn't necessarily advocating or recommending, just thinking on paper.  My
intended meaning was similar to what you're saying - the application could
extend Lua to include these features if it would be more comfortable for the
team - not that these should be added to Lua.  I think Lua has about all the
functionality it needs because it can be easily extended to suit a
particular application.

Again - thanks for clarifying.  This list is really cool.

Rich