table deep copy in C/CPP

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

table deep copy in C/CPP

Mariusz Stanisz
Hi,
 I need do a deep copy of lua table in C/CPP does anyone have an example?


Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Luiz Henrique de Figueiredo
>  I need do a deep copy of lua table in C/CPP does anyone have an example?

You might start from http://lua-users.org/wiki/CopyTable

But why do you need a deep copy? I never needed to copy tables, let alone
deep copy.
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mariusz Stanisz
I know that deep copy function. I need a function in C. I'm going to be  passing lua tables between pthreads, these are separate lua states. 
So i have to take a table of one stack copy and push it onto another stack.

On Tue, Mar 16, 2010 at 4:48 PM, Luiz Henrique de Figueiredo <[hidden email]> wrote:
>  I need do a deep copy of lua table in C/CPP does anyone have an example?

You might start from http://lua-users.org/wiki/CopyTable

But why do you need a deep copy? I never needed to copy tables, let alone
deep copy.


Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Luiz Henrique de Figueiredo
> I know that deep copy function. I need a function in C. I'm going to be
>  passing lua tables between pthreads, these are separate lua states.
> So i have to take a table of one stack copy and push it onto another stack.

Here's is totally untested code, just to give you an idea.

static int xcopy1(lua_State *L, lua_State *T, int n)
{
 switch (lua_type(L,n))
 {
  case LUA_TNIL:
    lua_pushnil(T);
    break;
  case LUA_TBOOLEAN:
    lua_pushboolean(T,lua_toboolean(L,n));
    break;
  case LUA_TNUMBER:
    lua_pushnumber(T,lua_tonumber(L,n));
    break;
  case LUA_TSTRING:
    lua_pushlstring(T,lua_tostring(L,n),lua_strlen(L,n));
    break;
  case LUA_TLIGHTUSERDATA:
    lua_pushlightuserdata(T,(void*)lua_touserdata(L,n));
    break;
  default:
    assert(0);
    break;
 }
}

/* table is in the stack at index 't' */
static int xcopy(lua_State *L, lua_State *T, int t)
{
 int w;
 lua_newtable(T);
 w=lua_gettop(T);
 lua_pushnil(L);  /* first key */
 while (lua_next(L, t) != 0) {
        xcopy1(L,T,-2);
        if (lua_type(L,-1)==LUA_TTABLE)
                xcopy(L,T,lua_gettop(L));
        else
                xcopy1(L,T,-1);
        lua_settable(T,w);
        lua_pop(L,1);
 }
}
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

liam mail
IIRC Lua Lanes has this functionality, there may also be others on Lua forge that escape me for the moment.

Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Pan, Shi Zhu
In reply to this post by Mariusz Stanisz
Does your table contains itself or contains the same table in multiple keys?

If not, then just serialize it, copy it as string, then reconstruct to
a new table.

If yes, then things are fairly complicated...

On Wed, Mar 17, 2010 at 5:45 AM, Mariusz Stanisz
<[hidden email]> wrote:
> Hi,
>  I need do a deep copy of lua table in C/CPP does anyone have an example?
>
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mark Hamburg
In reply to this post by Luiz Henrique de Figueiredo
On Mar 16, 2010, at 3:54 PM, Luiz Henrique de Figueiredo wrote:

>> I know that deep copy function. I need a function in C. I'm going to be
>> passing lua tables between pthreads, these are separate lua states.
>> So i have to take a table of one stack copy and push it onto another stack.
>
> Here's is totally untested code, just to give you an idea.
>
> static int xcopy1(lua_State *L, lua_State *T, int n)
> {
> switch (lua_type(L,n))
> {
>  case LUA_TNIL:
>    lua_pushnil(T);
>    break;
>  case LUA_TBOOLEAN:
>    lua_pushboolean(T,lua_toboolean(L,n));
>    break;
>  case LUA_TNUMBER:
>    lua_pushnumber(T,lua_tonumber(L,n));
>    break;
>  case LUA_TSTRING:
>    lua_pushlstring(T,lua_tostring(L,n),lua_strlen(L,n));
>    break;
>  case LUA_TLIGHTUSERDATA:
>    lua_pushlightuserdata(T,(void*)lua_touserdata(L,n));
>    break;
>  default:
>    assert(0);
>    break;
> }
> }
>
> /* table is in the stack at index 't' */
> static int xcopy(lua_State *L, lua_State *T, int t)
> {
> int w;
> lua_newtable(T);
> w=lua_gettop(T);
> lua_pushnil(L);  /* first key */
> while (lua_next(L, t) != 0) {
> xcopy1(L,T,-2);
> if (lua_type(L,-1)==LUA_TTABLE)
> xcopy(L,T,lua_gettop(L));
> else
> xcopy1(L,T,-1);
> lua_settable(T,w);
> lua_pop(L,1);
> }
> }

Having written similar code in the past, here are some notes on Luiz's generally nice and certainly concise code:

1. It doesn't handle tables as keys. But it does assert so you at least won't continue blindly on (though you may still not be happy when assert terminates the program). This probably doesn't matter for your intended use case.
2. It doesn't recognize shared tables. This probably isn't an issue for your use case.
3. It doesn't depth check. This is only a problem if you have cyclic data structures.
4. It has trickier error behavior than one would like because you can get errors on either Lua state and probably at least one of these isn't running protected at this point.

Of these, only the fourth really matters in most cases, but it's a pain to fix. If you depth limit, you can grow the source stack and then do everything within a cpcall for the destination state though you then have to worry about how to get a value out of the cpcall. Lua 5.2, I believe, addresses that. Otherwise, you stick it in the registry or you do the work in a temporary Lua thread and then transfer it to the stack for the main thread (though of course that requires allocating the temporary thread safely...) Lua 5.2 may make some of the other logic simpler as well through its improved cpcall interface.

Serialization in one state and deserialization in another becomes very tempting because it also allows one to reduce synchronization.

Mark

Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Pan, Shi Zhu
>
> 1. It doesn't handle tables as keys. But it does assert so you at least won't continue blindly on (though you may still not be happy when assert terminates the program). This probably doesn't matter for your intended use case.
> 2. It doesn't recognize shared tables. This probably isn't an issue for your use case.
> 3. It doesn't depth check. This is only a problem if you have cyclic data structures.
> 4. It has trickier error behavior than one would like because you can get errors on either Lua state and probably at least one of these isn't running protected at this point.
>
> Serialization in one state and deserialization in another becomes very tempting because it also allows one to reduce synchronization.
>

This is what I had done.
1. I can handle tables as keys.
2. I ignore shared tables, only the first data get copied.
3. I ignore cylic data structures, cylic data structures are not copied.
4. serialization/deserialization solves the synchronization problem so
that both Lua states can run in protected mode.

What I cannot deal with is

1. userdata structures, I simply ignored all unrecognized data. That
is not an issue for me.

2. functions, functions can be saved in key and values, I haven't
found a decent way to copy a function from one lua-state to another
lua-state. For this reason I don't think my code are very much useful.

since lua is a function-programming language I think a decent
serialization / deserialization should deal with function.
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mariusz Stanisz

What do you guys mean by "handle tables as keys" and "shared tables" . Can you guys give me a lua example of each. Pan i still would be interested in seeing your code if possible.

On Wed, Mar 17, 2010 at 12:13 AM, Pan Shi Zhu <[hidden email]> wrote:
>
> 1. It doesn't handle tables as keys. But it does assert so you at least won't continue blindly on (though you may still not be happy when assert terminates the program). This probably doesn't matter for your intended use case.
> 2. It doesn't recognize shared tables. This probably isn't an issue for your use case.
> 3. It doesn't depth check. This is only a problem if you have cyclic data structures.
> 4. It has trickier error behavior than one would like because you can get errors on either Lua state and probably at least one of these isn't running protected at this point.
>
> Serialization in one state and deserialization in another becomes very tempting because it also allows one to reduce synchronization.
>

This is what I had done.
1. I can handle tables as keys.
2. I ignore shared tables, only the first data get copied.
3. I ignore cylic data structures, cylic data structures are not copied.
4. serialization/deserialization solves the synchronization problem so
that both Lua states can run in protected mode.

What I cannot deal with is

1. userdata structures, I simply ignored all unrecognized data. That
is not an issue for me.

2. functions, functions can be saved in key and values, I haven't
found a decent way to copy a function from one lua-state to another
lua-state. For this reason I don't think my code are very much useful.

since lua is a function-programming language I think a decent
serialization / deserialization should deal with function.




Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Philippe Lhoste
On 17/03/2010 16:39, Mariusz Stanisz wrote:
> What do you guys mean by "handle tables as keys" and "shared tables" .

local t = { 1, 2, 3 } -- shared table
local ct =
{
   a = t,
   b =
   {
     c = "a",
     d = t
   },
   [t] = "Table as key"
}

--
Philippe Lhoste
--  (near) Paris -- France
--  http://Phi.Lho.free.fr
--  --  --  --  --  --  --  --  --  --  --  --  --  --

Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mariusz Stanisz
Tx

On Wed, Mar 17, 2010 at 10:53 AM, Philippe Lhoste <[hidden email]> wrote:
On 17/03/2010 16:39, Mariusz Stanisz wrote:
What do you guys mean by "handle tables as keys" and "shared tables" .

local t = { 1, 2, 3 } -- shared table
local ct =
{
 a = t,
 b =
 {
   c = "a",
   d = t
 },
 [t] = "Table as key"
}

--
Philippe Lhoste
--  (near) Paris -- France
--  http://Phi.Lho.free.fr
--  --  --  --  --  --  --  --  --  --  --  --  --  --



Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mariusz Stanisz
What about using lua_xmove. Does it handle table copy?

void lua_xmove (lua_State *from, lua_State *to, int n);

On Wed, Mar 17, 2010 at 11:01 AM, Mariusz Stanisz <[hidden email]> wrote:
Tx


On Wed, Mar 17, 2010 at 10:53 AM, Philippe Lhoste <[hidden email]> wrote:
On 17/03/2010 16:39, Mariusz Stanisz wrote:
What do you guys mean by "handle tables as keys" and "shared tables" .

local t = { 1, 2, 3 } -- shared table
local ct =
{
 a = t,
 b =
 {
   c = "a",
   d = t
 },
 [t] = "Table as key"
}

--
Philippe Lhoste
--  (near) Paris -- France
--  http://Phi.Lho.free.fr
--  --  --  --  --  --  --  --  --  --  --  --  --  --





Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Luiz Henrique de Figueiredo
> What about using lua_xmove. Does it handle table copy?

No. It only copies references, because it only works for sister states,
not for disjoint states. (Sister states are different threads of the
same global state.)
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Pan, Shi Zhu
In reply to this post by Mariusz Stanisz
This is my personal utilities module. Please ignore unrelevant stuffs
and concentrate only on the "repr" and "unrepr" function.

You will notice some linux-only headers in the source, please just
remove them if you're in windows, my "repr" and "unrepr" function does
not use them.

No build scripts are provided, since you're lua-c developer you should
have good knowledge about how to use only some of the functions in it.

To use the whole module without modification, you can just put it into
lua directory and compile it with lua, the call luaopen_tools() after
luaL_openlibs().

On Wed, Mar 17, 2010 at 11:39 PM, Mariusz Stanisz
<[hidden email]> wrote:
>
> What do you guys mean by "handle tables as keys" and "shared tables" . Can
> you guys give me a lua example of each. Pan i still would be interested in
> seeing your code if possible.
>

this means tables as keys:

a = {1,2}
b = {3,4}
c = { [a]=5, [b] = 6}

this means shared table:

a = {1,2}
b = {3, 4, a}
c = {a, b}

ltools.c (45K) Download Attachment
luatools.h (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mariusz Stanisz
. Great! Very clean code! Between your source code and lua lanes tools.c, I think I can come up with a good solution.

On Wed, Mar 17, 2010 at 8:12 PM, Pan Shi Zhu <[hidden email]> wrote:
This is my personal utilities module. Please ignore unrelevant stuffs
and concentrate only on the "repr" and "unrepr" function.

You will notice some linux-only headers in the source, please just
remove them if you're in windows, my "repr" and "unrepr" function does
not use them.

No build scripts are provided, since you're lua-c developer you should
have good knowledge about how to use only some of the functions in it.

To use the whole module without modification, you can just put it into
lua directory and compile it with lua, the call luaopen_tools() after
luaL_openlibs().

On Wed, Mar 17, 2010 at 11:39 PM, Mariusz Stanisz
<[hidden email]> wrote:
>
> What do you guys mean by "handle tables as keys" and "shared tables" . Can
> you guys give me a lua example of each. Pan i still would be interested in
> seeing your code if possible.
>

this means tables as keys:

a = {1,2}
b = {3,4}
c = { [a]=5, [b] = 6}

this means shared table:

a = {1,2}
b = {3, 4, a}
c = {a, b}



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Asko Kauppi
In reply to this post by Pan, Shi Zhu

If you are interested in copying functions, the Lanes code can do  
that. It also copies upvalues.

-asko


Pan Shi Zhu kirjoitti 17.3.2010 kello 7:13:

>>
>> 1. It doesn't handle tables as keys. But it does assert so you at  
>> least won't continue blindly on (though you may still not be happy  
>> when assert terminates the program). This probably doesn't matter  
>> for your intended use case.
>> 2. It doesn't recognize shared tables. This probably isn't an issue  
>> for your use case.
>> 3. It doesn't depth check. This is only a problem if you have  
>> cyclic data structures.
>> 4. It has trickier error behavior than one would like because you  
>> can get errors on either Lua state and probably at least one of  
>> these isn't running protected at this point.
>>
>> Serialization in one state and deserialization in another becomes  
>> very tempting because it also allows one to reduce synchronization.
>>
>
> This is what I had done.
> 1. I can handle tables as keys.
> 2. I ignore shared tables, only the first data get copied.
> 3. I ignore cylic data structures, cylic data structures are not  
> copied.
> 4. serialization/deserialization solves the synchronization problem so
> that both Lua states can run in protected mode.
>
> What I cannot deal with is
>
> 1. userdata structures, I simply ignored all unrecognized data. That
> is not an issue for me.
>
> 2. functions, functions can be saved in key and values, I haven't
> found a decent way to copy a function from one lua-state to another
> lua-state. For this reason I don't think my code are very much useful.
>
> since lua is a function-programming language I think a decent
> serialization / deserialization should deal with function.

Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Pan, Shi Zhu
On Sat, Mar 20, 2010 at 4:54 PM, Asko Kauppi <[hidden email]> wrote:
>
> If you are interested in copying functions, the Lanes code can do that. It
> also copies upvalues.
>

great, I'll check it.
Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mark Hamburg
In reply to this post by Asko Kauppi
On Mar 20, 2010, at 1:54 AM, Asko Kauppi wrote:

> If you are interested in copying functions, the Lanes code can do that. It also copies upvalues.
>
> -asko

Does it also copy the environment? How does it decide where to stop? For example, if I stick math.sin into a local variable math_sin and reference this as an upvalue in the function I'm passing, what does it do?

Mark

Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Mariusz Stanisz
Yes just look at the tools.c under lua lanes , It does everything accept threads. Here are the function you should look at
luaG_inter_copy and inter_copy_one. Only thing this file depends on is threading.h , you can use it or just redefine your own MUTEX_LOCK,MUTEX_UNLOCK and MUTEX_T. If you do use there threading.h you will need to patch line 170
put the following line. extern volatile bool_t sudo; or you will get a ton link errors.

On Sat, Mar 20, 2010 at 12:27 PM, Mark Hamburg <[hidden email]> wrote:
On Mar 20, 2010, at 1:54 AM, Asko Kauppi wrote:

> If you are interested in copying functions, the Lanes code can do that. It also copies upvalues.
>
> -asko

Does it also copy the environment? How does it decide where to stop? For example, if I stick math.sin into a local variable math_sin and reference this as an upvalue in the function I'm passing, what does it do?

Mark




Reply | Threaded
Open this post in threaded view
|

Re: table deep copy in C/CPP

Asko Kauppi
In reply to this post by Mark Hamburg

Mark Hamburg kirjoitti 20.3.2010 kello 19:27:

> On Mar 20, 2010, at 1:54 AM, Asko Kauppi wrote:
>
>> If you are interested in copying functions, the Lanes code can do  
>> that. It also copies upvalues.
>>
>> -asko
>
> Does it also copy the environment? How does it decide where to stop?  
> For example, if I stick math.sin into a local variable math_sin and  
> reference this as an upvalue in the function I'm passing, what does  
> it do?

I think it did not copy the environment, though that could actually be  
implemented if needed.

It would copy the function given as upvalue. It only stops when  
there's a recursion (i.e. it cannot copy a function which has itself  
as upvalue, though even this could possibly be implemented, if needed).

One reason I made it was to see just how far one can go with existing  
Lua API. Quite far, actually.

The responsibility of not overusing it (since any copying would  
naturally have a runtime penalty) is up to the application programmer.

Naturally, once the copying has been done the two upvalues in the two  
separate Lua states are completely detached. There is a separate  
concept of shared states for inter-state communications.

- Asko



>
> Mark
>

12