Howto return kind userdata from C to Lua?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Howto return kind userdata from C to Lua?

Ranier Vilela-2
Hi hackers,

Is it possible to create and use a userdata value, this way?

typedef struct MyItem {
   int size;
} MyItem;

typedef struct MyValue {
  const char * name;
  MyItem * item;
} MyValue;

static int lua_myvalue_new(lua_State *L) {
MyValue * value;
const char * name;
size_t l;
int n;

name = luaL_checklstring(L, 1, &l);
if (name == NULL) {
    return 0;
}
n = luaL_checkinteger(L, 2);
luaL_argcheck(L, n >= 1, 2, "invalid size");

value = (MyValue *) lua_newuserdata(L, sizeof(MyValue));
if (value == NULL) {
    return 0;
}
value->name = name;
luaL_getmetatable(L, MYVALUE_REGISTRY);
lua_setmetatable(L, -2);

value->item = (MyItem *) lua_newuserdata(L, sizeof(MyItem));
if (value->item == NULL) {
    return 0;
}
value->item->size = n;
luaL_getmetatable(L, UIMYITEM_REGISTRY);
lua_setmetatable(L, -2);

return 1; /* Must return value (MyValue) */
}

Howto return to Lua MyValue struct (userdata)?
MyItem needs to be allocated by malloc rather than lua_newuserdata?

best regards,
Ranier Vilela
Reply | Threaded
Open this post in threaded view
|

Re: Howto return kind userdata from C to Lua?

Jonathan Goble
On Sat, Nov 7, 2020 at 1:30 PM Ranier Vilela <[hidden email]> wrote:
[...]
return 1; /* Must return value (MyValue) */

Howto return to Lua MyValue struct (userdata)?
MyItem needs to be allocated by malloc rather than lua_newuserdata?

Without testing, it should be (almost) as simple as making sure that at the line above, the MyValue userdata is on the top of the Lua stack. There are API functions to move things around on the stack. Then "return 1;" will return the top item on the stack, which is MyValue.

MyItem can be a userdata (and in fact that's probably preferable to malloc because you gain automatic memory management and don't have to remember to call free() later), but the catch you must keep it (the userdata object, not just the struct) reachable by Lua or else it will be quickly garbage collected. Since MyItem is directly connected to MyValue, the best way to do this is most likely to add a user value to MyValue (using the API functions for that), with that user value being MyItem. Then MyItem will not be garbage collected as long as MyValue is alive.
Reply | Threaded
Open this post in threaded view
|

Re: Howto return kind userdata from C to Lua?

Sean Conner
In reply to this post by Ranier Vilela-2
It was thus said that the Great Ranier Vilela once stated:
> Hi hackers,

  Hello.

> Is it possible to create and use a userdata value, this way?

  What follows is your code, but I've added some commentary.

typedef struct MyItem {
   int size;
} MyItem;

typedef struct MyValue {
  const char * name;
  MyItem * item;
} MyValue;

static int lua_myvalue_new(lua_State *L) {
  MyValue * value;
  const char * name;
  size_t l;
  int n;

  name = luaL_checklstring(L, 1, &l);
  // The following check is not needed---luaL_checklstring() will do the
  // check for you and raise an error is the item isn't a string.  Also,
  // given that you do nothing with the size of the string, you can use
  // luaL_checkstring() instead.
 
  // if (name == NULL) {
  //   return 0;
  // }

  n = luaL_checkinteger(L, 2);
  luaL_argcheck(L, n >= 1, 2, "invalid size");

  value = (MyValue *) lua_newuserdata(L, sizeof(MyValue));
  // Again, no need for the following check as lua_newuserdata() will raise
  // an error if it can't allocate memory.
 
  // if (value == NULL) {
  //   return 0;
  // }

  // The issue with the following line is assigning the name to the
  // structure.  Strings are subject to garbage collection.  But here, Lua
  // has no idea that there's a reference to a string, so the string itself
  // may go away.  One way to handle this is to strore a reference to the
  // string in the registry; another way is to allocate memory and create a
  // copy, but then the __gc metamethod of MYVALUE_REGISTRY will have to
  // free this memory.

  value->name = name;

  luaL_getmetatable(L, MYVALUE_REGISTRY);
  lua_setmetatable(L, -2);

  value->item = (MyItem *) lua_newuserdata(L, sizeof(MyItem));
  // Again, the following check isn't needed.
 
  // if (value->item == NULL) {
  //     return 0;
  // }
 
  // Like with the name, this creates a value that Lua can garbage collect,
  // but Lua doesn't know you have a reference to it.  You may need to store
  // a reference to it in the registry.

  value->item->size = n;
  luaL_getmetatable(L, UIMYITEM_REGISTRY);
  lua_setmetatable(L, -2);

  // You can solve this several ways---one is to call lua_pop(L,1) to pop
  // MyItem off the stack, or to use lua_pushvalue(L,-2) to copy MyValue to
  // the top of the stack.  Another way is to generate the uservalues in
  // reverse order, thus avoiding the lua_pop() or lua_pushvalue() calls.
 
  return 1; /* Must return value (MyValue) */
}

> Howto return to Lua MyValue struct (userdata)?

  Mentioned in the comment above.
 
> MyItem needs to be allocated by malloc rather than lua_newuserdata?

  That is one way to do it.  But then the __gc metamethod of
MYVALUE_REGISTRY will have to explicitely free it.  Also doing it this way
prevents Lua from garbage collecting it if there's no reference to it.

  Now my question to you is---*how* are you going to use MyValue in Lua?  Is
it something that is opaque and not modified through Lua?  If so, this is
not a bad approach.  If you are going to modify it through Lua, then you may
want to use a different approach and use Lua tables until you can't.  I use
this method at work, and while I can't show the code, I can tell how it's
implemented (simplified):

  I have the following structure:
 
  struct data
  {
   char     *key;
   char     *picid;
   uint32_t  pictypes; // bitflags for image types, GIF, PNG, JPEG, etc.
   char     *info[];   // an array of strings
   size_t    numinfo;  // number of info entries
  };

  In the system, I have some C functions that look like:
 
  struct data const *record_get(char const *key);
  int                record_put(char const *key,struct data const *data);
 
  At the boundary between Lua and the rest of the system (in C/C++), I
convert this to/from a Lua table, since it can be manipulated and it's
easier (and less code) to write if I do so.  The record_get() function for
Lua looks something like:

        static int lua_record_get(lua_State *L)
        {
          struct data const *info = record_get(luaL_checkstring(L,1));
         
          if (info == NULL)
            return 0;
          lua_createtable(L,0,4);
          lua_pushvalue(L,1);
          lua_setfield(L,-2,"key");
          lua_pushlstring(L,info->picid);
          lua_setfield(L,-2,"picid");
         
          // turn the pictypes field into an array for easy processing in Lua
         
          lua_createtable(L,0,0);
          // blah blah
          lua_setfield(L,-2,"pictypes");
         
          lua_createtable(L,0,0);
          // blah blah
          lua_setfield(L,-2,"info");

          return 1;
        }

  And the reverse:
 
        static int lua_record_set(lua_State *L)
        {
          struct data info;
          int         rc;
         
          info.key = luaL_checkstring(L,1);
          luaL_checktype(L,2,LUA_TTABLE);
          lua_getfield(L,2,"picid");
          info.picid = luaL_checkstring(L,-1);
          lua_getfield(L,2,"pictypes");
         
          // code to loop through Lua table to get pictypes
         
          lua_getfield(L,2,"info");
         
          // code to loop through Lua table to get info
         
          rc = record_put(info.key,&info);
          lua_pushboolean(L,rc == 0);
          lua_pushinteger(L,rc);
          return 2;
        }

  I don't have to bother with making sure I have references to userdatas, or
write multiple __index and __newindex metamethods and what not.  The data is
converted to native Lua types upon inport, and is coverted from native Lua
types upon export, and for what we use it for, it's fast enough to not worry
about the speed of conversion.

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Howto return kind userdata from C to Lua?

Ranier Vilela-2
In reply to this post by Jonathan Goble
Em sáb., 7 de nov. de 2020 às 15:54, Jonathan Goble <[hidden email]> escreveu:
On Sat, Nov 7, 2020 at 1:30 PM Ranier Vilela <[hidden email]> wrote:
[...]
return 1; /* Must return value (MyValue) */

Howto return to Lua MyValue struct (userdata)?
MyItem needs to be allocated by malloc rather than lua_newuserdata?

Without testing, it should be (almost) as simple as making sure that at the line above, the MyValue userdata is on the top of the Lua stack. There are API functions to move things around on the stack. Then "return 1;" will return the top item on the stack, which is MyValue.

MyItem can be a userdata (and in fact that's probably preferable to malloc because you gain automatic memory management and don't have to remember to call free() later), but the catch you must keep it (the userdata object, not just the struct) reachable by Lua or else it will be quickly garbage collected. Since MyItem is directly connected to MyValue, the best way to do this is most likely to add a user value to MyValue (using the API functions for that), with that user value being MyItem. Then MyItem will not be garbage collected as long as MyValue is alive.
Hi  Jonatha, thanks for the hints.

best regards,
Ranier Vilela
Reply | Threaded
Open this post in threaded view
|

Re: Howto return kind userdata from C to Lua?

Ranier Vilela-2
In reply to this post by Sean Conner
Em sáb., 7 de nov. de 2020 às 22:07, Sean Conner <[hidden email]> escreveu:
  What follows is your code, but I've added some commentary.

typedef struct MyItem {
   int size;
} MyItem;

typedef struct MyValue {
  const char * name;
  MyItem * item;
} MyValue;

static int lua_myvalue_new(lua_State *L) {
  MyValue * value;
  const char * name;
  size_t l;
  int n;

  name = luaL_checklstring(L, 1, &l);
  // The following check is not needed---luaL_checklstring() will do the
  // check for you and raise an error is the item isn't a string.  Also,
  // given that you do nothing with the size of the string, you can use
  // luaL_checkstring() instead.
Good, I have used checklstring for future use of var l.

  // if (name == NULL) {
  //   return 0;
  // }

  n = luaL_checkinteger(L, 2);
  luaL_argcheck(L, n >= 1, 2, "invalid size");

  value = (MyValue *) lua_newuserdata(L, sizeof(MyValue));
  // Again, no need for the following check as lua_newuserdata() will raise
  // an error if it can't allocate memory.
Nice. I will remove, old habits.


  // if (value == NULL) {
  //   return 0;
  // }

  // The issue with the following line is assigning the name to the
  // structure.  Strings are subject to garbage collection.  But here, Lua
  // has no idea that there's a reference to a string, so the string itself
  // may go away.  One way to handle this is to strore a reference to the
  // string in the registry; another way is to allocate memory and create a
  // copy, but then the __gc metamethod of MYVALUE_REGISTRY will have to
  // free this memory.
The string name here, it is not the main problem.
 

  value->name = name;

  luaL_getmetatable(L, MYVALUE_REGISTRY);
  lua_setmetatable(L, -2);

  value->item = (MyItem *) lua_newuserdata(L, sizeof(MyItem));
  // Again, the following check isn't needed.
Ok, thanks.

  // if (value->item == NULL) {
  //     return 0;
  // }

  // Like with the name, this creates a value that Lua can garbage collect,
  // but Lua doesn't know you have a reference to it.  You may need to store
  // a reference to it in the registry.
Yes, I am doing this.

  value->item->size = n;
  luaL_getmetatable(L, UIMYITEM_REGISTRY);
  lua_setmetatable(L, -2);

  // You can solve this several ways---one is to call lua_pop(L,1) to pop
  // MyItem off the stack, or to use lua_pushvalue(L,-2) to copy MyValue to
  // the top of the stack.  Another way is to generate the uservalues in
  // reverse order, thus avoiding the lua_pop() or lua_pushvalue() calls.
Many thanks, I will try.
In reverse order, I think it is not possible, MyItem is part of MyValue.
But I think of using it (MyItem) independently of MyValue, when this is possible.


  return 1; /* Must return value (MyValue) */
}

> Howto return to Lua MyValue struct (userdata)?

  Mentioned in the comment above.

> MyItem needs to be allocated by malloc rather than lua_newuserdata?

  That is one way to do it.  But then the __gc metamethod of
MYVALUE_REGISTRY will have to explicitely free it.  Also doing it this way
prevents Lua from garbage collecting it if there's no reference to it.
Yes, but I'm just recording it because I intend to use it independently.

  Now my question to you is---*how* are you going to use MyValue in Lua?
It would be a new type, as in the example in the book chapter 28 (User-Defined types in C).
 
  Is
it something that is opaque and not modified through Lua?
Only modified by C.

  If so, this is
not a bad approach.  If you are going to modify it through Lua, then you may
want to use a different approach and use Lua tables until you can't.  I use
this method at work, and while I can't show the code, I can tell how it's
implemented (simplified):

  I have the following structure:

        struct data
        {
          char     *key;
          char     *picid;
          uint32_t  pictypes; // bitflags for image types, GIF, PNG, JPEG, etc.
          char     *info[];   // an array of strings
          size_t    numinfo;  // number of info entries
        };

  In the system, I have some C functions that look like:

        struct data const *record_get(char const *key);
        int                record_put(char const *key,struct data const *data);

  At the boundary between Lua and the rest of the system (in C/C++), I
convert this to/from a Lua table, since it can be manipulated and it's
easier (and less code) to write if I do so.  The record_get() function for
Lua looks something like:

        static int lua_record_get(lua_State *L)
        {
          struct data const *info = record_get(luaL_checkstring(L,1));

          if (info == NULL)
            return 0;
          lua_createtable(L,0,4);
          lua_pushvalue(L,1);
          lua_setfield(L,-2,"key");
          lua_pushlstring(L,info->picid);
          lua_setfield(L,-2,"picid");

          // turn the pictypes field into an array for easy processing in Lua

          lua_createtable(L,0,0);
          // blah blah
          lua_setfield(L,-2,"pictypes");

          lua_createtable(L,0,0);
          // blah blah
          lua_setfield(L,-2,"info");

          return 1;
        }

  And the reverse:

        static int lua_record_set(lua_State *L)
        {
          struct data info;
          int         rc;

          info.key = luaL_checkstring(L,1);
          luaL_checktype(L,2,LUA_TTABLE);
          lua_getfield(L,2,"picid");
          info.picid = luaL_checkstring(L,-1);
          lua_getfield(L,2,"pictypes");

          // code to loop through Lua table to get pictypes

          lua_getfield(L,2,"info");

          // code to loop through Lua table to get info

          rc = record_put(info.key,&info);
          lua_pushboolean(L,rc == 0);
          lua_pushinteger(L,rc);
          return 2;
        }

  I don't have to bother with making sure I have references to userdatas, or
write multiple __index and __newindex metamethods and what not.  The data is
converted to native Lua types upon inport, and is coverted from native Lua
types upon export, and for what we use it for, it's fast enough to not worry
about the speed of conversion.
Nice, but I think offer to user, sugar syntax like a[],
that way I need to implement all types of accesses.

best regards,
Ranier Vilela
Reply | Threaded
Open this post in threaded view
|

Re: Howto return kind userdata from C to Lua?

Ranier Vilela-2
Em dom., 8 de nov. de 2020 às 01:02, Ranier Vilela <[hidden email]> escreveu:
Em sáb., 7 de nov. de 2020 às 22:07, Sean Conner <[hidden email]> escreveu:
  // You can solve this several ways---one is to call lua_pop(L,1) to pop
  // MyItem off the stack, or to use lua_pushvalue(L,-2) to copy MyValue to
  // the top of the stack.  Another way is to generate the uservalues in
  // reverse order, thus avoiding the lua_pop() or lua_pushvalue() calls.
Many thanks, I will try.
In reverse order, I think it is not possible, MyItem is part of MyValue.
But I think of using it (MyItem) independently of MyValue, when this is possible.

lua_pushvalue, works, but only if disable that:
//luaL_getmetatable(L, MYITEM_REGISTRY);
//lua_setmetatable(L, -2);
lua_pushvalue(L, -2);
 
How to discover what MyValue position is?
Why does the register metatable MYITEM_REGISTRY confuse MYVALUE_REGISTRY?

regards,
Ranier Vilela
Reply | Threaded
Open this post in threaded view
|

Re: Howto return kind userdata from C to Lua?

Sean Conner
It was thus said that the Great Ranier Vilela once stated:

> Em dom., 8 de nov. de 2020 às 01:02, Ranier Vilela <[hidden email]>
> escreveu:
>
> > Em sáb., 7 de nov. de 2020 às 22:07, Sean Conner <[hidden email]>
> > escreveu:
> >   // You can solve this several ways---one is to call lua_pop(L,1) to pop
> >
> >>   // MyItem off the stack, or to use lua_pushvalue(L,-2) to copy MyValue
> >> to
> >>   // the top of the stack.  Another way is to generate the uservalues in
> >>   // reverse order, thus avoiding the lua_pop() or lua_pushvalue() calls.
> >>
> > Many thanks, I will try.
> > In reverse order, I think it is not possible, MyItem is part of MyValue.
> > But I think of using it (MyItem) independently of MyValue, when this is
> > possible.
> >
>
> lua_pushvalue, works, but only if disable that:
> //luaL_getmetatable(L, MYITEM_REGISTRY);
> //lua_setmetatable(L, -2);
> lua_pushvalue(L, -2);
>
> How to discover what MyValue position is?
> Why does the register metatable MYITEM_REGISTRY confuse MYVALUE_REGISTRY?

  When I need to figure out the Lua stack in C, I use this function:

static int cstack(lua_State *L)
{
  int max = lua_gettop(L);
 
  fprintf(stderr,"Stack dump\n");
 
  for (int i = 1 ; i <= max ; i++)
  {
    fprintf(
            stderr,
            "%d %d - %s %s\n",
            i,
            i - max - 1,
            luaL_typename(L,i),
            luaL_tolstring(L,i,NULL)
    );
    lua_pop(L,1);
  }
 
  fprintf(stderr,"Stack done\n");
  return max;
}

  Call it where ever you feel the need to see the contents of the Lua stack.
As for your second question, I don't have any ideas for that.

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Howto return kind userdata from C to Lua?

Ranier Vilela-2
Em dom., 8 de nov. de 2020 às 20:15, Sean Conner <[hidden email]> escreveu:
It was thus said that the Great Ranier Vilela once stated:
> Em dom., 8 de nov. de 2020 às 01:02, Ranier Vilela <[hidden email]>
> escreveu:
>
> > Em sáb., 7 de nov. de 2020 às 22:07, Sean Conner <[hidden email]>
> > escreveu:
> >   // You can solve this several ways---one is to call lua_pop(L,1) to pop
> >
> >>   // MyItem off the stack, or to use lua_pushvalue(L,-2) to copy MyValue
> >> to
> >>   // the top of the stack.  Another way is to generate the uservalues in
> >>   // reverse order, thus avoiding the lua_pop() or lua_pushvalue() calls.
> >>
> > Many thanks, I will try.
> > In reverse order, I think it is not possible, MyItem is part of MyValue.
> > But I think of using it (MyItem) independently of MyValue, when this is
> > possible.
> >
>
> lua_pushvalue, works, but only if disable that:
> //luaL_getmetatable(L, MYITEM_REGISTRY);
> //lua_setmetatable(L, -2);
> lua_pushvalue(L, -2);
>
> How to discover what MyValue position is?
> Why does the register metatable MYITEM_REGISTRY confuse MYVALUE_REGISTRY?

  When I need to figure out the Lua stack in C, I use this function:

static int cstack(lua_State *L)
{
  int max = lua_gettop(L);

  fprintf(stderr,"Stack dump\n");

  for (int i = 1 ; i <= max ; i++)
  {
    fprintf(
            stderr,
            "%d %d - %s %s\n",
            i,
            i - max - 1,
            luaL_typename(L,i),
            luaL_tolstring(L,i,NULL)
    );
    lua_pop(L,1);
  }

  fprintf(stderr,"Stack done\n");
  return max;
}

  Call it where ever you feel the need to see the contents of the Lua stack.
Many thanks for your contribution (cstack function)
I think that was solved with your help.

luaL_getmetatable(L, MYVALUE_REGISTRY);
lua_setmetatable(L, -2);

luaL_getmetatable(L, MYITEM_REGISTRY);
lua_setmetatable(L, -1);

return 2;
 
best regards,
Ranier Vilela