Lua C API

classic Classic list List threaded Threaded
15 messages Options
Jim
Reply | Threaded
Open this post in threaded view
|

Lua C API

Jim
when looking at Lua's C API i wondered why many of the API functions
do not return a result where that would make sense and just return
"void" (i. e. nothing), for example:

lua_call()
lua_close()
lua_concat()
lua_copy()
lua_createtable()
...
lua_len()
...
lua_newtable()
...
lua_pop()
lua_push*()
...

here it might be useful to indicate whether the call succceeded by returning
a result (probably an int). this does not even break compatibility since that
new return value can be easily ignored and/or casted away via (void) where
it is not wanted/needed.

in the case of the lua_to*() functions i would suggest to also return a success
result (an int) and return the requested value via an output pointer parameter
provided by the caller, for example:

int lua_toboolean ( lua_State * L, int index, int * result ) ;

int lua_tointeger ( lua_State * L, int index, lua_Integer * result ) ;
^^^ indicates whether the call succeeded, i. e. if the value at the
given stack index
could get converted to an lua_Integer, the result is copied into the
output argument
pointer provided by the caller.

this way it is possible to tell if the lua_tointeger() call succeeded
and the given
(by index) stack position contained a zero or if the call failed.
that is clearly an advantage IMO.

further such functions could be added, eg

int lua_tounsigned ( lua_State * L, int index, lua_Unsigned * result ) ;

for lua_Unsigned or

int lua_convert2integer ( lua_State * L, int index, lua_Integer * result ) ;

that tries to convert the value at the given stack position to an
integer an so on.
they should be named differently to preserve compatibility and the original
API functions could be implemented in terms of the new ones.

Lua C functions:

int  cfunc ( lua_State * L ) ;

what do negative results mean here ?
do they indicate error situations ? are they used anywhere ?
why not

int cfunc ( lua_State * L, unsigned int * nres ) ;

instead ?

the int result indicates to the Lua interpreter whether the C function call
succeeded and in that case the second output pointer argument "nres"
contains the number of results on the stack (could be zero of course).

so certain return codes could indicate a specific outcome like so:

enum {
  LUA_OK,
  LUA_ERROR,
  LUA_POSIX_ERROR,
  ...
} ;

so as a simple example the result LUA_POSIX_ERROR could indicate to
the Lua interpreter that a syscall failed and that in this case if 0 <
nres holds
the value copied to the output parameter pointer is the corresponding errno
value (just a little example of what could be possible).

BTW:

in the case of

int luaL_argerror (lua_State *L, int arg, const char *extramsg) ;

it would be nice if this could be changed to work akin to

int luaL_error ( lua_State * L, const char * fmt, ... ) ;

(which it calls anyway). this would enable the caller to not only pass
one string
but use a fmt and supplied values to create more helpful argument
error messages.

Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Dirk Laurie-2
Op Vr. 17 Mei 2019 om 17:14 het Jim <[hidden email]> geskryf:
>
> when looking at Lua's C API i wondered why many of the API functions
> do not return a result where that would make sense and just return
> "void" (i. e. nothing), for example:

TL;DR

The Lua API has evolved incrementally over over twenty years.
Compatibility as far as possible with the previous version seems to be
a design goal. Changes similar to what you propose have occasionally
been made when the authors of the code needed the functionality,
usuallly with a change in the name.

-- Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

JeanHeyd Meneide
As a side note, incremental improvements have been made to the C API. For example, functions which retrieved things from tables used to be `void` before. Now, they return type information in an "int" for what they retrieve, saving a good bit of repeated performance loss from the operation (death by 10000 cuts, etc.).

My C API wrapper still doesn't take advantage of this because LuaJIT and Lua 5.1 are still so wildly popular, but maybe one day LuaJIT and friends will move onto Lua 5.3 proper and we'll have a real time to migrate to take advantage of these new APIs!

On Fri, May 17, 2019 at 12:24 PM Dirk Laurie <[hidden email]> wrote:
Op Vr. 17 Mei 2019 om 17:14 het Jim <[hidden email]> geskryf:
>
> when looking at Lua's C API i wondered why many of the API functions
> do not return a result where that would make sense and just return
> "void" (i. e. nothing), for example:

TL;DR

The Lua API has evolved incrementally over over twenty years.
Compatibility as far as possible with the previous version seems to be
a design goal. Changes similar to what you propose have occasionally
been made when the authors of the code needed the functionality,
usuallly with a change in the name.

-- Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

William Ahern
In reply to this post by Jim
On Fri, May 17, 2019 at 05:14:14PM +0200, Jim wrote:

> when looking at Lua's C API i wondered why many of the API functions
> do not return a result where that would make sense and just return
> "void" (i. e. nothing), for example:
>
> lua_call()
> lua_close()
> lua_concat()
> lua_copy()
> lua_createtable()
> ...
> lua_len()
> ...
> lua_newtable()
> ...
> lua_pop()
> lua_push*()
> ...
>
> here it might be useful to indicate whether the call succceeded by returning
> a result (probably an int). this does not even break compatibility since that
> new return value can be easily ignored and/or casted away via (void) where
> it is not wanted/needed.

Most of those throw errors and only return to the immediate caller on
success, mirroring their Lua counter parts.
 

> in the case of the lua_to*() functions i would suggest to also return a success
> result (an int) and return the requested value via an output pointer parameter
> provided by the caller, for example:
>
> int lua_toboolean ( lua_State * L, int index, int * result ) ;
>
> int lua_tointeger ( lua_State * L, int index, lua_Integer * result ) ;
> ^^^ indicates whether the call succeeded, i. e. if the value at the
> given stack index
> could get converted to an lua_Integer, the result is copied into the
> output argument
> pointer provided by the caller.

This depends on use cases. Whether it's more convenient to signal an error
through the return value or a separate flag depends on the situation.
Imagine you're writing a simple integer arithmetic library that handles
overflow. The obvious way to do this is similar to the GCC and clang
builtin:

  bool __builtin_add_overflow(type1 x, type2 y, type3 *sum)
  bool __builtin_mul_overflow(type1 x, type2 y, type3 *prod);

Those are great if you're just performing a single operation. But what you
need to accumulate the result of 3 add operations and a multiplication?

   int a, b, c, d, e;
   if (__builtin_add_overflow(1234, 4321, &c))
     return ERANGE;
   if (__builtin_add_overflow(9999, &c, &d))
     return ERANGE;
   if (__builtin_add_overflow(8888, &d, &e))
     return ERANGE;
   if (__builtin_mul_overflow(7777, &e, &result))
     return ERANGE;
   return 0;

All those conditionals obscure the math--that's not good code. We could fix
it slighty like:

   bool overflow = 0;
   overflow |= __builtin_add_overflow(1234, 4321, &c);
   overflow |= __builtin_add_overflow(9999, &c, &d);
   overflow |= __builtin_add_overflow(8888, &d, &e);
   overflow |= __builtin_mul_overflow(7777, &e, &result);
   return (overflow)? ERANGE : 0;

But even better might be something like:

   bool overflow = 0;
   c = __xbuiltin_add_overflow(1234, 4321, &overflow);
   d = __xbuiltin_add_overflow(9999, &c, &overflow);
   e = __xbuiltin_add_overflow(8888, &d, &overflow);
   *result = __xbuiltin_mul_overflow(7777, &e, &overflow);
   return (overflow)? ERANGE : 0;

If the function names were shorter we might return some evaluations directly
into the function calls of others.

The latter method is how the assembly code usually works, where the result
of an operation is placed into the destination register and a flag is set in
a separate, specialized register on overflow. It's also closer to how you
might organize the code in a higher-level language like Lua. And not
coincidentally, I think, it's also how lua_tointegerx works.

While the Lua C API has some unpleasant artifacts from incremental
improvements, it's actually much nicer than you seem to give it credit for.
And in my experience much nicer thanh the vast majority of APIs I've run
across.
 

> this way it is possible to tell if the lua_tointeger() call succeeded
> and the given
> (by index) stack position contained a zero or if the call failed.
> that is clearly an advantage IMO.
>
> further such functions could be added, eg
>
> int lua_tounsigned ( lua_State * L, int index, lua_Unsigned * result ) ;
>
> for lua_Unsigned or
>
> int lua_convert2integer ( lua_State * L, int index, lua_Integer * result ) ;
>
> that tries to convert the value at the given stack position to an
> integer an so on.
> they should be named differently to preserve compatibility and the original
> API functions could be implemented in terms of the new ones.

See above.
 
<snip>

> BTW:
>
> in the case of
>
> int luaL_argerror (lua_State *L, int arg, const char *extramsg) ;
>
> it would be nice if this could be changed to work akin to
>
> int luaL_error ( lua_State * L, const char * fmt, ... ) ;
>
> (which it calls anyway). this would enable the caller to not only pass
> one string
> but use a fmt and supplied values to create more helpful argument
> error messages.

I can't disagree about the inconvenience but this is the sort of change
where the benefit is slight but the backwards incompatibility very painful.
I normally either write a helper routine or use the pattern found in the Lua
code itself: luaL_argerror(L, arg, lua_pushfstring(L, fmt, x, y));

Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Jim
In reply to this post by Jim
On 5/17/19, William Ahern <[hidden email]> wrote:
> Most of those throw errors and only return to the immediate caller on
> success, mirroring their Lua counter parts.

in this cases the mere fact that the function returned to the caller means
it succeeded and hence no return value is required to signal the outcome
(but it would not hurt to return one either). i understand.

Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Jim
In reply to this post by William Ahern
On 5/17/19, William Ahern <[hidden email]> wrote:
> Most of those throw errors and only return to the immediate caller on
> success, mirroring their Lua counter parts.

in this cases the mere fact that the function returned to the caller means
it succeeded and hence no return value is required to signal the outcome
(but it would not hurt to return one either). i understand.

> This depends on use cases. Whether it's more convenient to signal an error
> through the return value or a separate flag depends on the situation.
> Imagine you're writing a simple integer arithmetic library that handles
> overflow. The obvious way to do this is similar to the GCC and clang
> builtin:
>
>   bool __builtin_add_overflow(type1 x, type2 y, type3 *sum)
>   bool __builtin_mul_overflow(type1 x, type2 y, type3 *prod);
>
> Those are great if you're just performing a single operation. But what you
> need to accumulate the result of 3 add operations and a multiplication?
>
>    int a, b, c, d, e;
>    if (__builtin_add_overflow(1234, 4321, &c))
>      return ERANGE;
>    if (__builtin_add_overflow(9999, &c, &d))
>      return ERANGE;
>    if (__builtin_add_overflow(8888, &d, &e))
>      return ERANGE;
>    if (__builtin_mul_overflow(7777, &e, &result))
>      return ERANGE;
>    return 0;
>
> All those conditionals obscure the math--that's not good code. We could fix
> it slighty like:
>
>    bool overflow = 0;
>    overflow |= __builtin_add_overflow(1234, 4321, &c);
>    overflow |= __builtin_add_overflow(9999, &c, &d);
>    overflow |= __builtin_add_overflow(8888, &d, &e);
>    overflow |= __builtin_mul_overflow(7777, &e, &result);
>    return (overflow)? ERANGE : 0;
>
> But even better might be something like:
>
>    bool overflow = 0;
>    c = __xbuiltin_add_overflow(1234, 4321, &overflow);
>    d = __xbuiltin_add_overflow(9999, &c, &overflow);
>    e = __xbuiltin_add_overflow(8888, &d, &overflow);
>    *result = __xbuiltin_mul_overflow(7777, &e, &overflow);
>    return (overflow)? ERANGE : 0;
>
> If the function names were shorter we might return some evaluations
> directly
> into the function calls of others.
>
> The latter method is how the assembly code usually works, where the result
> of an operation is placed into the destination register and a flag is set in
> a separate, specialized register on overflow. It's also closer to how you
> might organize the code in a higher-level language like Lua. And not
> coincidentally, I think, it's also how lua_tointegerx works.

one could retain functions like
lua_Integer lua_tointegerx ( lua_State * L, int index, int * isnum ) ;
even under their old name. if NULL != isnum it returns the call's result via
the output parameter pointer. why not have both forms ?
the older functions could then be implemented in terms of the new ones,
hence compatibility would be preserved.

> While the Lua C API has some unpleasant artifacts from incremental
> improvements, it's actually much nicer than you seem to give it credit for.
> And in my experience much nicer than the vast majority of APIs I've run
> across.

interesting point, it depends what you compare here. this is definitely true
for python, perl and maybe even tcl and ruby (though those's C APIs are
not that bad, binding code can be written by hand). but what about - say -
Ring(-lang.net), Squirrel(-lang.org), and SGScript(.org) ?
is that still true when comparing with newer languages ?

> I can't disagree about the inconvenience but this is the sort of change
> where the benefit is slight but the backwards incompatibility very painful.
> I normally either write a helper routine or use the pattern found in the
> Lua code itself:
> luaL_argerror(L, arg, lua_pushfstring(L, fmt, x, y));

yes, this works. but isn't the lua_pushfstring function called again
in the underlying
lua_error() call ?

Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Jim
In reply to this post by William Ahern
On 5/17/19, William Ahern <[hidden email]> wrote:
> While the Lua C API has some unpleasant artifacts from incremental
> improvements, it's actually much nicer than you seem to give it credit for.
> And in my experience much nicer thanh the vast majority of APIs I've run
> across.

here is an example from the SGScript(.org) documentation:
http://www.sgscript.org/pages/advdocs/sgscript.docs.htm#Argument-handling

used in
if ( sgs_LoadArgs ( "s|f", & str, & q ) ) return 0 ;

sgs_LoadArgs() does type-based argument parsing

  's' requires a value that can be converted to a string, returns to
passed char**
  'f' requires a value that can be converted to a real value, returns
to passed float*
  '|' makes all arguments after it optional (always initialize data
passed to those!)

there are many more options and features for this function

Squirrel provides similar functionality (look it up yourself in its docs).
this is just an example to show what other languages' C APIs provide.

BTW:

it is still not clear to me what a negative return value from a call to a
Lua C function like "int cfunc ( lua_State * L )" means/represents/is used for.

Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Jim
In reply to this post by William Ahern
On 5/17/19, William Ahern <[hidden email]> wrote:
> While the Lua C API has some unpleasant artifacts from incremental
> improvements, it's actually much nicer than you seem to give it credit for.
> And in my experience much nicer thanh the vast majority of APIs I've run
> across.

really ? compared to which opponents ? you might have a look at this
list of embedded scripting languages:

https://github.com/dbohdan/embedded-scripting-languages

a github search for tags/topics like (embeddable-)scripting-language,
embeddable, embedded etc also yields some interesting results ...

Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Sean Conner
In reply to this post by Jim
It was thus said that the Great Jim once stated:

> On 5/17/19, William Ahern <[hidden email]> wrote:
> > While the Lua C API has some unpleasant artifacts from incremental
> > improvements, it's actually much nicer than you seem to give it credit for.
> > And in my experience much nicer thanh the vast majority of APIs I've run
> > across.
>
> here is an example from the SGScript(.org) documentation:
> http://www.sgscript.org/pages/advdocs/sgscript.docs.htm#Argument-handling
>
> used in
> if ( sgs_LoadArgs ( "s|f", & str, & q ) ) return 0 ;

  Shouldn't this be:

        if (!sgs_LoadArgs("s|f",&str,&q)) return 0;

> sgs_LoadArgs() does type-based argument parsing
>
>   's' requires a value that can be converted to a string, returns to
> passed char**
>   'f' requires a value that can be converted to a real value, returns
> to passed float*
>   '|' makes all arguments after it optional (always initialize data
> passed to those!)
>
> there are many more options and features for this function

  I can only speak for myself, but the lack of this function hasn't bothered
me.  For me, it's largely a difference of:

static int xdisplay_reparent(lua_State *L)
{
  XReparentWindow(
          *(Display **)lua_touserdata(L,1),
          *(Window *)luaL_checkudata(L,2,TYPE_XWINDOW),
          *(Window *)luaL_checkudata(L,3,TYPE_XWINDOW),
          luaL_optinteger(L,4,0),
          luaL_optinteger(L,5,0)
  );
  return 0;
}

vs (assuming Lua had a similar funtion):

static int xdisplay_reparent(lua_State *L)
{
  Display *display;
  Window   w;
  Window   parent;
  int      x = 0;
  int      y = 0;
 
  if (lua_loadargs(L,"ooo|ii",&display,&w,&parent,&x,&y))
    XReparentWindow(display,w,parent,x,y);
  else
    luaL_error(L,"bad parameter to xdisplay_reparent()");
  return 0;
}

  It's not like I have tons of functions each with scores of parameters.  
 
> Squirrel provides similar functionality (look it up yourself in its docs).
> this is just an example to show what other languages' C APIs provide.
>
> BTW:
>
> it is still not clear to me what a negative return value from a call to a
> Lua C function like "int cfunc ( lua_State * L )" means/represents/is used for.

  The return value for a lua_CFunction is the number of returned
values---negative values are undefined behavior.  For example, when I call
the following function:

        int badret(lua_State *L)
        {
          return -lua_gettop(L);
        }

I get:

        > badret(1)
        Segmentation fault (core dumped)

Your results may vary.

  -spc (It's undefined!)

Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Jim
On 5/22/19, Sean Conner <[hidden email]> wrote:

> The return value for a lua_CFunction is the number of returned
> values---negative values are undefined behavior.  For example, when I call
> the following function:
>
> int badret(lua_State *L)
> {
>  return -lua_gettop(L);
> }
>
> I get:
>
> > badret(1)
> Segmentation fault (core dumped)
>
> Your results may vary.

ok, i see.
maybe specifying unsigned int as return/result type could fix this
(unsigned int cfunc ( lua_State * L )) easily ?

Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Tim Hill
In reply to this post by Jim


On May 22, 2019, at 11:55 AM, Jim <[hidden email]> wrote:

While the Lua C API has some unpleasant artifacts from incremental
improvements, it's actually much nicer than you seem to give it credit for.
And in my experience much nicer than the vast majority of APIs I've run
across.


+1 .. its one of the cleanest APIs I’ve come across over MANY years.

—Tim

Reply | Threaded
Open this post in threaded view
|

RE: Lua C API

Tim McCracken

 

On May 22, 2019, at 11:55 AM, Jim <[hidden email]> wrote:

 

While the Lua C API has some unpleasant artifacts from incremental
improvements, it's actually much nicer than you seem to give it credit for.
And in my experience much nicer than the vast majority of APIs I've run
across.

 

 

+1 from me as well.  As an extendable/embeddable script engine, to me it is (much, much) simpler and easier than TCL, Perl or Python.

Jim
Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Jim
On 5/23/19, Tim McCracken <[hidden email]> wrote:
> +1 from me as well.  As an extendable/embeddable script engine, to me it is
> (much, much) simpler and easier than TCL, Perl or Python.

ok, no i see to what APIs you compare.
all 3 use only reference counting since they are older implementations.
perl and python were not designed with embedding in mind, though
Tcl was (although back in the 80ies). how comparing with contemporary
embeddable scripting languages ? is the claim still true in that cases ?

the Tcl C API is not that bad btw, for simple binding functions dealing
with ref counting is unnecessary. with this API it is possible to access
all args to the C function at once (via the array containg all arguments).
using the old string based API comes in handy when all args are expected
to be strings as in bindings for execve(2) and similar. such a direct solution
is not possible with the Lua C API.

Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Dirk Laurie-2
Op Do. 23 Mei 2019 om 21:39 het Jim <[hidden email]> geskryf:

>
> On 5/23/19, Tim McCracken <[hidden email]> wrote:
> > +1 from me as well.  As an extendable/embeddable script engine, to me it is
> > (much, much) simpler and easier than TCL, Perl or Python.
>
> ok, no i see to what APIs you compare.
> all 3 use only reference counting since they are older implementations.
> perl and python were not designed with embedding in mind, though
> Tcl was (although back in the 80ies). how comparing with contemporary
> embeddable scripting languages ? is the claim still true in that cases ?
>
> the Tcl C API is not that bad btw, for simple binding functions dealing
> with ref counting is unnecessary. with this API it is possible to access
> all args to the C function at once (via the array containg all arguments).
> using the old string based API comes in handy when all args are expected
> to be strings as in bindings for execve(2) and similar. such a direct solution
> is not possible with the Lua C API.

You know, LuaJIT has a different API to Lua's. Moreover, that project
needs a new maintainer.

Reply | Threaded
Open this post in threaded view
|

Re: Lua C API

Jim-2
In reply to this post by Jim
22.05.2019, 20:55, "Jim" <[hidden email]>:
>>  I can't disagree about the inconvenience but this is the sort of change
>>  where the benefit is slight but the backwards incompatibility very painful.
>>  I normally either write a helper routine or use the pattern found in the
>>  Lua code itself:
>>  luaL_argerror(L, arg, lua_pushfstring(L, fmt, x, y));
>
> yes, this works. but isn't the lua_pushfstring function called again
> in the underlying lua_error() call ?

why not just add a new macro/function like - say -

int luaL_argerrorf ( lua_State * L, int arg, const char * fmt, ... )

(or luaL_f(unc)argerror) to Lua's C API ?