Iterate over vararg

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Iterate over vararg

Dirk Laurie-2
The task is to write an iterator factory 'args' so that

   for k,v in args(...) do --[[whatever]] end

does the same as

   for k=1,select('#',...) do local v = select(k,...) --[[whatever]] end

It is quite easy by using a table, of course:

~~~~
args = function(...)
  local t = table.pack(...)
  local n = t.n
  local i = 0
  return function()
    i=i+1
    if i<=n then return i,t[i] end
  end
end
~~~~

It seems to be quite hard to do, even in the C API, without creating a
table. In fact, I have so far not succeeded.

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

云风 Cloud Wu
Dirk Laurie <[hidden email]> 于2018年12月20日周四 下午2:12写道:
>
> It seems to be quite hard to do, even in the C API, without creating a
> table. In fact, I have so far not succeeded.
>

Using a thread instead of a table in the C API may be a little cheaper.

--
http://blog.codingnow.com

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Gabriel Bertilson
In reply to this post by Dirk Laurie-2
I guess without a table, the arguments can be stored as upvalues. The
following makes an iterator generator that sets its arguments as
upvalues to a closure, with the total number of upvalues as the first
upvalue. (Is there a way to avoid that first upvalue?) Then the
closure receives an unused first argument and a second argument i, and
it returns the i-plus-one-th argument to the original function
(similar to the behavior of the function returned by ipairs). Not sure
if this is more or less efficient memory-wise than using a table.

#include <stdio.h>
#include <lua.h>
#include <lualib.h>
#include <lauxlib.h>

static int iter_varargs (lua_State * L) {
    lua_Integer i = luaL_checkinteger(L, 2);
    // Get total number of upvalues.
    lua_Integer n = luaL_checkinteger(L, lua_upvalueindex(1));
    if (i + 2 <= n) {
        lua_pushinteger(L, i + 1);
        // Return the i-th argument to gen_iter_varargs.
        lua_pushvalue(L, lua_upvalueindex(i + 2));
        return 2;
    } else {
        return 0;
    }
}

static int gen_iter_varargs (lua_State * L) {
    int n = lua_gettop(L);
    if (n + 1 > 255) { // maximum number of upvalues
        return luaL_error(L, "too many arguments");
    }
    // Insert total number of upvalues as first upvalue.
    lua_pushinteger(L, (lua_Integer) n + 1);
    lua_insert(L, 1);
    // Set arguments as remaining upvalues.
    lua_pushcclosure(L, iter_varargs, n + 1);
    lua_pushnil(L);
    lua_pushinteger(L, 0);
    return 3;
}

int main (int argc, char * * argv) {
    lua_State * L = luaL_newstate();
    luaL_openlibs(L);
    lua_register(L, "args", gen_iter_varargs);

    if (luaL_dostring(L, "for i, val in args(1, 2, 3) do print(val * val) end"))
        printf("failed to execute script\n");

    lua_close(L);

    return 0;
}

— Gabriel

On Thu, Dec 20, 2018 at 12:12 AM Dirk Laurie <[hidden email]> wrote:

>
> The task is to write an iterator factory 'args' so that
>
>    for k,v in args(...) do --[[whatever]] end
>
> does the same as
>
>    for k=1,select('#',...) do local v = select(k,...) --[[whatever]] end
>
> It is quite easy by using a table, of course:
>
> ~~~~
> args = function(...)
>   local t = table.pack(...)
>   local n = t.n
>   local i = 0
>   return function()
>     i=i+1
>     if i<=n then return i,t[i] end
>   end
> end
> ~~~~
>
> It seems to be quite hard to do, even in the C API, without creating a
> table. In fact, I have so far not succeeded.
>

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Dirk Laurie-2
Op Do. 20 Des. 2018 om 10:08 het Gabriel Bertilson
<[hidden email]> geskryf:
>
> I guess without a table, the arguments can be stored as upvalues. The
> following makes an iterator generator that sets its arguments as
> upvalues to a closure, with the total number of upvalues as the first
> upvalue. (Is there a way to avoid that first upvalue?) Then the
> closure receives an unused first argument and a second argument i, and
> it returns the i-plus-one-th argument to the original function
> (similar to the behavior of the function returned by ipairs). Not sure
> if this is more or less efficient memory-wise than using a table.

Thanks, I learnt something from your elegant code.

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Roberto Ierusalimschy
In reply to this post by Gabriel Bertilson
> [...] (Is there a way to avoid that first upvalue?) [...]

Yes:

  Any access to lua_upvalueindex(n), where n is greater than the number
  of upvalues of the current function (but not greater than 256, which
  is one plus the maximum number of upvalues in a closure), produces an
  acceptable but invalid index.

  https://www.lua.org/manual/5.3/manual.html#4.4

Plus:

  For functions that can be called with acceptable indices, any
  non-valid index is treated as if it contains a value of a virtual type
  LUA_TNONE, [...]

  https://www.lua.org/manual/5.3/manual.html#4.3

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Andrew Gierth
In reply to this post by Gabriel Bertilson
>>>>> "Gabriel" == Gabriel Bertilson <[hidden email]> writes:

 Gabriel> I guess without a table, the arguments can be stored as
 Gabriel> upvalues. The following makes an iterator generator that sets
 Gabriel> its arguments as upvalues to a closure, with the total number
 Gabriel> of upvalues as the first upvalue. (Is there a way to avoid
 Gabriel> that first upvalue?)

Yes. Quoth the docs (for 5.3 at least):

    The first upvalue associated with a function is at index
    lua_upvalueindex(1), and so on. Any access to lua_upvalueindex(n),
    where n is greater than the number of upvalues of the current
    function (but not greater than 256, which is one plus the maximum
    number of upvalues in a closure), produces an acceptable but invalid
    index.

This means that if there are n upvalues, then lua_upvalueindex(n+1) is
always an acceptable index, which means it's safe to call lua_type on
it, which will return LUA_TNONE since it's an invalid index. Whereas
upvalues 1..n will have types other than LUA_TNONE.

 Gabriel> Then the closure receives an unused first argument and a
 Gabriel> second argument i, and it returns the i-plus-one-th argument
 Gabriel> to the original function (similar to the behavior of the
 Gabriel> function returned by ipairs). Not sure if this is more or less
 Gabriel> efficient memory-wise than using a table.

I don't see any obvious reason why it would be more efficient - creating
the closure is allocating an object, after all, and it seems that
closures with upvalues probably would be more overhead than tables. But
I've not tried it.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Gabriel Bertilson
In reply to this post by Roberto Ierusalimschy
Thank you, that simplifies the code. I've also added a nil argument to
the script to make sure it works.

#include <stdio.h>
#include <lua.h>
#include <lualib.h>
#include <lauxlib.h>

// from lfunc.h
#define MAXUPVAL 255

static int iter_varargs (lua_State * L) {
    lua_Integer i = luaL_checkinteger(L, 2) + 1;
    if (!lua_isnone(L, lua_upvalueindex(i))) {
        lua_pushinteger(L, i);
        // Return the i-th argument to gen_iter_varargs.
        lua_pushvalue(L, lua_upvalueindex(i));
        return 2;
    }
    lua_pushnil(L);
    return 1;
}

static int gen_iter_varargs (lua_State * L) {
    int n = lua_gettop(L);
    if (n > MAXUPVAL) { // impossible?
        return luaL_error(L, "too many arguments");
    }
    // Set all arguments as upvalues to iter_varargs.
    lua_pushcclosure(L, iter_varargs, n);
    lua_pushnil(L);
    lua_pushinteger(L, 0);
    return 3;
}

int main (int argc, char * * argv) {
    lua_State * L = luaL_newstate();
    luaL_openlibs(L);
    lua_register(L, "args", gen_iter_varargs);

    if (luaL_dostring(L, "for i, val in args(1, 2, 3, nil, 4) do
print(val and val * val) end"))
        printf("failed to execute script\n");

    lua_close(L);

    return 0;
}

— Gabriel

On Thu, Dec 20, 2018 at 5:38 AM Roberto Ierusalimschy
<[hidden email]> wrote:

>
> > [...] (Is there a way to avoid that first upvalue?) [...]
>
> Yes:
>
>   Any access to lua_upvalueindex(n), where n is greater than the number
>   of upvalues of the current function (but not greater than 256, which
>   is one plus the maximum number of upvalues in a closure), produces an
>   acceptable but invalid index.
>
>   https://www.lua.org/manual/5.3/manual.html#4.4
>
> Plus:
>
>   For functions that can be called with acceptable indices, any
>   non-valid index is treated as if it contains a value of a virtual type
>   LUA_TNONE, [...]
>
>   https://www.lua.org/manual/5.3/manual.html#4.3
>
> -- Roberto
>

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Roberto Ierusalimschy
> Thank you, that simplifies the code. [...]

-     if (!lua_isnone(L, lua_upvalueindex(i))) {
+     if (0 < i <= 256 && !lua_isnone(L, lua_upvalueindex(i))) {


-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Philipp Janda
In reply to this post by Andrew Gierth
Am 20.12.18 um 12:55 schröbte Andrew Gierth:

>>>>>> "Gabriel" == Gabriel Bertilson <[hidden email]> writes:
>
>   Gabriel> Then the closure receives an unused first argument and a
>   Gabriel> second argument i, and it returns the i-plus-one-th argument
>   Gabriel> to the original function (similar to the behavior of the
>   Gabriel> function returned by ipairs). Not sure if this is more or less
>   Gabriel> efficient memory-wise than using a table.
>
> I don't see any obvious reason why it would be more efficient - creating
> the closure is allocating an object, after all, and it seems that
> closures with upvalues probably would be more overhead than tables. But
> I've not tried it.

The table version is 40 bytes more expensive on x86_64 (probably a bit
more because there are two allocations for the table version and just
one for the C closure).

Philipp


Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Stephen Irons-3
In reply to this post by Roberto Ierusalimschy


On Fri, Dec 21, 2018 at 6:12 AM, Roberto Ierusalimschy <[hidden email]> wrote:
Thank you, that simplifies the code. [...]
- if (!lua_isnone(L, lua_upvalueindex(i))) { + if (0 < i <= 256 && !lua_isnone(L, lua_upvalueindex(i))) {

I don't think the expression (0 < i <= 256) does what you expect in C. Do you mean ((i > 0) && (i <= 256)), or perhaps this was intended as psuedo-code?

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Gabriel Bertilson
I went with

if (0 < i && i <= 256 && !lua_isnone(L, lua_upvalueindex(i))) { ... }

— Gabriel

On Thu, Dec 20, 2018 at 5:17 PM Stephen Irons <[hidden email]> wrote:

>
>
>
> On Fri, Dec 21, 2018 at 6:12 AM, Roberto Ierusalimschy <[hidden email]> wrote:
>
> Thank you, that simplifies the code. [...]
>
> - if (!lua_isnone(L, lua_upvalueindex(i))) { + if (0 < i <= 256 && !lua_isnone(L, lua_upvalueindex(i))) {
>
>
> I don't think the expression (0 < i <= 256) does what you expect in C. Do you mean ((i > 0) && (i <= 256)), or perhaps this was intended as psuedo-code?
>
> -- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Jonathan Goble
On Thu, Dec 20, 2018 at 6:56 PM Gabriel Bertilson <[hidden email]> wrote:
I went with

if (0 < i && i <= 256 && !lua_isnone(L, lua_upvalueindex(i))) { ... }

But is this even necessary? i originates from the third return value from the generator function, which starts at 0 and only increments from there. Additionally, MAXUPVAL is 255, so itervarargs cannot be called with a second argument greater than 255, which means that i cannot be greater than 256 in the first place. In short, i is already guaranteed to be between 1 and 256 inclusive, and therefore guaranteed to be an acceptable argument to lua_upvalueindex, in the first place.

So unless I'm missing something here (I admit my Lua/C API skills are rusty), the if condition can be reduced back to just the original !lua_isnone call.
Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Soni "They/Them" L.


On 2018-12-20 10:19 p.m., Jonathan Goble wrote:

> On Thu, Dec 20, 2018 at 6:56 PM Gabriel Bertilson
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     I went with
>
>     if (0 < i && i <= 256 && !lua_isnone(L, lua_upvalueindex(i))) { ... }
>
>
> But is this even necessary? i originates from the third return value
> from the generator function, which starts at 0 and only increments
> from there. Additionally, MAXUPVAL is 255, so itervarargs cannot be
> called with a second argument greater than 255, which means that i
> cannot be greater than 256 in the first place. In short, i is already
> guaranteed to be between 1 and 256 inclusive, and therefore guaranteed
> to be an acceptable argument to lua_upvalueindex, in the first place.
>
> So unless I'm missing something here (I admit my Lua/C API skills are
> rusty), the if condition can be reduced back to just the original
> !lua_isnone call.

local a,b,c = generator_fn()
a(b,-1)
a(b,1024)

?

Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Jonathan Goble
On Thu, Dec 20, 2018 at 7:54 PM Soni "They/Them" L. <[hidden email]> wrote:
local a,b,c = generator_fn()
a(b,-1)
a(b,1024)

?

Ah yes. I forgot that one could obtain and call the iterator function directly with arbitrary values. I told you I was rusty (but apparently not in the area I thought I was). Maybe I should start using less Python and more Lua to keep the latter skills up.
Reply | Threaded
Open this post in threaded view
|

Re: Iterate over vararg

Roberto Ierusalimschy
In reply to this post by Stephen Irons-3
> >-     if (!lua_isnone(L, lua_upvalueindex(i))) {
> >+     if (0 < i <= 256 && !lua_isnone(L, lua_upvalueindex(i))) {
> >
> >
>
> I don't think the expression (0 < i <= 256) does what you expect in C. Do
> you mean ((i > 0) && (i <= 256)), or perhaps this was intended as
> psuedo-code?

No excuses. My bad :-)

-- Roberto