Interesting stack overflow in Lua 5.3

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Interesting stack overflow in Lua 5.3

Dirk Laurie-2
The following function, when invoked, causes a C stack overflow
in Lua 5.3.

local subsystem_mt = {__index =
   function(spec,auction)
      for key,bid in ipairs(spec) do
         print("Checking '"..auction.."' against '"..key.."'")
      end
   end}

It works under 5.1 and 5.2. It also works if "ipairs" is replaced
by "pairs", which is what the application code does now.

Having scrutinized the C source of "ipairs", I know why this
happens: ipairs just goes on until an item is not found, and
that triggers a call to the metamethod, bang.

After that exercise, I can now do what I could not do at first
reading: deduce from the deceptively simple specification
of 'ipairs' that a metamethod for __index should not invoke
'ipairs' on the same table.

Should the manual contain an explicit warning for this gotcha?
I usually say "no" to such suggestions, but having been caught
out this time ...

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Roberto Ierusalimschy
> The following function, when invoked, causes a C stack overflow
> in Lua 5.3.
>
> local subsystem_mt = {__index =
>    function(spec,auction)
>       for key,bid in ipairs(spec) do
>          print("Checking '"..auction.."' against '"..key.."'")
>       end
>    end}

I may be missing something, but "The following function" does not seem
to be a function. It looks like a table... (Therefore, I do not know
how to invoke it to cause something.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Soni "They/Them" L.


On 03/09/15 02:47 PM, Roberto Ierusalimschy wrote:

>> The following function, when invoked, causes a C stack overflow
>> in Lua 5.3.
>>
>> local subsystem_mt = {__index =
>>     function(spec,auction)
>>        for key,bid in ipairs(spec) do
>>           print("Checking '"..auction.."' against '"..key.."'")
>>        end
>>     end}
> I may be missing something, but "The following function" does not seem
> to be a function. It looks like a table... (Therefore, I do not know
> how to invoke it to cause something.)
>
> -- Roberto
>
Well there is a function, it's inside the table...

But I don't see how invoking it would cause something, either.

subsystem_mt.__index()
subsystem_mt:__index()
subsystem_mt.__index(table)
...

--
Disclaimer: these emails are public and can be accessed from <TODO: get a non-DHCP IP and put it here>. If you do not agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Parke
>>> The following function, when invoked, causes a C stack overflow
>>> in Lua 5.3.
>>>
>>> local subsystem_mt = {__index =
>>>     function(spec,auction)
>>>        for key,bid in ipairs(spec) do
>>>           print("Checking '"..auction.."' against '"..key.."'")
>>>        end
>>>     end}

On Thu, Sep 3, 2015 at 11:01 AM, Soni L. <[hidden email]> wrote:
> But I don't see how invoking it would cause something, either.

Eventually, ipairs will get to the end of the spec.  At which point,
ipairs will cause a recursive call to __index, which calls ipairs
again, etc.

-Parke

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Jonathan Goble
In reply to this post by Soni "They/Them" L.
On Thu, Sep 3, 2015 at 2:01 PM, Soni L. <[hidden email]> wrote:

> On 03/09/15 02:47 PM, Roberto Ierusalimschy wrote:
>>>
>>> The following function, when invoked, causes a C stack overflow
>>> in Lua 5.3.
>>>
>>> local subsystem_mt = {__index =
>>>     function(spec,auction)
>>>        for key,bid in ipairs(spec) do
>>>           print("Checking '"..auction.."' against '"..key.."'")
>>>        end
>>>     end}
>>
>> I may be missing something, but "The following function" does not seem
>> to be a function. It looks like a table... (Therefore, I do not know
>> how to invoke it to cause something.)
>>
>> -- Roberto
>>
> Well there is a function, it's inside the table...
>
> But I don't see how invoking it would cause something, either.
>
> subsystem_mt.__index()
> subsystem_mt:__index()
> subsystem_mt.__index(table)

Presumably by "when invoked", he means "when indexing a field with a
nil value in a table that uses subsystem_mt as its metatable". At
least that's how I understood it.

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Paul K-2
In reply to this post by Soni "They/Them" L.
>> I may be missing something, but "The following function" does not seem
>> to be a function. It looks like a table... (Therefore, I do not know
>> how to invoke it to cause something.)

> But I don't see how invoking it would cause something, either.

Add: setmetatable({1,2,3}, subsystem_mt).foo()

Lua 5.2 prints:

Checking 'foo' against '1'
Checking 'foo' against '2'
Checking 'foo' against '3'
metaindex.lua:7: attempt to call field 'foo' (a nil value)

But Lua 5.3 prints:

Checking 'foo' against '1'
Checking 'foo' against '2'
Checking 'foo' against '3'
Checking '4' against '1'
Checking '4' against '2'
Checking '4' against '3'
...last three lines are repeated...
metaindex.lua:3: C stack overflow

Paul.

Reply | Threaded
Open this post in threaded view
|

RE: Interesting stack overflow in Lua 5.3

张睿
In reply to this post by Dirk Laurie-2
I think it's because ipairs now respects __index metamethod in 5.3.


-------- Original message --------
From: Paul K <[hidden email]>
Date: 9/3/2015 11:09 AM (GMT-08:00)
To: Lua mailing list <[hidden email]>
Subject: Re: Interesting stack overflow in Lua 5.3

>> I may be missing something, but "The following function" does not seem
>> to be a function. It looks like a table... (Therefore, I do not know
>> how to invoke it to cause something.)

> But I don't see how invoking it would cause something, either.

Add: setmetatable({1,2,3}, subsystem_mt).foo()

Lua 5.2 prints:

Checking 'foo' against '1'
Checking 'foo' against '2'
Checking 'foo' against '3'
metaindex.lua:7: attempt to call field 'foo' (a nil value)

But Lua 5.3 prints:

Checking 'foo' against '1'
Checking 'foo' against '2'
Checking 'foo' against '3'
Checking '4' against '1'
Checking '4' against '2'
Checking '4' against '3'
...last three lines are repeated...
metaindex.lua:3: C stack overflow

Paul.

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Sean Conner
In reply to this post by Roberto Ierusalimschy
It was thus said that the Great Roberto Ierusalimschy once stated:

> > The following function, when invoked, causes a C stack overflow
> > in Lua 5.3.
> >
> > local subsystem_mt = {__index =
> >    function(spec,auction)
> >       for key,bid in ipairs(spec) do
> >          print("Checking '"..auction.."' against '"..key.."'")
> >       end
> >    end}
>
> I may be missing something, but "The following function" does not seem
> to be a function. It looks like a table... (Therefore, I do not know
> how to invoke it to cause something.)

  All I did was take that code, then add

x = setmetatable({},subsystem_mt)
print(x.one)

Under Lua 5.1, I got:

nil

Under Lua 5.3, I got:

lua-53: y.lua:5: C stack overflow
stack traceback:
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        ...
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in function <y.lua:4>
        [C]: in for iterator 'for iterator'
        y.lua:5: in metamethod '__index'
        y.lua:11: in main chunk
        [C]: in ?

  -spc (seems pretty straightforward to me ... )

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Tim Hill

On Sep 3, 2015, at 11:12 AM, Sean Conner <[hidden email]> wrote:

The following function, when invoked, causes a C stack overflow
in Lua 5.3.

local subsystem_mt = {__index =
  function(spec,auction)
     for key,bid in ipairs(spec) do
        print("Checking '"..auction.."' against '"..key.."'")
     end
  end}

I may be missing something, but "The following function" does not seem
to be a function. It looks like a table... (Therefore, I do not know
how to invoke it to cause something.)

 All I did was take that code, then add

x = setmetatable({},subsystem_mt)
print(x.one)


Isn’t this expected behavior? Since ipairs() respects metamethods (since 5.3), looks like you have an infinite recursion (if you don’t have an item at key 1)?

—Tim

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Dirk Laurie-2
2015-09-04 4:12 GMT+02:00 Tim Hill <[hidden email]>:

>
> On Sep 3, 2015, at 11:12 AM, Sean Conner <[hidden email]> wrote:
>
> The following function, when invoked, causes a C stack overflow
> in Lua 5.3.
>
> local subsystem_mt = {__index =
>   function(spec,auction)
>      for key,bid in ipairs(spec) do
>         print("Checking '"..auction.."' against '"..key.."'")
>      end
>   end}
>
>
> I may be missing something, but "The following function" does not seem
> to be a function. It looks like a table... (Therefore, I do not know
> how to invoke it to cause something.)
>
>
>  All I did was take that code, then add
>
> x = setmetatable({},subsystem_mt)
> print(x.one)
>
>
> Isn’t this expected behavior? Since ipairs() respects metamethods (since
> 5.3), looks like you have an infinite recursion (if you don’t have an item
> at key 1)?

The unexpected part is that you _always_ have an infinite recursion.
Sheldon Cooper and Don Tillman would have seen this immediately,
of course. For others, a little hint in the manual, where __index is being
documented, would be useful.

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Coda Highland
On Fri, Sep 4, 2015 at 2:50 AM, Dirk Laurie <[hidden email]> wrote:

> 2015-09-04 4:12 GMT+02:00 Tim Hill <[hidden email]>:
>>
>> On Sep 3, 2015, at 11:12 AM, Sean Conner <[hidden email]> wrote:
>>
>> The following function, when invoked, causes a C stack overflow
>> in Lua 5.3.
>>
>> local subsystem_mt = {__index =
>>   function(spec,auction)
>>      for key,bid in ipairs(spec) do
>>         print("Checking '"..auction.."' against '"..key.."'")
>>      end
>>   end}
>>
>>
>> I may be missing something, but "The following function" does not seem
>> to be a function. It looks like a table... (Therefore, I do not know
>> how to invoke it to cause something.)
>>
>>
>>  All I did was take that code, then add
>>
>> x = setmetatable({},subsystem_mt)
>> print(x.one)
>>
>>
>> Isn’t this expected behavior? Since ipairs() respects metamethods (since
>> 5.3), looks like you have an infinite recursion (if you don’t have an item
>> at key 1)?
>
> The unexpected part is that you _always_ have an infinite recursion.
> Sheldon Cooper and Don Tillman would have seen this immediately,
> of course. For others, a little hint in the manual, where __index is being
> documented, would be useful.
>

As for a potential solution, consider a numeric for loop and rawget to
avoid retriggering the metamethod.

/s/ Adam

Reply | Threaded
Open this post in threaded view
|

Re: Interesting stack overflow in Lua 5.3

Dirk Laurie-2
2015-09-04 16:45 GMT+02:00 Coda Highland <[hidden email]>:

> As for a potential solution, consider a numeric for loop and rawget to
> avoid retriggering the metamethod.

Oh, I have no shortage of solutions. The point is merely that
something that works in Lua 5.1 and 5.2 fails in Lua 5.3 for
reasons that require reading between the lines of the manual.