Heap overflow in luaH_get

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Heap overflow in luaH_get

Yongheng Chen

Hi,

 

We found a heap overflow in lua. Here’s the details:

 

Version:

Lua 5.4.0, git hash c33b1728aeb7dfeec4013562660e07d32697aa6b

 

POC:

function test(do_yield) error 'fail!' end coro = coroutine.wrap(

function() function errfunc() function errfunc(x)

    pcall(function()

        u = setmetatable({},

        {

            __gc = function(a) B =

                coroutine

                .create(function() do assert(

                    setmetatable(

                    {},

                    {

                        __gc

                        =

                        function(  )

                            function crash(  )

                                t[pairs( a )]=0

                            end

                            for i = 1, 5 do crash( )end

                        end

                    }))

                    if k then end end end)

                    coroutine.resume(B) end

                }) require 'mod' end) 'fail' end coro =

                coroutine.wrap(function() print(xpcall(test, errfunc)) end)

            end print(xpcall(test, errfunc)) end) coro() coro() 

 

How to reproduce:

./lua poc.lua

 

Tested on Ubuntu16.

 

Stack dump:

=================================================================

==14014==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060000020d8 at pc 0x000000431a77 bp 0x7ffd9222c630 sp 0x7ffd9222c620

READ of size 8 at 0x6060000020d8 thread T0

    #0 0x431a76 in luaH_get (/home/yongheng/lua_asan/lua+0x431a76)

    #1 0x40a007 in lua_rawget (/home/yongheng/lua_asan/lua+0x40a007)

    #2 0x44af37 in luaL_getmetafield (/home/yongheng/lua_asan/lua+0x44af37)

    #3 0x45683f in luaB_pairs (/home/yongheng/lua_asan/lua+0x45683f)

    #4 0x414de1 in luaD_call (/home/yongheng/lua_asan/lua+0x414de1)

    #5 0x43d4cc in luaV_execute (/home/yongheng/lua_asan/lua+0x43d4cc)

    #6 0x43d4cc in luaV_execute (/home/yongheng/lua_asan/lua+0x43d4cc)

    #7 0x415194 in luaD_callnoyield (/home/yongheng/lua_asan/lua+0x415194)

    #8 0x4127d0 in luaD_rawrunprotected (/home/yongheng/lua_asan/lua+0x4127d0)

    #9 0x415d70 in luaD_pcall (/home/yongheng/lua_asan/lua+0x415d70)

    #10 0x41ac34 in GCTM (/home/yongheng/lua_asan/lua+0x41ac34)

    #11 0x41e812 in finishgencycle (/home/yongheng/lua_asan/lua+0x41e812)

    #12 0x41ff00 in luaC_step (/home/yongheng/lua_asan/lua+0x41ff00)

    #13 0x448dd6 in luaL_error (/home/yongheng/lua_asan/lua+0x448dd6)

    #14 0x468a84 in findloader (/home/yongheng/lua_asan/lua+0x468a84)

    #15 0x468b97 in ll_require (/home/yongheng/lua_asan/lua+0x468b97)

    #16 0x414de1 in luaD_call (/home/yongheng/lua_asan/lua+0x414de1)

    #17 0x43d4cc in luaV_execute (/home/yongheng/lua_asan/lua+0x43d4cc)

    #18 0x415194 in luaD_callnoyield (/home/yongheng/lua_asan/lua+0x415194)

    #19 0x4127d0 in luaD_rawrunprotected (/home/yongheng/lua_asan/lua+0x4127d0)

    #20 0x415d70 in luaD_pcall (/home/yongheng/lua_asan/lua+0x415d70)

    #21 0x40bd47 in lua_pcallk (/home/yongheng/lua_asan/lua+0x40bd47)

    #22 0x456d0f in luaB_pcall (/home/yongheng/lua_asan/lua+0x456d0f)

    #23 0x414de1 in luaD_call (/home/yongheng/lua_asan/lua+0x414de1)

    #24 0x43d4cc in luaV_execute (/home/yongheng/lua_asan/lua+0x43d4cc)

    #25 0x415194 in luaD_callnoyield (/home/yongheng/lua_asan/lua+0x415194)

    #26 0x4112ae in luaG_errormsg (/home/yongheng/lua_asan/lua+0x4112ae)

    #27 0x411491 in luaG_runerror (/home/yongheng/lua_asan/lua+0x411491)

    #28 0x411595 in luaG_typeerror (/home/yongheng/lua_asan/lua+0x411595)

    #29 0x4138bc in luaD_tryfuncTM (/home/yongheng/lua_asan/lua+0x4138bc)

#30 0x41480d in luaD_call (/home/yongheng/lua_asan/lua+0x41480d) 

 

Found by: Yongheng Chen and Rui Zhong

 

Best,

Yongheng

 

 

Sent from Mail for Windows 10

 

Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Sergey Zakharchenko
Hello Yongheng,

Yongheng Chen <[hidden email]>:
> We found a heap overflow in lua.

FWIW: Confirmed on 32-bit MIPS with moderately patched 5.4.0 (source
package md5sum dbf155764e5d433fc55ae80ea7060b60).

(gdb) bt
#0  0x7716194f in luaH_getshortstr () from .../usr/lib/liblua.so.5.4.0
#1  0x77161a69 in luaH_get () from .../usr/lib/liblua.so.5.4.0
Backtrace stopped: frame did not save the PC

Best regards,

--
DoubleF
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Andrew Gierth
In reply to this post by Yongheng Chen
>>>>> "Yongheng" == Yongheng Chen <[hidden email]> writes:

 Yongheng> READ of size 8 at 0x6060000020d8 thread T0
 Yongheng>     #0 0x431a76 in luaH_get (/home/yongheng/lua_asan/lua+0x431a76)

OK. I think I see some of what's going on with this one.

The generational GC code seems to be assuming that a G_OLD1 object must
be somewhere between g->survival and g->reallyold, or between g->finobj
and g->finobjrold.

But what happens in this case is that some table with an age of OLD1 has
a metatable with a __gc metamethod, and when GCTM is being called for
it, it gets moved back to the allgc list _at the front_, while remaining
G_OLD1 (this is in udata2finalize).

But since the table is not between g->survival and g->reallyold, and
nevertheless is only OLD1 (and its metatable is at this point only
SURVIVAL and white), it's not being processed by markold. Also, at some
point in correctgraylist, the table was changed from grey to black while
its metatable remained white. This results in the metatable being freed
before the object that references it, hence the later crash.

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

云风 Cloud Wu
Andrew Gierth <[hidden email]> 于2020年7月9日周四 下午12:58写道:

>
> >>>>> "Yongheng" == Yongheng Chen <[hidden email]> writes:
>
>  Yongheng> READ of size 8 at 0x6060000020d8 thread T0
>  Yongheng>     #0 0x431a76 in luaH_get (/home/yongheng/lua_asan/lua+0x431a76)
>
> OK. I think I see some of what's going on with this one.
>
> The generational GC code seems to be assuming that a G_OLD1 object must
> be somewhere between g->survival and g->reallyold, or between g->finobj
> and g->finobjrold.
>
> But what happens in this case is that some table with an age of OLD1 has
> a metatable with a __gc metamethod, and when GCTM is being called for
> it, it gets moved back to the allgc list _at the front_, while remaining
> G_OLD1 (this is in udata2finalize).

I'm curious about that if the object moved back to the front of allgc
list and remaining G_OLD1, why this assert in sweepgen() haven't been
triggered ?

lua_assert(!isold(curr) && isdead(g, curr))
(https://github.com/lua/lua/blob/master/lgc.c#L1014)

I think  `psurvival = sweepgen(L, g, &g->allgc, g->survival);` in
youngcollection() may check whether the object is remaining G_OLD1 .
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Andrew Gierth
>>>>> "云风" == 云风 Cloud Wu <[hidden email]> writes:

 >> But what happens in this case is that some table with an age of OLD1 has
 >> a metatable with a __gc metamethod, and when GCTM is being called for
 >> it, it gets moved back to the allgc list _at the front_, while remaining
 >> G_OLD1 (this is in udata2finalize).

 云风> I'm curious about that if the object moved back to the front of
 云风> allgc list and remaining G_OLD1, why this assert in sweepgen()
 云风> haven't been triggered ?

 云风> lua_assert(!isold(curr) && isdead(g, curr))
 云风> (https://github.com/lua/lua/blob/master/lgc.c#L1014)

That assert is not reached if the object (in this case the table, not
the metatable) is not white, and in this case it's black. The metatable,
which is in fact freed in that code, is white and SURVIVAL (not OLD*).

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

云风 Cloud Wu
Andrew Gierth <[hidden email]> 于2020年7月9日周四 下午3:27写道:
> That assert is not reached if the object (in this case the table, not
> the metatable) is not white, and in this case it's black. The metatable,
> which is in fact freed in that code, is white and SURVIVAL (not OLD*).
>

I see.

1. A G_OLD1 table run finalizer in the end of youngcollection() , and
moved back to allgc .
2. Some object contains this table has been resurrected by a finalizer.
3. In the next youngcollection cycle, the table with G_OLD1 could be
marked to black by the object contains it.
4. The table set to G_OLD by `setage(curr, nextage[getage(curr)]);` in
sweepgen() , and then it is in reallyold list .
5. The table is regular old now, so it remains black until next major
collection, and the black flag will prevent marking its metatable and
its contents.

So , adding this line in udata2finalize() may fix this bug ?

static GCObject *udata2finalize (global_State *g) {
  ...
  resetbit(o->marked, FINALIZEDBIT);  /* object is "normal" again */
  setage(o, G_NEW);  /* Treat it as a new object */

--
http://blog.codingnow.com
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Andrew Gierth
>>>>> "云风" == 云风 Cloud Wu <[hidden email]> writes:

 >> That assert is not reached if the object (in this case the table,
 >> not the metatable) is not white, and in this case it's black. The
 >> metatable, which is in fact freed in that code, is white and
 >> SURVIVAL (not OLD*).

 云风> I see.

 云风> 1. A G_OLD1 table run finalizer in the end of youngcollection() ,
 云风> and moved back to allgc .
 
 云风> 2. Some object contains this table has been resurrected by a
 云风> finalizer.

If you look at the original code, the table being finalized has a
reference captured in an upvalue from within the finalizer.

 云风> 3. In the next youngcollection cycle, the table with G_OLD1 could
 云风> be marked to black by the object contains it.

No, it's already marked black at this point; the cycle that changed it
to OLD1 marks it black in correctgraylists at the end of the cycle.

Had it not been marked black, then marking it via some reference such as
the upvalue that holds it would end up marking the metatable too.

 云风> 4. The table set to G_OLD by `setage(curr,
 云风> nextage[getage(curr)]);` in sweepgen() , and then it is in
 云风> reallyold list .

Currently it never gets this far (at least with ASAN on) because we
crash first.

 云风> So , adding this line in udata2finalize() may fix this bug ?

 云风> static GCObject *udata2finalize (global_State *g) {
 云风>   ...
 云风>   resetbit(o->marked, FINALIZEDBIT);  /* object is "normal" again */
 云风>   setage(o, G_NEW);  /* Treat it as a new object */

Well, there may be other invariants to consider that this might end up
violating.

I follow the "don't propose solutions until the problem is
satisfactorily understood" method of problem solving (this has
considerable experimental support in the field of experimental
psychology). I know it's tempting to post patches as soon as you see a
possible fix, but this is the wrong approach in the long run.

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Dibyendu Majumdar
In reply to this post by Andrew Gierth
On Thu, 9 Jul 2020 at 05:58, Andrew Gierth <[hidden email]> wrote:

>
> >>>>> "Yongheng" == Yongheng Chen <[hidden email]> writes:
>
>  Yongheng> READ of size 8 at 0x6060000020d8 thread T0
>  Yongheng>     #0 0x431a76 in luaH_get (/home/yongheng/lua_asan/lua+0x431a76)
>
> OK. I think I see some of what's going on with this one.
>
> The generational GC code seems to be assuming that a G_OLD1 object must
> be somewhere between g->survival and g->reallyold, or between g->finobj
> and g->finobjrold.
>
> But what happens in this case is that some table with an age of OLD1 has
> a metatable with a __gc metamethod, and when GCTM is being called for
> it, it gets moved back to the allgc list _at the front_, while remaining
> G_OLD1 (this is in udata2finalize).
>
> But since the table is not between g->survival and g->reallyold, and
> nevertheless is only OLD1 (and its metatable is at this point only
> SURVIVAL and white), it's not being processed by markold. Also, at some
> point in correctgraylist, the table was changed from grey to black while
> its metatable remained white. This results in the metatable being freed
> before the object that references it, hence the later crash.

I don't know if below is relevant at all but I thought it worth
mentioning it anyway.

While testing in Ravi I saw the ASAN error in a version of Ravi code
where I had missed protecting base after a call to LuaF_close() - in
OP_RETURN in my case.
But I can't generate the error in the latest Ravi version which has a
fix for this.
Now Lua 5.4 has a different way of doing things so this might not be
what is happening but the stack trace of the ASAN error looks the
same.

Anyway, I attach below the full trace:

=================================================================
==1768==ERROR: AddressSanitizer: heap-use-after-free on address
0x6080000119b8 at pc 0x7f6c6671228c bp 0x7fff36cad3a0 sp
0x7fff36cad390
READ of size 1 at 0x6080000119b8 thread T0
    #0 0x7f6c6671228b in lua_getmetatable /home/d/github/ravi/src/lapi.c:945
    #1 0x7f6c666e39dc in luaL_getmetafield /home/d/github/ravi/src/lauxlib.c:773
    #2 0x7f6c666e5e18 in pairsmeta /home/d/github/ravi/src/lbaselib.c:248
    #3 0x7f6c666e5f3a in luaB_pairs /home/d/github/ravi/src/lbaselib.c:275
    #4 0x7f6c6673f0ba in luaD_precall /home/d/github/ravi/src/ldo.c:444
    #5 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #6 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #7 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #8 0x7f6c66752afc in dothecall /home/d/github/ravi/src/lgc.c:951
    #9 0x7f6c6673b274 in luaD_rawrunprotected /home/d/github/ravi/src/ldo.c:148
    #10 0x7f6c6674321c in luaD_pcall /home/d/github/ravi/src/ldo.c:830
    #11 0x7f6c667536ad in GCTM /home/d/github/ravi/src/lgc.c:971
    #12 0x7f6c667539ab in callallpendingfinalizers
/home/d/github/ravi/src/lgc.c:1001
    #13 0x7f6c66755153 in finishgencycle /home/d/github/ravi/src/lgc.c:1248
    #14 0x7f6c66755626 in youngcollection /home/d/github/ravi/src/lgc.c:1284
    #15 0x7f6c6675631d in genstep /home/d/github/ravi/src/lgc.c:1452
    #16 0x7f6c66757e5a in luaC_step /home/d/github/ravi/src/lgc.c:1690
    #17 0x7f6c667209b8 in lua_concat /home/d/github/ravi/src/lapi.c:1520
    #18 0x7f6c666e11f1 in luaL_error /home/d/github/ravi/src/lauxlib.c:229
    #19 0x7f6c666fec6c in findloader /home/d/github/ravi/src/loadlib.c:581
    #20 0x7f6c666fee43 in ll_require /home/d/github/ravi/src/loadlib.c:606
    #21 0x7f6c6673f0ba in luaD_precall /home/d/github/ravi/src/ldo.c:444
    #22 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #23 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #24 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #25 0x7f6c6671cdeb in f_call /home/d/github/ravi/src/lapi.c:1284
    #26 0x7f6c6673b274 in luaD_rawrunprotected /home/d/github/ravi/src/ldo.c:148
    #27 0x7f6c6674321c in luaD_pcall /home/d/github/ravi/src/ldo.c:830
    #28 0x7f6c6671d463 in lua_pcallk /home/d/github/ravi/src/lapi.c:1310
    #29 0x7f6c666e6805 in luaB_pcall /home/d/github/ravi/src/lbaselib.c:460
    #30 0x7f6c6673f0ba in luaD_precall /home/d/github/ravi/src/ldo.c:444
    #31 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #32 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #33 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #34 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #35 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #36 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #37 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #38 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #39 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #40 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #41 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #42 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #43 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #44 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #45 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #46 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #47 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #48 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #49 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #50 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #51 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #52 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #53 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #54 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #55 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #56 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #57 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #58 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #59 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #60 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #61 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #62 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #63 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #64 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #65 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #66 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #67 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #68 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #69 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #70 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #71 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #72 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #73 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #74 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #75 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #76 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #77 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #78 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #79 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #80 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #81 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #82 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #83 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #84 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #85 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #86 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #87 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #88 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #89 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #90 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #91 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #92 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #93 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #94 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #95 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #96 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #97 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #98 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #99 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #100 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #101 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #102 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #103 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #104 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #105 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #106 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #107 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #108 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #109 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #110 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #111 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #112 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #113 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #114 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #115 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #116 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #117 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #118 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #119 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #120 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #121 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #122 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #123 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #124 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #125 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #126 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #127 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #128 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #129 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #130 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #131 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #132 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #133 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #134 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #135 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #136 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #137 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #138 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #139 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #140 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #141 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #142 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #143 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #144 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #145 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #146 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #147 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #148 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #149 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #150 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #151 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #152 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #153 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #154 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #155 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #156 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #157 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #158 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #159 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #160 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #161 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #162 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #163 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #164 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #165 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #166 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #167 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #168 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #169 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #170 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #171 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #172 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #173 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #174 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #175 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #176 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #177 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #178 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #179 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #180 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #181 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #182 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #183 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #184 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #185 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #186 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #187 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #188 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #189 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #190 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #191 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #192 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #193 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #194 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #195 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #196 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #197 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #198 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #199 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #200 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #201 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #202 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #203 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #204 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #205 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #206 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #207 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #208 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #209 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #210 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #211 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #212 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #213 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #214 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #215 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #216 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #217 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #218 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #219 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #220 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #221 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #222 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #223 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #224 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #225 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #226 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #227 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #228 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #229 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #230 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #231 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #232 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #233 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #234 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #235 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #236 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #237 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #238 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #239 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #240 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #241 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #242 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #243 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #244 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #245 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #246 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #247 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #248 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #249 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #250 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686

0x6080000119b8 is located 24 bytes inside of 96-byte region
[0x6080000119a0,0x608000011a00)
freed by thread T0 here:
    #0 0x7f6c669a27cf in __interceptor_free
(/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
    #1 0x7f6c667ea2cd in freeblock /home/d/github/ravi/src/ltests.c:121
    #2 0x7f6c667ea4df in debug_realloc /home/d/github/ravi/src/ltests.c:146
    #3 0x7f6c6675fb16 in luaM_realloc_ /home/d/github/ravi/src/lmem.c:86
    #4 0x7f6c6678972a in luaH_free /home/d/github/ravi/src/ltable.c:535
    #5 0x7f6c66751ff2 in freeobj /home/d/github/ravi/src/lgc.c:841
    #6 0x7f6c667545a0 in sweepgen /home/d/github/ravi/src/lgc.c:1135
    #7 0x7f6c6675535f in youngcollection /home/d/github/ravi/src/lgc.c:1269
    #8 0x7f6c6675631d in genstep /home/d/github/ravi/src/lgc.c:1452
    #9 0x7f6c66757e5a in luaC_step /home/d/github/ravi/src/lgc.c:1690
    #10 0x7f6c667209b8 in lua_concat /home/d/github/ravi/src/lapi.c:1520
    #11 0x7f6c666e11f1 in luaL_error /home/d/github/ravi/src/lauxlib.c:229
    #12 0x7f6c666fec6c in findloader /home/d/github/ravi/src/loadlib.c:581
    #13 0x7f6c666fee43 in ll_require /home/d/github/ravi/src/loadlib.c:606
    #14 0x7f6c6673f0ba in luaD_precall /home/d/github/ravi/src/ldo.c:444
    #15 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #16 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #17 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #18 0x7f6c6671cdeb in f_call /home/d/github/ravi/src/lapi.c:1284
    #19 0x7f6c6673b274 in luaD_rawrunprotected /home/d/github/ravi/src/ldo.c:148
    #20 0x7f6c6674321c in luaD_pcall /home/d/github/ravi/src/ldo.c:830
    #21 0x7f6c6671d463 in lua_pcallk /home/d/github/ravi/src/lapi.c:1310
    #22 0x7f6c666e6805 in luaB_pcall /home/d/github/ravi/src/lbaselib.c:460
    #23 0x7f6c6673f0ba in luaD_precall /home/d/github/ravi/src/ldo.c:444
    #24 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #25 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #26 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #27 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #28 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #29 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628

previously allocated by thread T0 here:
    #0 0x7f6c669a2bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
    #1 0x7f6c667ea63c in debug_realloc /home/d/github/ravi/src/ltests.c:162
    #2 0x7f6c6675fb16 in luaM_realloc_ /home/d/github/ravi/src/lmem.c:86
    #3 0x7f6c667499f5 in luaC_newobj /home/d/github/ravi/src/lgc.c:335
    #4 0x7f6c66788fcc in luaH_new /home/d/github/ravi/src/ltable.c:496
    #5 0x7f6c667b173a in luaV_execute /home/d/github/ravi/src/lvm.c:1407
    #6 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #7 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #8 0x7f6c6671cdeb in f_call /home/d/github/ravi/src/lapi.c:1284
    #9 0x7f6c6673b274 in luaD_rawrunprotected /home/d/github/ravi/src/ldo.c:148
    #10 0x7f6c6674321c in luaD_pcall /home/d/github/ravi/src/ldo.c:830
    #11 0x7f6c6671d463 in lua_pcallk /home/d/github/ravi/src/lapi.c:1310
    #12 0x7f6c666e6805 in luaB_pcall /home/d/github/ravi/src/lbaselib.c:460
    #13 0x7f6c6673f0ba in luaD_precall /home/d/github/ravi/src/ldo.c:444
    #14 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #15 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #16 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #17 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #18 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #19 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #20 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #21 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
    #22 0x7f6c667bde0e in luaV_execute /home/d/github/ravi/src/lvm.c:1697
    #23 0x7f6c6674098a in luaD_call /home/d/github/ravi/src/ldo.c:595
    #24 0x7f6c66740a64 in luaD_callnoyield /home/d/github/ravi/src/ldo.c:606
    #25 0x7f6c6673918f in luaG_errormsg /home/d/github/ravi/src/ldebug.c:686
    #26 0x7f6c66739605 in luaG_runerror /home/d/github/ravi/src/ldebug.c:702
    #27 0x7f6c667384d5 in luaG_typeerror /home/d/github/ravi/src/ldebug.c:628
    #28 0x7f6c6673d04a in tryfuncTM /home/d/github/ravi/src/ldo.c:326
    #29 0x7f6c66740787 in luaD_precall /home/d/github/ravi/src/ldo.c:562
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Roberto Ierusalimschy
In reply to this post by Andrew Gierth
> OK. I think I see some of what's going on with this one.
>
> The generational GC code seems to be assuming that a G_OLD1 object must
> be somewhere between g->survival and g->reallyold, or between g->finobj
> and g->finobjrold.
>
> But what happens in this case is that some table with an age of OLD1 has
> a metatable with a __gc metamethod, and when GCTM is being called for
> it, it gets moved back to the allgc list _at the front_, while remaining
> G_OLD1 (this is in udata2finalize).
>
> But since the table is not between g->survival and g->reallyold, and
> nevertheless is only OLD1 (and its metatable is at this point only
> SURVIVAL and white), it's not being processed by markold. Also, at some
> point in correctgraylist, the table was changed from grey to black while
> its metatable remained white. This results in the metatable being freed
> before the object that references it, hence the later crash.

Exactly! I guess the correction is moving its age back to OLD0, but
I have to check that. Similar problems should occurr with other ages.

Usually, an object's metatable is set right after its creation,
so the object is still new and everything works fine. In the
crashing code, the «setmetatable({}, ...)» looks like the metable
is being set right after the «{}», but the code to create the
metatable is long and slow, aging the table.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Roberto Ierusalimschy
In reply to this post by Andrew Gierth
>  云风> I'm curious about that if the object moved back to the front of
>  云风> allgc list and remaining G_OLD1, why this assert in sweepgen()
>  云风> haven't been triggered ?
>
>  云风> lua_assert(!isold(curr) && isdead(g, curr))
>  云风> (https://github.com/lua/lua/blob/master/lgc.c#L1014)
>
> That assert is not reached if the object (in this case the table, not
> the metatable) is not white, and in this case it's black. The metatable,
> which is in fact freed in that code, is white and SURVIVAL (not OLD*).

Moreover, are the assertions on in the tests? They are no on by
default in Lua.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Roberto Ierusalimschy
In reply to this post by Roberto Ierusalimschy
> > The generational GC code seems to be assuming that a G_OLD1 object must
> > be somewhere between g->survival and g->reallyold, or between g->finobj
> > and g->finobjrold.
> >
> > But what happens in this case is that some table with an age of OLD1 has
> > a metatable with a __gc metamethod, and when GCTM is being called for
> > it, it gets moved back to the allgc list _at the front_, while remaining
> > G_OLD1 (this is in udata2finalize).
> >
> > But since the table is not between g->survival and g->reallyold, and
> > nevertheless is only OLD1 (and its metatable is at this point only
> > SURVIVAL and white), it's not being processed by markold. Also, at some
> > point in correctgraylist, the table was changed from grey to black while
> > its metatable remained white. This results in the metatable being freed
> > before the object that references it, hence the later crash.
>
> Exactly! I guess the correction is moving its age back to OLD0, but
> I have to check that. Similar problems should occurr with other ages.
>
> Usually, an object's metatable is set right after its creation,
> so the object is still new and everything works fine. In the
> crashing code, the «setmetatable({}, ...)» looks like the metable
> is being set right after the «{}», but the code to create the
> metatable is long and slow, aging the table.

I think we have a dual problem in luaC_checkfinalizer, when we move
the object from the allgc list to the finobj list. Again, it must
go into the beginning of the list (to keep the order right), but
it may have the wrong age to go there.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

云风 Cloud Wu
In reply to this post by Roberto Ierusalimschy
Roberto Ierusalimschy <[hidden email]> 于2020年7月9日周四 下午10:02写道:
> so the object is still new and everything works fine. In the
> crashing code, the «setmetatable({}, ...)» looks like the metable
> is being set right after the «{}», but the code to create the
> metatable is long and slow, aging the table.

As Andrew Gierth said, the table is G_OLD1 and the metatable is white
and G_SURVIVAL. The age difference is strange.

How it happens ?
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Roberto Ierusalimschy
> Roberto Ierusalimschy <[hidden email]> 于2020年7月9日周四 下午10:02写道:
> > so the object is still new and everything works fine. In the
> > crashing code, the «setmetatable({}, ...)» looks like the metable
> > is being set right after the «{}», but the code to create the
> > metatable is long and slow, aging the table.
>
> As Andrew Gierth said, the table is G_OLD1 and the metatable is white
> and G_SURVIVAL. The age difference is strange.
>
> How it happens ?

Sorry, I mixed my comments. That comment was actually about the
problem with luaC_checkfinalizer, not with separatetobefnz. In
that stage, the table is still survival (not old1). Anyway,
a survival object should not be in the beninning of the finobj
list, before the finobjsur pointer.

But keep in mind that the age difference is potentially unbounded,
as an object can have its metatable set way after being created.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Andrew Gierth
>>>>> "Roberto" == Roberto Ierusalimschy <[hidden email]> writes:

 >> As Andrew Gierth said, the table is G_OLD1 and the metatable is
 >> white and G_SURVIVAL. The age difference is strange.
 >>
 >> How it happens ?

 [...]

 Roberto> But keep in mind that the age difference is potentially
 Roberto> unbounded, as an object can have its metatable set way after
 Roberto> being created.

Presumably not unbounded, because if the table is already old when the
metatable is set, then the barrier that is done when setting the
metatable will advance the metatable's age to OLD0. So the only allowed
differences after the setmetatable call would be SURVIVAL -> NEW
(becoming OLD1 -> SURVIVAL after one collection cycle), OLD1 -> OLD0,
OLD -> OLD0, OLD -> OLD1 etc.

In this case, the metatable (newly created and thus NEW) is set when the
table's age is SURVIVAL, so after one further collection pass the
metatable is SURVIVAL while the object table is OLD1. (This shows why it
is necessary for the collector to re-mark black OLD1 objects, because
they might still point to white young objects, which need to be marked
black for this cycle.) Had the marking happened in this case (which it
did not because of the object table's incorrect positioning on the allgc
list), then the metatable would have become OLD1 itself, with the object
table going to OLD.

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

云风 Cloud Wu
In reply to this post by Roberto Ierusalimschy
Roberto Ierusalimschy <[hidden email]> 于2020年7月9日周四 下午10:02写道:
>
> Exactly! I guess the correction is moving its age back to OLD0, but
> I have to check that. Similar problems should occurr with other ages.

I make a simpler test case to reveal this bug. It always crashs when I
define a especial allocater for lua .
It may be helpful.

setmetatable ({}, { __gc = function(a) -- 1st finalizer
   setmetatable(a, { __gc = function (b) -- 2nd finalizer
     print(getmetatable(b))
     print(getmetatable(b).x) -- should be 42
     collectgarbage "step"
     collectgarbage "step"
     print(getmetatable(b))
     print(getmetatable(b).x) -- may crash !! use 2nd metatable after free
   end,
   x = 42,
   })
   a = nil
   collectgarbage "step" -- trigger 2nd finalizer
end })

collectgarbage "step" -- trigger 1st finalizer

--
http://blog.codingnow.com
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Dibyendu Majumdar
On Fri, 10 Jul 2020 at 05:53, 云风 Cloud Wu <[hidden email]> wrote:

>
> Roberto Ierusalimschy <[hidden email]> 于2020年7月9日周四 下午10:02写道:
> >
> > Exactly! I guess the correction is moving its age back to OLD0, but
> > I have to check that. Similar problems should occurr with other ages.
>
> I make a simpler test case to reveal this bug. It always crashs when I
> define a especial allocater for lua .
> It may be helpful.
>
> setmetatable ({}, { __gc = function(a) -- 1st finalizer
>    setmetatable(a, { __gc = function (b) -- 2nd finalizer
>      print(getmetatable(b))
>      print(getmetatable(b).x) -- should be 42
>      collectgarbage "step"
>      collectgarbage "step"
>      print(getmetatable(b))
>      print(getmetatable(b).x) -- may crash !! use 2nd metatable after free
>    end,
>    x = 42,
>    })
>    a = nil
>    collectgarbage "step" -- trigger 2nd finalizer
> end })
>
> collectgarbage "step" -- trigger 1st finalizer
>

Nice.
Did you try building Lua with ltests/debug support?
In Ravi, when I run this - I see GC being invoked recursively ...
should that happen?

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) where
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7d2c859 in __GI_abort () at abort.c:79
#2  0x00007ffff7d2c729 in __assert_fail_base (fmt=0x7ffff7ec2588
"%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=0x7ffff7f9f149 "(((x_)->tt) & 0x0F) < (9+1)",
file=0x7ffff7f9e0c8 "/home/d/github/ravi/src/lapi.c",
    line=945, function=<optimized out>) at assert.c:92
#3  0x00007ffff7d3df36 in __GI___assert_fail (assertion=0x7ffff7f9f149
"(((x_)->tt) & 0x0F) < (9+1)",
    file=0x7ffff7f9e0c8 "/home/d/github/ravi/src/lapi.c", line=945,
    function=0x7ffff7fa08f0 <__PRETTY_FUNCTION__.5171>
"lua_getmetatable") at assert.c:101
#4  0x00007ffff7f2e76b in lua_getmetatable (L=0x55555555b2c0,
objindex=1) at /home/d/github/ravi/src/lapi.c:945
#5  0x00007ffff7f181f8 in luaB_getmetatable (L=0x55555555b2c0) at
/home/d/github/ravi/src/lbaselib.c:116
#6  0x00007ffff7f3ea3c in luaD_precall (L=0x55555555b2c0,
func=0x555555567e00, nresults=-1, op_call=1)
    at /home/d/github/ravi/src/ldo.c:453
#7  0x00007ffff7f706c0 in luaV_execute (L=0x55555555b2c0) at
/home/d/github/ravi/src/lvm.c:1747
#8  0x00007ffff7f3f3bb in luaD_call (L=0x55555555b2c0,
func=0x555555567dd0, nResults=0)
    at /home/d/github/ravi/src/ldo.c:604
#9  0x00007ffff7f3f41d in luaD_callnoyield (L=0x55555555b2c0,
func=0x555555567dd0, nResults=0)
    at /home/d/github/ravi/src/ldo.c:615
#10 0x00007ffff7f4612f in dothecall (L=0x55555555b2c0, ud=0x0) at
/home/d/github/ravi/src/lgc.c:954
#11 0x00007ffff7f3d373 in luaD_rawrunprotected (L=0x55555555b2c0,
f=0x7ffff7f460f3 <dothecall>, ud=0x0)
    at /home/d/github/ravi/src/ldo.c:157
#12 0x00007ffff7f401ef in luaD_pcall (L=0x55555555b2c0,
func=0x7ffff7f460f3 <dothecall>, u=0x0, old_top=192, ef=0)
    at /home/d/github/ravi/src/ldo.c:837
#13 0x00007ffff7f464a2 in GCTM (L=0x55555555b2c0) at
/home/d/github/ravi/src/lgc.c:974
#14 0x00007ffff7f46592 in callallpendingfinalizers (L=0x55555555b2c0)
at /home/d/github/ravi/src/lgc.c:1004
#15 0x00007ffff7f46f30 in finishgencycle (L=0x55555555b2c0,
g=0x55555555b390) at /home/d/github/ravi/src/lgc.c:1251
#16 0x00007ffff7f470fa in youngcollection (L=0x55555555b2c0, g=0x55555555b390)
    at /home/d/github/ravi/src/lgc.c:1287
#17 0x00007ffff7f4767d in genstep (L=0x55555555b2c0, g=0x55555555b390)
at /home/d/github/ravi/src/lgc.c:1455
#18 0x00007ffff7f4820b in luaC_step (L=0x55555555b2c0) at
/home/d/github/ravi/src/lgc.c:1693
--Type <RET> for more, q to quit, c to continue without paging--
#19 0x00007ffff7f32f98 in lua_gc (L=0x55555555b2c0, what=5) at
/home/d/github/ravi/src/lapi.c:1416
#20 0x00007ffff7f185de in luaB_collectgarbage (L=0x55555555b2c0) at
/home/d/github/ravi/src/lbaselib.c:196
#21 0x00007ffff7f3ea3c in luaD_precall (L=0x55555555b2c0,
func=0x555555567db0, nresults=0, op_call=1)
    at /home/d/github/ravi/src/ldo.c:453
#22 0x00007ffff7f706c0 in luaV_execute (L=0x55555555b2c0) at
/home/d/github/ravi/src/lvm.c:1747
#23 0x00007ffff7f3f3bb in luaD_call (L=0x55555555b2c0,
func=0x55555555d260, nResults=0)
    at /home/d/github/ravi/src/ldo.c:604
#24 0x00007ffff7f3f41d in luaD_callnoyield (L=0x55555555b2c0,
func=0x55555555d260, nResults=0)
    at /home/d/github/ravi/src/ldo.c:615
#25 0x00007ffff7f4612f in dothecall (L=0x55555555b2c0, ud=0x0) at
/home/d/github/ravi/src/lgc.c:954
#26 0x00007ffff7f3d373 in luaD_rawrunprotected (L=0x55555555b2c0,
f=0x7ffff7f460f3 <dothecall>, ud=0x0)
    at /home/d/github/ravi/src/ldo.c:157
#27 0x00007ffff7f401ef in luaD_pcall (L=0x55555555b2c0,
func=0x7ffff7f460f3 <dothecall>, u=0x0, old_top=128, ef=0)
    at /home/d/github/ravi/src/ldo.c:837
#28 0x00007ffff7f464a2 in GCTM (L=0x55555555b2c0) at
/home/d/github/ravi/src/lgc.c:974
#29 0x00007ffff7f46592 in callallpendingfinalizers (L=0x55555555b2c0)
at /home/d/github/ravi/src/lgc.c:1004
#30 0x00007ffff7f46f30 in finishgencycle (L=0x55555555b2c0,
g=0x55555555b390) at /home/d/github/ravi/src/lgc.c:1251
#31 0x00007ffff7f470fa in youngcollection (L=0x55555555b2c0, g=0x55555555b390)
    at /home/d/github/ravi/src/lgc.c:1287
#32 0x00007ffff7f4767d in genstep (L=0x55555555b2c0, g=0x55555555b390)
at /home/d/github/ravi/src/lgc.c:1455
#33 0x00007ffff7f4820b in luaC_step (L=0x55555555b2c0) at
/home/d/github/ravi/src/lgc.c:1693
#34 0x00007ffff7f32f98 in lua_gc (L=0x55555555b2c0, what=5) at
/home/d/github/ravi/src/lapi.c:1416
#35 0x00007ffff7f185de in luaB_collectgarbage (L=0x55555555b2c0) at
/home/d/github/ravi/src/lbaselib.c:196
#36 0x00007ffff7f3ea3c in luaD_precall (L=0x55555555b2c0,
func=0x55555555d240, nresults=0, op_call=1)
    at /home/d/github/ravi/src/ldo.c:453
#37 0x00007ffff7f706c0 in luaV_execute (L=0x55555555b2c0) at
/home/d/github/ravi/src/lvm.c:1747
#38 0x00007ffff7f3f3bb in luaD_call (L=0x55555555b2c0,
func=0x55555555d230, nResults=-1)
    at /home/d/github/ravi/src/ldo.c:604
#39 0x00007ffff7f3f41d in luaD_callnoyield (L=0x55555555b2c0,
func=0x55555555d230, nResults=-1)
--Type <RET> for more, q to quit, c to continue without paging--
    at /home/d/github/ravi/src/ldo.c:615
#40 0x00007ffff7f32455 in f_call (L=0x55555555b2c0, ud=0x7fffffffdac0)
at /home/d/github/ravi/src/lapi.c:1284
#41 0x00007ffff7f3d373 in luaD_rawrunprotected (L=0x55555555b2c0,
f=0x7ffff7f3241c <f_call>, ud=0x7fffffffdac0)
    at /home/d/github/ravi/src/ldo.c:157
#42 0x00007ffff7f401ef in luaD_pcall (L=0x55555555b2c0,
func=0x7ffff7f3241c <f_call>, u=0x7fffffffdac0, old_top=80,
    ef=64) at /home/d/github/ravi/src/ldo.c:837
#43 0x00007ffff7f326a2 in lua_pcallk (L=0x55555555b2c0, nargs=0,
nresults=-1, errfunc=3, ctx=0, k=0x0)
    at /home/d/github/ravi/src/lapi.c:1310
#44 0x0000555555556afb in docall (L=0x55555555b2c0, narg=0, nres=-1)
at /home/d/github/ravi/src/lua.c:209
#45 0x00005555555575f3 in handle_script (L=0x55555555b2c0, argv=0x7fffffffe010)
    at /home/d/github/ravi/src/lua.c:455
#46 0x0000555555557c3a in pmain (L=0x55555555b2c0) at
/home/d/github/ravi/src/lua.c:591
#47 0x00007ffff7f3ea3c in luaD_precall (L=0x55555555b2c0,
func=0x55555555b950, nresults=1, op_call=0)
    at /home/d/github/ravi/src/ldo.c:453
#48 0x00007ffff7f3f3ab in luaD_call (L=0x55555555b2c0,
func=0x55555555b950, nResults=1)
    at /home/d/github/ravi/src/ldo.c:603
#49 0x00007ffff7f3f41d in luaD_callnoyield (L=0x55555555b2c0,
func=0x55555555b950, nResults=1)
    at /home/d/github/ravi/src/ldo.c:615
#50 0x00007ffff7f32455 in f_call (L=0x55555555b2c0, ud=0x7fffffffdeb0)
at /home/d/github/ravi/src/lapi.c:1284
#51 0x00007ffff7f3d373 in luaD_rawrunprotected (L=0x55555555b2c0,
f=0x7ffff7f3241c <f_call>, ud=0x7fffffffdeb0)
    at /home/d/github/ravi/src/ldo.c:157
#52 0x00007ffff7f401ef in luaD_pcall (L=0x55555555b2c0,
func=0x7ffff7f3241c <f_call>, u=0x7fffffffdeb0, old_top=16,
    ef=0) at /home/d/github/ravi/src/ldo.c:837
#53 0x00007ffff7f326a2 in lua_pcallk (L=0x55555555b2c0, nargs=2,
nresults=1, errfunc=0, ctx=0, k=0x0)
    at /home/d/github/ravi/src/lapi.c:1310
#54 0x0000555555557dda in main (argc=2, argv=0x7fffffffe008) at
/home/d/github/ravi/src/lua.c:627
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

云风 Cloud Wu
Dibyendu Majumdar <[hidden email]> 于2020年7月10日周五 下午3:51写道:

> Did you try building Lua with ltests/debug support?

Haven't yet.

> In Ravi, when I run this - I see GC being invoked recursively ...
> should that happen?

 I did this on purpose. And I think GC is designed to be reentrant.
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Dibyendu Majumdar
In reply to this post by Dibyendu Majumdar
On Fri, 10 Jul 2020 at 08:51, Dibyendu Majumdar <[hidden email]> wrote:

>
> On Fri, 10 Jul 2020 at 05:53, 云风 Cloud Wu <[hidden email]> wrote:
> >
> > Roberto Ierusalimschy <[hidden email]> 于2020年7月9日周四 下午10:02写道:
> > >
> > > Exactly! I guess the correction is moving its age back to OLD0, but
> > > I have to check that. Similar problems should occurr with other ages.
> >
> > I make a simpler test case to reveal this bug. It always crashs when I
> > define a especial allocater for lua .
> > It may be helpful.
> >
> > setmetatable ({}, { __gc = function(a) -- 1st finalizer
> >    setmetatable(a, { __gc = function (b) -- 2nd finalizer
> >      print(getmetatable(b))
> >      print(getmetatable(b).x) -- should be 42
> >      collectgarbage "step"
> >      collectgarbage "step"
> >      print(getmetatable(b))
> >      print(getmetatable(b).x) -- may crash !! use 2nd metatable after free
> >    end,
> >    x = 42,
> >    })
> >    a = nil
> >    collectgarbage "step" -- trigger 2nd finalizer
> > end })
> >
> > collectgarbage "step" -- trigger 1st finalizer
> >
>
> Nice.
> Did you try building Lua with ltests/debug support?
> In Ravi, when I run this - I see GC being invoked recursively ...
> should that happen?
>

BTW this assertion does not fire for the other cases which makes me
believe this is a different issue - maybe caused by GC being invoked
while it is running GC?

> Program received signal SIGABRT, Aborted.
> __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> 50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
> (gdb) where
> #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
> #1  0x00007ffff7d2c859 in __GI_abort () at abort.c:79
> #2  0x00007ffff7d2c729 in __assert_fail_base (fmt=0x7ffff7ec2588
> "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
>     assertion=0x7ffff7f9f149 "(((x_)->tt) & 0x0F) < (9+1)",
> file=0x7ffff7f9e0c8 "/home/d/github/ravi/src/lapi.c",
>     line=945, function=<optimized out>) at assert.c:92
s=-1)
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Andrew Gierth
In reply to this post by Dibyendu Majumdar
>>>>> "Dibyendu" == Dibyendu Majumdar <[hidden email]> writes:

 Dibyendu> In Ravi, when I run this - I see GC being invoked recursively
 Dibyendu> ... should that happen?

The Lua GC disables itself when running finalizers, so it shouldn't
recurse normally, but an explicit call to collectgarbage() overrides
that.

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Heap overflow in luaH_get

Dibyendu Majumdar
On Fri, 10 Jul 2020 at 09:18, Andrew Gierth <[hidden email]> wrote:

>
> >>>>> "Dibyendu" == Dibyendu Majumdar <[hidden email]> writes:
>
>  Dibyendu> In Ravi, when I run this - I see GC being invoked recursively
>  Dibyendu> ... should that happen?
>
> The Lua GC disables itself when running finalizers, so it shouldn't
> recurse normally, but an explicit call to collectgarbage() overrides
> that.
>

Okay thanks.
12