question: lua deterministic gc

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

question: lua deterministic gc

teckno.query
lua-l,

I have been looking at how to manage/minimize memory usage in a C/Lua process/program running on Linux. I have observed some non-deterministic behavior with Lua’s collectgarbage() in “freeing” up memory for other processes to be able to use.

Details may be found here:

Specifically, is there a way to ensure all unused memory is freed from within Lua?

Thanks in advance.

Reply | Threaded
Open this post in threaded view
|

Re: question: lua deterministic gc

Roberto Ierusalimschy
> lua-l,
>
> I have been looking at how to manage/minimize memory usage in a C/Lua process/program running on Linux. I have observed some non-deterministic behavior with Lua’s collectgarbage() in “freeing” up memory for other processes to be able to use.
>
> Details may be found here:
> https://stackoverflow.com/questions/65375638/in-lua-is-there-a-limitation-to-the-number-of-objects-tables-that-can-be-determ
>
> Specifically, is there a way to ensure all unused memory is freed from within Lua?

Calling 'collectgarbage' twice frees all memory not in use by Lua.
"Frees" here means to call 'free'; that does not ensure the memory is
returned to the OS and therefore is available to other processes.
That is all that can be done in ISO C.

Returning that free memory to the OS is a task to the memory allocator.
It probably depends on the memory fragmentation, specifically whether
the free memory forms blocks large enough to be returned. And
fragmentation depends on the pattern of memory allocations and, again,
on how the allocator behaves. All of this seems to be outside the
control of Lua.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: question: lua deterministic gc

Dibyendu Majumdar
On Wed, 23 Dec 2020 at 14:23, Roberto Ierusalimschy
<[hidden email]> wrote:

> Calling 'collectgarbage' twice frees all memory not in use by Lua.

Why twice? Also is this true of both incremental / generational modes,
as well as of Lua 5.3?

Regards
Reply | Threaded
Open this post in threaded view
|

Re: question: lua deterministic gc

Andrew Gierth
>>>>> "Dibyendu" == Dibyendu Majumdar <[hidden email]> writes:

 >> Calling 'collectgarbage' twice frees all memory not in use by Lua.

 Dibyendu> Why twice?

As I understand it, after the first call, unreferenced objects with
finalizers have had the finalizer called, but the object itself is not
freed until the next GC pass (since it had to be revived in order to
run the finalizer, and it's not known if the finalizer stored any
reference to it until we do another full mark/sweep cycle).

 Dibyendu>  Also is this true of both incremental / generational modes,
 Dibyendu> as well as of Lua 5.3?

Yes.

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: question: lua deterministic gc

Thijs Schreijer
In reply to this post by Dibyendu Majumdar


> On 26 Dec 2020, at 23:19, Dibyendu Majumdar <[hidden email]> wrote:
>
> On Wed, 23 Dec 2020 at 14:23, Roberto Ierusalimschy
> <[hidden email]> wrote:
>
>> Calling 'collectgarbage' twice frees all memory not in use by Lua.
>
> Why twice? Also is this true of both incremental / generational modes,
> as well as of Lua 5.3?
>
> Regards

First one finalises (calling GC metamethods), 2nd call actually cleans them out

Not entirely sure, but most likely applies to all versions.
Reply | Threaded
Open this post in threaded view
|

Re: question: lua deterministic gc

Gé Weijers
On Mon, Dec 28, 2020 at 8:24 AM Thijs Schreijer <[hidden email]> wrote:
> First one finalises (calling GC metamethods), 2nd call actually cleans them out

That's assuming you don't write a __gc metamethod that does this:

metatable = {
    __gc = function(object)
        someGlobal = object
    end
}

In this case the in object will not be collected until something
modifies "someGlobal" or the program terminates.

The finalizer will only run once for each object, unless you use
"setmetatable" again, which creates an object that is unreachable but
can never be collected, it's a zombie object of sorts:

local mt; mt = {
    __gc = function(ob)
        print("collecting")
        setmetatable(ob, mt)
    end,
}

local object = setmetatable({}, mt)

object = nil
while true do
    -- generate lots of trash to keep the garbage collector busy
    local trash = ("xxxxx"):rep(1000000)
end

There are plenty of ways to shoot yourself in the foot with __gc.



--