emergency garbage collector

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

emergency garbage collector

cynthia powers
Hi,

I am working with a legacy code base, based off of lua 5.0.2.

I want to only integrate the emergency garbage collector from lua5,2 into out codebase.

Anyone see any problems with this? or are they not compatible at all.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Sam Roberts
On Tue, Feb 14, 2012 at 6:48 PM, cynthia powers <[hidden email]> wrote:
> I am working with a legacy code base, based off of lua 5.0.2.
>
> I want to only integrate the emergency garbage collector from lua5,2 into
> out codebase.

You might find this one easier to backport from 5.1:

http://lua-users.org/wiki/EmergencyGarbageCollector

Lua doesn't have any public repo, so you'd have to diff 5.2 and 5.1
and identify only the differences related to emergency gc.

Cheers,
Sam

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

cynthia powers


On Wed, Feb 15, 2012 at 12:13 AM, Sam Roberts <[hidden email]> wrote:


You might find this one easier to backport from 5.1:

http://lua-users.org/wiki/EmergencyGarbageCollector


Thanks Sam. Isn't it default in 5.2? Was it for some specific design issues that you recommend picking it up from 5.1.4, or simply because it would be easier to diff the file?



Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Luiz Henrique de Figueiredo
> I am working with a legacy code base, based off of lua 5.0.2.

Time to move to Lua 5.2 then... Perhaps it won't be that difficult.
We can try to help.

> I want to only integrate the emergency garbage collector from lua5,2 into
> out codebase.

Probably very hard to do. The emergency garbage collector contains lots
of small details.

> so you'd have to diff 5.2 and 5.1 and identify only the differences
> related to emergency gc.

Very hard to identify. The diffs bewteen 5.1 and 5.2 are huge.
diff -u contains around 20000 lines, the size of the Lua code itself.

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

cynthia powers

Thanks. But I have to isolate the emergency  gc and integrate it. Is there any documentation about it somewhere, that gives me more insight on the files/code changes that this would involve?

Thanks!
On Wed, Feb 15, 2012 at 5:15 AM, Luiz Henrique de Figueiredo <[hidden email]> wrote:
> I am working with a legacy code base, based off of lua 5.0.2.

Time to move to Lua 5.2 then... Perhaps it won't be that difficult.
We can try to help.

> I want to only integrate the emergency garbage collector from lua5,2 into
> out codebase.

Probably very hard to do. The emergency garbage collector contains lots
of small details.

> so you'd have to diff 5.2 and 5.1 and identify only the differences
> related to emergency gc.

Very hard to identify. The diffs bewteen 5.1 and 5.2 are huge.
diff -u contains around 20000 lines, the size of the Lua code itself.


Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Sam Roberts
In reply to this post by cynthia powers
On Tue, Feb 14, 2012 at 10:03 PM, cynthia powers <[hidden email]> wrote:
> On Wed, Feb 15, 2012 at 12:13 AM, Sam Roberts <[hidden email]> wrote:
>> You might find this one easier to backport from 5.1:
>>
>> http://lua-users.org/wiki/EmergencyGarbageCollector
>
> Thanks Sam. Isn't it default in 5.2?
Sure, but its not likely to be a simple cut-n-paste from one code base
to the other.

> Was it for some specific design issues
> that you recommend picking it up from 5.1.4, or simply because it would be
> easier to diff the file?

You won't be able to figure out what parts of lua5.2 are related to
emergency garbage collection, not easily, anyhow.

Since the patch above is already specific to your needs, it might be
easier to understand and integrate. Have you tried it? You might get
really lucky, and find it almost applies cleanly.

Also, lua 5.2 changed a lot from 5.1, particularly in the core
language and gc (ephemeron tables). Its been a while, but I recall 5.0
and 5.1 being much more similar, and 5.1 can be compiled to be
backwards compatible for almost every change.

You didn't say why upgrading wasn't an option, though its the obvious
way to get this feature. Upgrading to 5.1 should be very easy. 5.2
might be harder, depends on your situation.

Cheers,
Sam

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Roberto Ierusalimschy
In reply to this post by cynthia powers
> Thanks. But I have to isolate the emergency  gc and integrate it. Is there
> any documentation about it somewhere, that gives me more insight on the
> files/code changes that this would involve?

No. The main problem is that, with the emergency gc, every single memory
allocation can trigger a complete gc. That means that every single block
of memory must be rooted somewhere before Lua can allocate the next
block (otherwise the first bock may be collected while Lua allocates the
second). That means the changes are somewhat spread all over places
where Lua creates objects, which are plenty.

As an example, in 5.1 there was this function:

  Table *luaH_new (lua_State *L, int narray, int nhash)

that created a table with certain sizes. But this function does not know
where to root the table head while creating the arrays. So, in 5.2,
you must call

  Table *luaH_new (lua_State *L)

that creates the header with empty arrays, anchor that header somewhere
(e.g., in the top of the stack), and then call luaH_resize to set the
proper sizes for the arrays. Therefore, most places that create tables
have to be fixed.

To make things worse, it is tricky to properly test the code, as a
bug will only appear when there is an emergency collector in a
specific allocation in the code.

-- Roberto


Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Miles Bader-2
In reply to this post by Sam Roberts
Sam Roberts <[hidden email]> writes:
> You didn't say why upgrading wasn't an option, though its the obvious
> way to get this feature. Upgrading to 5.1 should be very easy. 5.2
> might be harder, depends on your situation.

Though 5.2 has a (compile-time) 5.1-compatibility mode ...

-Miles

--
Bacchus, n. A convenient deity invented by the ancients as an excuse for
getting drunk.

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Luiz Henrique de Figueiredo
> Though 5.2 has a (compile-time) 5.1-compatibility mode ...

Note that this compatibility mode is strictly for 5.1 features,
not for the 5.0 deprecated features left on by default in 5.1.

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

cynthia powers
Thank you very much for all the help.
Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Rena
In reply to this post by Luiz Henrique de Figueiredo
On Wed, Feb 15, 2012 at 03:15, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
>> I am working with a legacy code base, based off of lua 5.0.2.
>
> Time to move to Lua 5.2 then... Perhaps it won't be that difficult.
> We can try to help.
>

Kind of amusing to me, when someone complains about compatibility with
5.2, they're told "you can stay with an older version if you need".
But when asking for help with an older version, they're told to
upgrade...

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Miles Bader-2
HyperHacker <[hidden email]> writes:
>>> I am working with a legacy code base, based off of lua 5.0.2.
>>
>> Time to move to Lua 5.2 then... Perhaps it won't be that difficult.
>> We can try to help.
>
> Kind of amusing to me, when someone complains about compatibility with
> 5.2, they're told "you can stay with an older version if you need".
> But when asking for help with an older version, they're told to
> upgrade...

I dunno, the "you can stay with the older version" line comes with a
fairly obvious implication:  "... unless, of course, you actually need
the features in the newer version."  [and that's what's being requested
here...]

-Miles

--
We are all lying in the gutter, but some of us are looking at the stars.
-Oscar Wilde

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Alex Queiroz
In reply to this post by Rena
Hallo,

On Thu, Feb 16, 2012 at 5:47 AM, HyperHacker <[hidden email]> wrote:
>
> Kind of amusing to me, when someone complains about compatibility with
> 5.2, they're told "you can stay with an older version if you need".
> But when asking for help with an older version, they're told to
> upgrade...
>

What's really amusing is that when someone needs *new* features they
should get it in the *old* version.

--
-alex
http://www.artisancoder.com/

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

Javier Guerra Giraldez
On Thu, Feb 16, 2012 at 3:46 AM, Alex Queiroz <[hidden email]> wrote:
> What's really amusing is that when someone needs *new* features they
> should get it in the *old* version.

well, to be fair; the original post wasn't "please get new features in
old version", she wanted to do it herself and asked some tips.

(is Cynthia a female name everywhere?  there's nothing as inconclusive
as trying to fathom metainformation from a name...)

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

martinwguy
In reply to this post by cynthia powers
[off list]

Hi Cynthia.  A couple of weeks ago I had to do something very similar:
In the product I'm working on someone had applied the 5.1 EGC patch to
a modified versin of lua-5.1, and I had to go through and make its
inclusion conditional on a compile-time flag instead of always being
applied, which involved examining every modification is makes
individually. That wasn't too hard, just tedious, taking two or three
hours plus some testing.

Someone said that the 5.0 VM is similar to the 5.1 one and that the
patch already almost applies cleanly. so it might be worth trying.

If I can be of help in doing this, let me know. My rates for open
source work are modest.

Good luck!

    M

Reply | Threaded
Open this post in threaded view
|

Re: emergency garbage collector

martinwguy
oops. Oh well.