Userdata/Table Unification

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Userdata/Table Unification

Mike Pall-44
Hi,

Edgar Toernig wrote (in the thread 'Lua 5 and tags'):
> Jamie Webb wrote:
> > I'd like to add that adding an array part to userdatas would also
> > provide fast typing:
> 
> Hmm... that gets me back to the old idea of merging userdata and
> tables: give every table an additional void* field and remove the
> userdata type completely *g*

Maybe it's time to start a 'Userdata/Table Unification' thread ...

The same topic pops up every so often. Summing up all advantages and
disadvantages in one thread may be helpful. And it may be a first step
in convincing the authors of Lua to add this to the core.

Just a few thoughts to start the discussion:

- Adding a void* to the table structure incurs no additional memory
  overhead on most systems. I.e. the userdata pointer is free!
  The current table structure is 32 bytes long (on 32 bit systems) and
  anything between 29 and 36 bytes leads to the same effective memory
  allocation due to malloc overhead and alignment constraints. YMMV.

- A plain userdata object has 16 bytes (on 32 bit systems) plus the
  userdata itself plus any malloc and alignment overhead.
  The savings provided by a trivial userdata object over a unified
  userdata/table object are negligible. If you really need to save space
  there is still lightuserdata (I think this type is valuable and should
  not be affected by the unification).
  More complex libraries actually need less memory because they no longer
  need table wrappers, auxiliary tables or per-object metatables.

- I cannot see any noticeable speed difference between the current approach
  and the unified approach (method access, userdata pointer access, type
  checking ...).

- Current userdata objects are allocated by the Lua core and cannot be
  resized nor moved nor shared. Quite a few libraries end up storing
  a single void* in a userdata object. This is wasteful.
  Leaving the allocation management to the C library seems to be the better
  solution in the long term IMHO.

- Many libraries need to store additional Lua objects along with userdata
  values and end up abusing the metatable or weak tables. This complicates
  type checking and/or garbage collection.

- Tables do not have a __gc() metamethod. This prevents wrapping userdata
  objects into a table because __gc() metamethods often need to have
  a look at the full object context and not just at individual userdata
  objects. Unifying userdata and tables would provide such a __gc()
  metamethod for tables. This is advantageous even for pure Lua code.

- Current userdata objects cannot be subclassed easily. Unless you have
  the cooperation of the C library you need to wrap them up in a table and
  store any subclass fields there. But this approach is troublesome
  due to the lack of a __gc() metamethod for tables. See above.
  Subclassing gets trivial with a unified userdata/table object.

- Every unified table object has a void*. This may be useful for many more
  scopes than just plain library objects (e.g. module namespace tables).

- I have not taken a closer look at the issues that the unification rises
  for the Lua garbage collector. I may be wrong, but I think it gets simpler
  because the tricky handling for userdata objects can be dropped.

- Reducing diversity usually simplifies the implementation and this leads
  to fewer bugs. Some people believe this is the only measure that counts.

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Jamie Webb-3
On Mon, Jul 26, 2004 at 02:51:00PM +0200, Mike Pall wrote:
> Maybe it's time to start a 'Userdata/Table Unification' thread ...
> 
> The same topic pops up every so often. Summing up all advantages and
> disadvantages in one thread may be helpful. And it may be a first step
> in convincing the authors of Lua to add this to the core.
> 
> Just a few thoughts to start the discussion:
> 
> - Adding a void* to the table structure incurs no additional memory
>   overhead on most systems. I.e. the userdata pointer is free!
>   The current table structure is 32 bytes long (on 32 bit systems) and
>   anything between 29 and 36 bytes leads to the same effective memory
>   allocation due to malloc overhead and alignment constraints. YMMV.

Looks like this hole appears on 16-bit systems as well, but not on
most 64-bit systems (except Itanium) since they still only use 8-byte
alignment.

> - A plain userdata object has 16 bytes (on 32 bit systems) plus the
>   userdata itself plus any malloc and alignment overhead.
>   The savings provided by a trivial userdata object over a unified
>   userdata/table object are negligible. If you really need to save space
>   there is still lightuserdata (I think this type is valuable and should
>   not be affected by the unification).
>   More complex libraries actually need less memory because they no longer
>   need table wrappers, auxiliary tables or per-object metatables.

Makes sense.

> - Current userdata objects are allocated by the Lua core and cannot be
>   resized nor moved nor shared. Quite a few libraries end up storing
>   a single void* in a userdata object. This is wasteful.
>   Leaving the allocation management to the C library seems to be the better
>   solution in the long term IMHO.

The trouble with that is that it forces everyone to write __gc
metamethods, even if they only want the simple case. The
implementation at present caters quite well for both simple uses (e.g.
lmd5), and complex ones, in this respect.

> - Many libraries need to store additional Lua objects along with userdata
>   values and end up abusing the metatable or weak tables. This complicates
>   type checking and/or garbage collection.

Absolutely. I don't think unifying tables and userdatas alone provides
a solution though: that data should be private. If __index and
__newindex were changed to always be called, that would work since
they could completely disallow access to a table (or to certain
members), but that might get expensive.

> - Tables do not have a __gc() metamethod. This prevents wrapping userdata
>   objects into a table because __gc() metamethods often need to have
>   a look at the full object context and not just at individual userdata
>   objects. Unifying userdata and tables would provide such a __gc()
>   metamethod for tables. This is advantageous even for pure Lua code.

I agree that __gc metamethods for tables would be very useful, but
there are complications: the reason AIUI that tables do not have them
is because a destructor might create new references to an object.
Unless Lua code can somehow be prevented from setting __gc (it's not
sufficient to require that it be a C function), probably the only good
solution is to move to a two-stage destruction, where the object has
to be shown unreachable again after the destructor has been called.
This is how Java does it.

One other possibility might be to make the destruction hook not a
metamethod, but give it some specialised C interface. That would also
make it easier to create a convenience function that created and
allocated a userdata and handled its deallocation, given a change to
pointer-only userdatas.

> - Current userdata objects cannot be subclassed easily. Unless you have
>   the cooperation of the C library you need to wrap them up in a table and
>   store any subclass fields there. But this approach is troublesome
>   due to the lack of a __gc() metamethod for tables. See above.
>   Subclassing gets trivial with a unified userdata/table object.

Yes, provided there is no private data in the table that the
subclassing might clobber. 

> - Every unified table object has a void*. This may be useful for many more
>   scopes than just plain library objects (e.g. module namespace tables).

True.

> - I have not taken a closer look at the issues that the unification rises
>   for the Lua garbage collector. I may be wrong, but I think it gets simpler
>   because the tricky handling for userdata objects can be dropped.

Why? The reasons for that tricky handling don't seem to change.

> - Reducing diversity usually simplifies the implementation and this leads
>   to fewer bugs. Some people believe this is the only measure that counts.

...provided the complexity of what's left doesn't increase too much.

-- Jamie Webb

Reply | Threaded
Open this post in threaded view
|

RE: Userdata/Table Unification

Virgil Smith
In reply to this post by Mike Pall-44
'Userdata/Table Unification' definitely gets my vote as well.

My primary reasons for supporting this measure are...

1. It greatly simplifies handling per instance data for userdata.
2. It improves Lua's orthogonality (which means it makes Lua conceptually
simpler and more efficient, and these improvements would probably apply to
Lua's VM code as well).

My primary concern is...

1. Sandboxing.  The fact that Lua code cannot modify the metatable of
Userdata is an important security benefit.  Consider for instance the effect
of a script changing the gc metamethod of a userdata.  On the other hand,
Lua extends a method for Lua side code to mark a metatable as protected and
so this is duplicated effort.  Also, the most important aspect of script
security <i.e. sandboxing> is not protected by the "special status" of
userdata.  That aspect is the protection of variables that reference
userdata values so that malicious/buggy scripts can't "hijack" the variable.


-----------
Comments about Mike's statements...


- Tables do not have a __gc() metamethod.
I am in complete agreement that tables should have __gc metamethods, but
their lack was a deliberate decision on the part of the Lua authors.  When I
commented on this lack previously <reasonably early in the 5.0 lifecycle>
the only responses I received were that they felt __gc metamethods were not
needed by Lua tables.  However, I've seen this come up on the list several
times as "the need to use a userdata proxy" in order to obtain a __gc
metamethod.




- Current userdata objects are allocated by the Lua core and cannot be
  resized nor moved nor shared. Quite a few libraries end up storing
  a single void* in a userdata object. This is wasteful.
  Leaving the allocation management to the C library seems to be the better
  solution in the long term IMHO.

The Lua authors stated reasons for this handling is to remove the need for
__gc metamethods in "most cases".  However, I for one have never made a
userdata that didn't need a __gc metamethod.  In fact I would expect that
most userdatas that allocated no resources other than memory should simply
be tables so that their values are directly accessible to Lua code.  Anyone
have counter examples?

In the past I would have greatly appreciated a mechanism for reporting the
size of a userdata to Lua so that the gc triggered properly, but I expect
this will be irrelevant with the incremental gc.  BTW: Allocating userdata
through Lua is only a partial solution for this because of allocations done
internally by userdata's.




-----------
As always, my sincere thanks to the Lua authors and those who have helped
along the way.  This is a marvelous language.  In no way should my
suggestions for small tweaks be taken as disapproval of the work that has
been done in the past.  Thanks especially for offering your labors up to
world for use and abuse!!






-----Original Message-----
From: [hidden email]
[[hidden email] Behalf Of Mike Pall
Sent: Monday, July 26, 2004 6:51 AM
To: Lua list
Subject: Userdata/Table Unification


Hi,

Edgar Toernig wrote (in the thread 'Lua 5 and tags'):
> Jamie Webb wrote:
> > I'd like to add that adding an array part to userdatas would also
> > provide fast typing:
>
> Hmm... that gets me back to the old idea of merging userdata and
> tables: give every table an additional void* field and remove the
> userdata type completely *g*

Maybe it's time to start a 'Userdata/Table Unification' thread ...

The same topic pops up every so often. Summing up all advantages and
disadvantages in one thread may be helpful. And it may be a first step
in convincing the authors of Lua to add this to the core.

Just a few thoughts to start the discussion:

- Adding a void* to the table structure incurs no additional memory
  overhead on most systems. I.e. the userdata pointer is free!
  The current table structure is 32 bytes long (on 32 bit systems) and
  anything between 29 and 36 bytes leads to the same effective memory
  allocation due to malloc overhead and alignment constraints. YMMV.

- A plain userdata object has 16 bytes (on 32 bit systems) plus the
  userdata itself plus any malloc and alignment overhead.
  The savings provided by a trivial userdata object over a unified
  userdata/table object are negligible. If you really need to save space
  there is still lightuserdata (I think this type is valuable and should
  not be affected by the unification).
  More complex libraries actually need less memory because they no longer
  need table wrappers, auxiliary tables or per-object metatables.

- I cannot see any noticeable speed difference between the current approach
  and the unified approach (method access, userdata pointer access, type
  checking ...).

- Current userdata objects are allocated by the Lua core and cannot be
  resized nor moved nor shared. Quite a few libraries end up storing
  a single void* in a userdata object. This is wasteful.
  Leaving the allocation management to the C library seems to be the better
  solution in the long term IMHO.

- Many libraries need to store additional Lua objects along with userdata
  values and end up abusing the metatable or weak tables. This complicates
  type checking and/or garbage collection.

- Tables do not have a __gc() metamethod. This prevents wrapping userdata
  objects into a table because __gc() metamethods often need to have
  a look at the full object context and not just at individual userdata
  objects. Unifying userdata and tables would provide such a __gc()
  metamethod for tables. This is advantageous even for pure Lua code.

- Current userdata objects cannot be subclassed easily. Unless you have
  the cooperation of the C library you need to wrap them up in a table and
  store any subclass fields there. But this approach is troublesome
  due to the lack of a __gc() metamethod for tables. See above.
  Subclassing gets trivial with a unified userdata/table object.

- Every unified table object has a void*. This may be useful for many more
  scopes than just plain library objects (e.g. module namespace tables).

- I have not taken a closer look at the issues that the unification rises
  for the Lua garbage collector. I may be wrong, but I think it gets simpler
  because the tricky handling for userdata objects can be dropped.

- Reducing diversity usually simplifies the implementation and this leads
  to fewer bugs. Some people believe this is the only measure that counts.

Bye,
     Mike



Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Rici Lake-2
Alternative implementation, which is not quite so "unified":

Add a peertable field to every full userdata, and modify lua_raw{get, set, geti, seti} to transparently reference this table, if it exists.
(But not the Lua base library raw* functions.)

I have experimented with a simple patch which does this, and it seems quite adequate to the purpose. The actual implementation I use adds two pointers to a userdata: a Lua table, and a void*; the latter is the "C peer". The Cpeer field can be used to hold boxed pointers, which avoids increasing the size of userdatas which only hold boxed pointers.

I believe this addresses Virgil's (and my) sandboxing requirements and also solves many of the issues around associating Lua objects with userdata. In particular, it allows the use of the userdata's metatable for class identification because instance-specific Lua objects can be held in the peer table.

The use of lua_raw* limits API explosion (although the API does need to be expanded to allow setting and getting peer information) and also "feels" quite natural in practice.

Patch (for 5.0.2) available upon request.

Rici


Reply | Threaded
Open this post in threaded view
|

RE: Userdata/Table Unification

Virgil Smith
In reply to this post by Jamie Webb-3
> > - Many libraries need to store additional Lua objects along with
userdata
> >   values and end up abusing the metatable or weak tables. This
complicates
> >   type checking and/or garbage collection.
>
> Absolutely. I don't think unifying tables and userdatas alone provides
> a solution though: that data should be private. If __index and
> __newindex were changed to always be called, that would work since
> they could completely disallow access to a table (or to certain
> members), but that might get expensive.

Good point, but as this is a parallel issue for both userdata and tables
this should be a separate thread.

However, as I stated this a parallel issue between userdata and tables, and
as such is another example of how very similar userdata and tables are,
which is why they should be combined (improvement of orthogonality).



Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Mike Pall-44
In reply to this post by Virgil Smith
Hi,

Virgil Smith wrote:
> 1. Sandboxing.  The fact that Lua code cannot modify the metatable of
> Userdata is an important security benefit.  Consider for instance the effect
> of a script changing the gc metamethod of a userdata.

Ok, setmetatable() does not work on userdata objects. But you can change
the contents of the metatable itself:

u = any_userdata_constructor()
getmetatable(u).__gc = nil      -- Ouch!
u = nil
collectgarbage()

This will lead to resource depletion in the best case (e.g. lost file
handles) and to memory corruption in the worst case (there may be
dangling backreferences to userdata memory).

The current approach is *not* safe unless you deny the sandbox access to
getmetatable() or use a wrapper. But then you might as well protect
setmetatable(), too. I think the setmetatable() pseudo-safeguard provides
a false sense of security.

> On the other hand,
> Lua extends a method for Lua side code to mark a metatable as protected and
> so this is duplicated effort.  Also, the most important aspect of script
> security <i.e. sandboxing> is not protected by the "special status" of
> userdata.  That aspect is the protection of variables that reference
> userdata values so that malicious/buggy scripts can't "hijack" the variable.

As Jamie Webb pointed out we need to be able to completely override *all*
access methods (and maybe a special GC API).

Someone should do some benchmarks with a large application to see whether
the additional checks are really such a bad performance hit. Maybe they
can be merged with an existing flow control statement or other parts can be
optimized to compensate for it.

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Adam D. Moss
In reply to this post by Jamie Webb-3
Jamie Webb wrote:
I agree that __gc metamethods for tables would be very useful, but
there are complications: the reason AIUI that tables do not have them
is because a destructor might create new references to an object.
Unless Lua code can somehow be prevented from setting __gc (it's not
sufficient to require that it be a C function), probably the only good
solution is to move to a two-stage destruction, where the object has
to be shown unreachable again after the destructor has been called.
This is how Java does it.

__gc metamethods for tables are fairly easily fakeable -- I do it
by using a mapping table to associate weak-refs of the table to
a heavy userdata with the appropriate gc method attached to it.

I'm fuzzy on whether this indirectly allows resurrection of the
original table; to date I don't think I've had to worry about that.
IIRC I've asked on this list in the past whether it was safe for
a destructor to create new references, and was told that was fine.

__gc on a table is useful for the same reasons that one might
unify (heavy) userdata and tables, I think.  I use it so that I
can have the flexibility of a table attached to a C-side light
userdata and still have my C-side objects gc'd.

--Adam
--
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3

Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Eero Pajarre-3
In reply to this post by Mike Pall-44
Mike Pall wrote:

> Ok, setmetatable() does not work on userdata objects. But you can change
> the contents of the metatable itself:
>
> u = any_userdata_constructor()
> getmetatable(u).__gc = nil      -- Ouch!
> u = nil
> collectgarbage()
>
> This will lead to resource depletion in the best case (e.g. lost file
> handles) and to memory corruption in the worst case (there may be
> dangling backreferences to userdata memory).
>
> The current approach is *not* safe unless you deny the sandbox access to
> getmetatable() or use a wrapper. But then you might as well protect
> setmetatable(), too. I think the setmetatable() pseudo-safeguard provides
> a false sense of security.
>

But you can protect the real metatable from user by setting
the __metatable field in it. The protection mechanism
seems ok to me.


        Eero





Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Mike Pall-44
Hi,

Eero Pajarre wrote:
> But you can protect the real metatable from user by setting
> the __metatable field in it. The protection mechanism
> seems ok to me.

Oops. Forgot about that one. I guess I've been looking too much at the
C API lately. |-)

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Edgar Toernig
In reply to this post by Mike Pall-44
Mike Pall wrote:
>
> Edgar Toernig wrote (in the thread 'Lua 5 and tags'):
> > 
> > Hmm... that gets me back to the old idea of merging userdata and
> > tables: give every table an additional void* field and remove the
> > userdata type completely *g*
> 
> Maybe it's time to start a 'Userdata/Table Unification' thread ...

Umm... ehh...  I wasn't really serious.  Sorry.

Yes, some time ago I thought about it.  But the (roughly) doubled memory
consumption, the need to protect the gc-callback and problems with the
metatable==type assumption let me drop that idea.

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Asko Kauppi-3
In reply to this post by Mike Pall-44

As a _user_ of Lua (not into the internals) I'd see this as a welcome simplification.

I've already noticed a common trend in many of the wrappers I do (especially big ones, s.a. ParaGUI). The objects all have '._ud' member (the actual C++ object pointer) and the C code allows functions to be fed either raw userdata or a table containing that _ud field.

So, it seems, I'm _already_ implementing the kind of system you're proposing. To me, this usage scenario itself is a reason to vote for the combination of table & userdata.

-ak

ps. Also, that would reduce the confusion around light vs. normal userdata that now exists, at least for newcomers (and me :).


26.7.2004 kello 15:51, Mike Pall kirjoitti:

 Hi,

Edgar Toernig wrote (in the thread 'Lua 5 and tags'):
Jamie Webb wrote:
I'd like to add that adding an array part to userdatas would also
provide fast typing:

Hmm... that gets me back to the old idea of merging userdata and
tables: give every table an additional void* field and remove the
userdata type completely *g*

Maybe it's time to start a 'Userdata/Table Unification' thread ...

The same topic pops up every so often. Summing up all advantages and
disadvantages in one thread may be helpful. And it may be a first step
in convincing the authors of Lua to add this to the core.

Just a few thoughts to start the discussion:

- Adding a void* to the table structure incurs no additional memory
  overhead on most systems. I.e. the userdata pointer is free!
  The current table structure is 32 bytes long (on 32 bit systems) and
  anything between 29 and 36 bytes leads to the same effective memory
  allocation due to malloc overhead and alignment constraints. YMMV.

- A plain userdata object has 16 bytes (on 32 bit systems) plus the
  userdata itself plus any malloc and alignment overhead.
  The savings provided by a trivial userdata object over a unified
userdata/table object are negligible. If you really need to save space there is still lightuserdata (I think this type is valuable and should
  not be affected by the unification).
More complex libraries actually need less memory because they no longer
  need table wrappers, auxiliary tables or per-object metatables.

- I cannot see any noticeable speed difference between the current approach and the unified approach (method access, userdata pointer access, type
  checking ...).

- Current userdata objects are allocated by the Lua core and cannot be
  resized nor moved nor shared. Quite a few libraries end up storing
  a single void* in a userdata object. This is wasteful.
Leaving the allocation management to the C library seems to be the better
  solution in the long term IMHO.

- Many libraries need to store additional Lua objects along with userdata values and end up abusing the metatable or weak tables. This complicates
  type checking and/or garbage collection.

- Tables do not have a __gc() metamethod. This prevents wrapping userdata
  objects into a table because __gc() metamethods often need to have
  a look at the full object context and not just at individual userdata
  objects. Unifying userdata and tables would provide such a __gc()
  metamethod for tables. This is advantageous even for pure Lua code.

- Current userdata objects cannot be subclassed easily. Unless you have
the cooperation of the C library you need to wrap them up in a table and
  store any subclass fields there. But this approach is troublesome
  due to the lack of a __gc() metamethod for tables. See above.
  Subclassing gets trivial with a unified userdata/table object.

- Every unified table object has a void*. This may be useful for many more scopes than just plain library objects (e.g. module namespace tables).

- I have not taken a closer look at the issues that the unification rises for the Lua garbage collector. I may be wrong, but I think it gets simpler
  because the tricky handling for userdata objects can be dropped.

- Reducing diversity usually simplifies the implementation and this leads to fewer bugs. Some people believe this is the only measure that counts.

Bye,
     Mike



Reply | Threaded
Open this post in threaded view
|

__gc metamethod (was Re: Userdata/Table Unification)

Mark Hamburg-4
In reply to this post by Jamie Webb-3
Lua's __gc metamethod is actually based on the notion that the universe can
be divided into reachable objects and unreachable objects and all that the
__gc metamethod tells one is that the object made it's first and possibly
only transition from reachable to unreachable. If you don't resurrect the
object, this is for all intents and purposes a finalization notice.

Viewed in this light, it makes reasonable sense for tables as well as
userdata.

On the other hand, there is an expense to supporting it -- one has to check,
for example, whether or not an object has a __gc metamethod.

The implementation also seems to assume that relatively little work will
happen during a __gc metamethod. The Lua 5.1 garbage collector, for example,
can get into a very deep recursion if the work done by a __gc metamethod
drives the collector forward. That can be fixed by not driving the collector
forward while running __gc metamethods, but that probably adds to the
pressure to do relatively little during collection. Not supporting such
methods too widely probably helps discourage work being done in them.

Finally, one thing I have found myself wanting on occasion is a way to
report resurrection in a __gc metamethod so that I would get another call if
the object became unreachable again. This should just be the matter of
setting a flag back. Unfortunately, I can't think right now of why I thought
this might be useful.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: __gc metamethod (was Re: Userdata/Table Unification)

Edgar Toernig
Mark Hamburg wrote:
>
> Finally, one thing I have found myself wanting on occasion is a way to
> report resurrection in a __gc metamethod so that I would get another call if
> the object became unreachable again. This should just be the matter of
> setting a flag back. Unfortunately, I can't think right now of why I thought
> this might be useful.

Maybe the gc-method noticed that the userdata is still referenced by
other C structures (i.e. ref-count not zero) and has to be kept for
a while...

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: Userdata/Table Unification

Mike Pall-44
In reply to this post by Mike Pall-44
Hi,

Following up to myself:
> - Adding a void* to the table structure incurs no additional memory
>   overhead on most systems. I.e. the userdata pointer is free!
>   The current table structure is 32 bytes long (on 32 bit systems) and
>   anything between 29 and 36 bytes leads to the same effective memory
>   allocation due to malloc overhead and alignment constraints. YMMV.

I have to retract my comment. Apparently the word 'most' was a bit
optimistic. I was under the false impression that most OSS malloc
implementations had a common ancestor in Doug Lea's dlmalloc. So I just
tested my assumptions with Linux glibc and got the above results.

As Rici Lake pointed out to me in private e-mail his experiments on FreeBSD
and Mac OS X show different results. Upon closer inspection it seems that
the OSS malloc ancestry has diverged quite a bit. Everyone is using a
different set of optimizations. These lead to very different allocation
steps for small allocations (8*{2..2^14}-4 for Linux (32 bit),
2^{4..12} for FreeBSD (i386), 16*{1..32} for Mac OS X).

As the ongoing discussion shows, memory consumption is certainly not the
biggest issue for the table/userdata unification.

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: __gc metamethod (was Re: Userdata/Table Unification)

Mark Hamburg-4
In reply to this post by Edgar Toernig
Maybe, but then you potentially want to keep other things alive as well, so
it seems generally better to have a live objects table for userdata that is
externally reachable. The external references issue may have been one of the
reasons I found myself thinking about wanting a way to resurrect userdata in
the __gc metamethod (or to provide resurrection notification), but the
object graph issues lead me to realize that this wouldn't necessarily solve
the problem.

Mark

on 7/26/04 7:43 PM, Edgar Toernig at [hidden email] wrote:

> Mark Hamburg wrote:
>> 
>> Finally, one thing I have found myself wanting on occasion is a way to
>> report resurrection in a __gc metamethod so that I would get another call if
>> the object became unreachable again. This should just be the matter of
>> setting a flag back. Unfortunately, I can't think right now of why I thought
>> this might be useful.
> 
> Maybe the gc-method noticed that the userdata is still referenced by
> other C structures (i.e. ref-count not zero) and has to be kept for
> a while...
> 
> Ciao, ET.