Feature request: per-value uservalue

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Feature request: per-value uservalue

Xavier Wang
Some a long time ago Roberto make a joke about per-value metatable
here[1], in a feature request raised by cloud wu[2].

I found that's very exciting[3], but I missing the message that
Roberto says that is just a joke[4]. So sad, because this won't in Lua
because it can not support __gc meta-method.

__gc method may called when last value in Lua disappeared, if a
metatable attach to value, it's no way to know how to free the gc
objects. So metatable can not attach with value together.

but, for now only full user data has the user value, setting/getting
it using Lua API: lua_getuservalue()/lua_setuservalue(). So can we use
the extra bits in value and make every value has uservalue? a integer
is enough because we could put real value into a ref table. or make a
internal array for the uservalue? When a value disappear we could just
free the slot that this value use? or any ways to put a pointer in
this field?

I just very want this feature in future Lua, I think the per-value
metatable will in next Lua by mistake. So could anyone interesting
about this and re-discuss it in formal?


[1]: http://lua-users.org/lists/lua-l/2015-08/msg00336.html
[2]: http://lua-users.org/lists/lua-l/2015-08/msg00256.html
[3]: http://lua-users.org/lists/lua-l/2015-08/msg00393.html
[4]: http://lua-users.org/lists/lua-l/2015-08/msg00511.html

--
regards,
Xavier Wang.

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Tim Hill

> On Mar 19, 2017, at 12:03 PM, Xavier Wang <[hidden email]> wrote:
>
> Some a long time ago Roberto make a joke about per-value metatable
> here[1], in a feature request raised by cloud wu[2].
>
> I found that's very exciting[3], but I missing the message that
> Roberto says that is just a joke[4]. So sad, because this won't in Lua
> because it can not support __gc meta-method.
>
> __gc method may called when last value in Lua disappeared, if a
> metatable attach to value, it's no way to know how to free the gc
> objects. So metatable can not attach with value together.
>
> but, for now only full user data has the user value, setting/getting
> it using Lua API: lua_getuservalue()/lua_setuservalue(). So can we use
> the extra bits in value and make every value has uservalue? a integer
> is enough because we could put real value into a ref table. or make a
> internal array for the uservalue? When a value disappear we could just
> free the slot that this value use? or any ways to put a pointer in
> this field?
>

Other than tables or full userdata (which have per-value metatables), what do you need? I’m unclear on what use a metatable on a particular string, or a light userdata would have (why would it not be a full userdata in that case?).

—Tim



Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Xavier Wang

Tim Hill <[hidden email]>于2017年3月20日 周一上午11:12写道:

> On Mar 19, 2017, at 12:03 PM, Xavier Wang <[hidden email]> wrote:
>
> Some a long time ago Roberto make a joke about per-value metatable
> here[1], in a feature request raised by cloud wu[2].
>
> I found that's very exciting[3], but I missing the message that
> Roberto says that is just a joke[4]. So sad, because this won't in Lua
> because it can not support __gc meta-method.
>
> __gc method may called when last value in Lua disappeared, if a
> metatable attach to value, it's no way to know how to free the gc
> objects. So metatable can not attach with value together.
>
> but, for now only full user data has the user value, setting/getting
> it using Lua API: lua_getuservalue()/lua_setuservalue(). So can we use
> the extra bits in value and make every value has uservalue? a integer
> is enough because we could put real value into a ref table. or make a
> internal array for the uservalue? When a value disappear we could just
> free the slot that this value use? or any ways to put a pointer in
> this field?
>

Other than tables or full userdata (which have per-value metatables), what do you need? I’m unclear on what use a metatable on a particular string, or a light userdata would have (why would it not be a full userdata in that case?).

It opens safe global metatable for numbers and light userdata, etc. metatable could check uservalue to detect whether you could do specific things on specific values. Though I really want is per-value metatables...

It seems only full userdata and string has __gc metamethod, another solution is make per value metatable, and move __gc to gc object, use collectgarbage() or lua_gc() to specific __gc.




—Tim



--
regards,
Xavier Wang.
Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

云风 Cloud Wu
In reply to this post by Tim Hill


Tim Hill <[hidden email]>于2017年3月20日周一 上午11:12写道:
Other than tables or full userdata (which have per-value metatables), what do you need? I’m unclear on what use a metatable on a particular string, or a light userdata would have (why would it not be a full userdata in that case?).

lightuserdata can be a sub struct of an complex C/C++ object .

For example, in 3d engine, a node of a 3d object as a lightuserdata would be better than full userdata. If you query a node of a 3d object from lua , you can create a full userdata proxy for it, but you should also cache it in uservalue because it should be the same value in next query ( __eq is not enough if you want to use it as a table key).

Using lightuserdata (with metatable) for a 3d node would simplify the bindings of 3d engine and reduce the overhead.

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Tim Hill

On Mar 19, 2017, at 8:55 PM, 云风 Cloud Wu <[hidden email]> wrote:



Tim Hill <[hidden email]>于2017年3月20日周一 上午11:12写道:
Other than tables or full userdata (which have per-value metatables), what do you need? I’m unclear on what use a metatable on a particular string, or a light userdata would have (why would it not be a full userdata in that case?).

lightuserdata can be a sub struct of an complex C/C++ object .

For example, in 3d engine, a node of a 3d object as a lightuserdata would be better than full userdata. If you query a node of a 3d object from lua , you can create a full userdata proxy for it, but you should also cache it in uservalue because it should be the same value in next query ( __eq is not enough if you want to use it as a table key).

Using lightuserdata (with metatable) for a 3d node would simplify the bindings of 3d engine and reduce the overhead.


But a light userdata with a metatable basically *IS* a full userdata. The only difference is that the full userdata is allocated by Lua, and undergoes garbage collection .. which you ALSO want for the __gc. As you noted, the technique here is to allocate a full userdata that just holds a C pointer (aka a light userdata).

Your request won’t get you a lower overhead .. it will just get you something else that is identical to full userdata but with a different name.

—Tim

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Xavier Wang
2017-03-21 2:26 GMT+08:00 Tim Hill <[hidden email]>:

>
> On Mar 19, 2017, at 8:55 PM, 云风 Cloud Wu <[hidden email]> wrote:
>
>
>
> Tim Hill <[hidden email]>于2017年3月20日周一 上午11:12写道:
>>
>> Other than tables or full userdata (which have per-value metatables), what
>> do you need? I’m unclear on what use a metatable on a particular string, or
>> a light userdata would have (why would it not be a full userdata in that
>> case?).
>>
> lightuserdata can be a sub struct of an complex C/C++ object .
>
> For example, in 3d engine, a node of a 3d object as a lightuserdata would be
> better than full userdata. If you query a node of a 3d object from lua , you
> can create a full userdata proxy for it, but you should also cache it in
> uservalue because it should be the same value in next query ( __eq is not
> enough if you want to use it as a table key).
>
> Using lightuserdata (with metatable) for a 3d node would simplify the
> bindings of 3d engine and reduce the overhead.
>
>
> But a light userdata with a metatable basically *IS* a full userdata. The
> only difference is that the full userdata is allocated by Lua, and undergoes
> garbage collection .. which you ALSO want for the __gc. As you noted, the
> technique here is to allocate a full userdata that just holds a C pointer
> (aka a light userdata).
>
> Your request won’t get you a lower overhead .. it will just get you
> something else that is identical to full userdata but with a different name.
>
> —Tim
>

IMO there are two different things in Lua: values and objects. values
are the thing that in local value or table slots, and objects are the
subject of garbage collect.

A 123 (number) is a value, and a table {} is a object, and if you write:

local t = {}

then t is a table value, refers to a gc-able table object (the {} itself).

All of all, there are only two object type has the __gc behavior now,
table and userdata. so make such a api:

collectgarbage({}, function() print "collect!" end)

or:

lua_gc(LUA_SETGC, 1);

will hand the gc, so per-value metatable is not impossible, we only
need move __gc out of metatable.

meta-methods makes a fall-back behavior for Lua values (not only
objects), gc is a theoretical standalone feature, it only works for
specific object, even not for all gc subjects.

but this will break API (I think for Lua is not a big deal), so let's
backward a step: just make per-value uservalue.

for that, we can make different behavior for different value (not only
object!), percent 4 will different with digree 4. In game engine, we
only need a smart pointer, not need gc for node (node can be collect
when swap scene). well Lua has metatable to change value behavior, but
only for heavy object! We have change to make a cheap way to make all
Lua values could have it's own behavior.

in short, if you need the __gc behavior, use the full userdata. But
when you need change a single value's behavior and needn't gc support,
per-value (uservalue|metatable) makes more freedom to do what you
want.



--
regards,
Xavier Wang.

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Sean Conner
It was thus said that the Great Xavier Wang once stated:

> 2017-03-21 2:26 GMT+08:00 Tim Hill <[hidden email]>:
> >
> > On Mar 19, 2017, at 8:55 PM, 云风 Cloud Wu <[hidden email]> wrote:
> > Tim Hill <[hidden email]>于2017年3月20日周一 上午11:12写道:
> >>
> >> Other than tables or full userdata (which have per-value metatables), what
> >> do you need? I’m unclear on what use a metatable on a particular string, or
> >> a light userdata would have (why would it not be a full userdata in that
> >> case?).
> >>
> > lightuserdata can be a sub struct of an complex C/C++ object .
> >
> > For example, in 3d engine, a node of a 3d object as a lightuserdata would be
> > better than full userdata. If you query a node of a 3d object from lua , you
> > can create a full userdata proxy for it, but you should also cache it in
> > uservalue because it should be the same value in next query ( __eq is not
> > enough if you want to use it as a table key).
> >
> > Using lightuserdata (with metatable) for a 3d node would simplify the
> > bindings of 3d engine and reduce the overhead.
> >
> >
> > But a light userdata with a metatable basically *IS* a full userdata. The
> > only difference is that the full userdata is allocated by Lua, and undergoes
> > garbage collection .. which you ALSO want for the __gc. As you noted, the
> > technique here is to allocate a full userdata that just holds a C pointer
> > (aka a light userdata).
> >
> > Your request won’t get you a lower overhead .. it will just get you
> > something else that is identical to full userdata but with a different name.
>
> IMO there are two different things in Lua: values and objects. values
> are the thing that in local value or table slots, and objects are the
> subject of garbage collect.

  Lua has the following types:

        nil
        boolean
        number
        string gc but value semantics
        table gc per-instance metatable
        function gc
        thread gc
        userdata gc per-instance metatable
        lightuserdata*

(* In C, you can distinguish between a full userdata and a lightuserdata,
but in Lua, type() does not make that distinction and simply returns
'userdata')

> A 123 (number) is a value, and a table {} is a object, and if you write:
>
> local t = {}
>
> then t is a table value, refers to a gc-able table object (the {} itself).
>
> All of all, there are only two object type has the __gc behavior now,
> table and userdata. so make such a api:

  Actually, all the above types marked with 'gc' are subject to garbage
collection if no references exist to the "object".  

> collectgarbage({}, function() print "collect!" end)
>
> or:
>
> lua_gc(LUA_SETGC, 1);
>
> will hand the gc, so per-value metatable is not impossible, we only
> need move __gc out of metatable.
>
> meta-methods makes a fall-back behavior for Lua values (not only
> objects), gc is a theoretical standalone feature, it only works for
> specific object, even not for all gc subjects.

  First off, metamethods are not "fall-back" (they don't chain
indefinitely).  Second, the __gc metamethod *IS* required for proper
resource cleanup in the face of userdata (not all certainly, but anything
that deals with non-memory resources does).

> but this will break API (I think for Lua is not a big deal), so let's
> backward a step: just make per-value uservalue.
>
> for that, we can make different behavior for different value (not only
> object!), percent 4 will different with digree 4.

  What you are asking for is a typed Lua.  The above example could be done
with a proper type system (where derived types are considered distinct from
each other, unlike in C where derived types like:

        typedef double percent;
        typedef double degreee;

are considered the same).  There are probably ways to accomplish what you
want without a radical redesign of Lua.

> In game engine, we
> only need a smart pointer, not need gc for node (node can be collect
> when swap scene). well Lua has metatable to change value behavior, but
> only for heavy object! We have change to make a cheap way to make all
> Lua values could have it's own behavior.
>
> in short, if you need the __gc behavior, use the full userdata. But
> when you need change a single value's behavior and needn't gc support,
> per-value (uservalue|metatable) makes more freedom to do what you
> want.

  I can't tell if you have a version of Lua that does what you want, or are
asking for Lua to support what you want ..

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Xavier Wang
2017-03-21 15:28 GMT+08:00 Sean Conner <[hidden email]>:

> It was thus said that the Great Xavier Wang once stated:
>> 2017-03-21 2:26 GMT+08:00 Tim Hill <[hidden email]>:
>> >
>> > On Mar 19, 2017, at 8:55 PM, 云风 Cloud Wu <[hidden email]> wrote:
>> > Tim Hill <[hidden email]>于2017年3月20日周一 上午11:12写道:
>> >>
>> >> Other than tables or full userdata (which have per-value metatables), what
>> >> do you need? I’m unclear on what use a metatable on a particular string, or
>> >> a light userdata would have (why would it not be a full userdata in that
>> >> case?).
>> >>
>> > lightuserdata can be a sub struct of an complex C/C++ object .
>> >
>> > For example, in 3d engine, a node of a 3d object as a lightuserdata would be
>> > better than full userdata. If you query a node of a 3d object from lua , you
>> > can create a full userdata proxy for it, but you should also cache it in
>> > uservalue because it should be the same value in next query ( __eq is not
>> > enough if you want to use it as a table key).
>> >
>> > Using lightuserdata (with metatable) for a 3d node would simplify the
>> > bindings of 3d engine and reduce the overhead.
>> >
>> >
>> > But a light userdata with a metatable basically *IS* a full userdata. The
>> > only difference is that the full userdata is allocated by Lua, and undergoes
>> > garbage collection .. which you ALSO want for the __gc. As you noted, the
>> > technique here is to allocate a full userdata that just holds a C pointer
>> > (aka a light userdata).
>> >
>> > Your request won’t get you a lower overhead .. it will just get you
>> > something else that is identical to full userdata but with a different name.
>>
>> IMO there are two different things in Lua: values and objects. values
>> are the thing that in local value or table slots, and objects are the
>> subject of garbage collect.
>
>   Lua has the following types:
>
>         nil
>         boolean
>         number
>         string                  gc but value semantics
>         table                   gc per-instance metatable
>         function                gc
>         thread                  gc
>         userdata                gc per-instance metatable
>         lightuserdata*
>
> (* In C, you can distinguish between a full userdata and a lightuserdata,
> but in Lua, type() does not make that distinction and simply returns
> 'userdata')
>

the "value" I mean is the slot that has C type TValue, and the
"object" I mean is the slot of GCObject.

string, clearly, is the "immutable" sematics, you just can't change
it. What I mean "control" the gc, means it support __gc, i.e. you will
give a callback when Lua decide to delete it.

Let clearly in theoretical. Lua is a language that have "forever
lifetime" sematics. All value are live forever. garbage collect is
just a engineering optimization for that, if a value you really don't
need, Lua know she could collect it safely. In this way, the real
sematics thing is "strong/weak ref", Lua leave you a way to tell her
whether a value you really want, or doesn't matter. It real changes
Lua's behavior, but in another way.

In metatable, __gc and __mode are only two thing to specific this. it
just a subset of Lua's collect system. beside table and full userdata,
all other value are silently collected, without let you know about it.

But all other things in metatable are real hooks to Lua when "it
doesn't know how to do", e.g. call a number, or concat two table.
metatable makes some of value has the different behavior than others.
it's a "behavior trait".

the uservalue propersal, add the ability to "identity each other" with
Lua values. it's a value tag to specific "where this value from", it
could come with value assignment, to track the value's transform in
Lua context. It's just another complete different function vs.
metatable. it's more like something like "metadata".



>   I can't tell if you have a version of Lua that does what you want, or are
> asking for Lua to support what you want ..
>
>   -spc
>
>

I of course know that Lua is "open source and close development", I
used years of Lua and really LOVE this language. I just express my own
idea about Lua, whether Roberto decide what to do, I still love Lua,
All thing I want is to make her more perfect. So if I'm wrong, I'll
never mind someone tell me that. But I don't need someone (beside
Roberto) to teach me "Lua is not my own pet", I already know that.


--
regards,
Xavier Wang.

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

彭 书呆
In reply to this post by Tim Hill
在 2017/3/21 2:26, Tim Hill 写道:

>
>> On Mar 19, 2017, at 8:55 PM, 云风 Cloud Wu <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>
>> Tim Hill <[hidden email] <mailto:[hidden email]>>于2017年3月20日周一 上午11:12写道:
>>
>
> But a light userdata with a metatable basically *IS* a full userdata. The only difference is that the full userdata is allocated by Lua, and undergoes garbage collection .. which you ALSO want for the __gc. As you noted, the technique here is to allocate a full userdata that just holds a C pointer (aka a light userdata).
>
> Your request won’t get you a lower overhead .. it will just get you something else that is identical to full userdata but with a different name.
>
> —Tim
>


one major difference between light- and full- userdata is the equality test.

light userdata is tested by value while full userdata is tested by reference.

this make big difference when you need to use them as table keys.

--
the nerdy Peng / 书呆彭 /

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Tim Hill

>
> one major difference between light- and full- userdata is the equality test.
>
> light userdata is tested by value while full userdata is tested by reference.
>
> this make big difference when you need to use them as table keys.

hmmm, I’m thinking about that one. So a light userdata is really a void* pointer as a user value, so assignment just copies the “value” which is the void* pointer. Two values that have the same pointer, and will be equal. A full userdata is a garbage collected wrapper around a void* pointer, so assignment just copies the reference to this. Two values that refer to the same full userdata will be equal. Same semantics.

So I’m not not sure how they are different? Do you mean that a light userdata and a full userdata to the SAME void* will be different? If so, then yes, but that’s *really* not a good thing to do anyway.

—Tim



Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Viacheslav Usov
In reply to this post by 彭 书呆
On Thu, Mar 23, 2017 at 11:03 AM, 彭 书呆 <[hidden email]> wrote:

> this make big difference when you need to use them as table keys.

Like Tim, I fail to see any real difference here.

What is different is the allocation strategy. With light userdata, the host program has full control over memory allocation. With full userdata, it is fully[*] controlled by Lua. In the light case, not only is the host program solely responsible for memory allocation (and deallocation), it also gets no indication from Lua that a given userdatum is accessible in Lua (or not).

As consequence, I cannot use a custom allocator for a particular user data type, because it will be either unsafe (light userdata) or unusable (full userdata).

[*] I am aware that finalizers can decide whether memory is deallocated. I am talking about how memory gets deallocated, in the context of allocators customised for a particular data type.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

彭 书呆
In reply to this post by Tim Hill
在 2017/3/23 19:21, Tim Hill 写道:
>
> So I’m not not sure how they are different?
>
> —Tim
>
>

I mean, sure, I can make them the same semantic for the Lua side, but the difference comes at the binding layer. and I was not saying which way is better than the other. different problems need different solutions.

for instance, consider the APIs in the following script,

------------------------------------------------

   local obj1 = get_object_by_id(some_id)
   local obj2 = get_subobject_by_name(obj1, some_name)
   local obj3 = get_buddy_object(some_object)

------------------------------------------------

most of the time, an object is just an opaque pointer returned by the C library and points to some internal data structure. assume I don't need Lua's gc to manage lifetime, the binding layer can be trivially written (or auto generated) if light userdata is used to represent the objects. but more than often, there could be many different binding types and I want to set metatables on them, or the host program might use light userdata for other purpose, so I have to use full userdata, then store the pointer itself into the full userdata.

sometimes, the script only use == and ~= for equality test so the __eq metamethod might be sufficient. in other cases, the script need to store the userdata as table keys, so the binding layer need to keep track all the full userdata already created, and map C pointers to full userdata. I usually do this via a (weak) table, whose values are those full userdata containing pointer values, and whose keys are, well, pointer themselves hashed as light userdata.


--
the nerdy Peng / 书呆彭 /

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Francisco Olarte
In reply to this post by Tim Hill
On Thu, Mar 23, 2017 at 12:21 PM, Tim Hill <[hidden email]> wrote:
>> one major difference between light- and full- userdata is the equality test.
>> light userdata is tested by value while full userdata is tested by reference.
>> this make big difference when you need to use them as table keys.
> hmmm, I’m thinking about that one. So a light userdata is really a void* pointer as a user value, so assignment just copies the “value” which is the void* pointer. Two values that have the same pointer, and will be equal. A full userdata is a garbage collected wrapper around a void* pointer, so assignment just copies the reference to this. Two values that refer to the same full userdata will be equal. Same semantics.
> So I’m not not sure how they are different? Do you mean that a light userdata and a full userdata to the SAME void* will be different? If so, then yes, but that’s *really* not a good thing to do anyway.

Thaye make difference when locating them. In the program I'm currently
working I have objects with are managed by reference counted pointers
on the C++ side, and which emit callbacks into lua.

For the ones using light userdata I increment the C++ refcount when
passing them to lua. The lua side then wraps them into an object wich

1.- Registers into a module table using LUD as key.
2.- Deregisters from said table when no more callbacks coming and
3.- decrements the reference count of the LUD on GC.

The C++ object callback then calls to a module callback passing this
as a LUD, the module callback locates the object using it and calls
the object lua callback.

For others I use full userdata. In this case I put a metatable on them
but I have to store them somewhere where the C++ object can find them,
( I tipically use the registry using this as a LUD as a key ), because
in C++ I can not just build a new FUD, as it will compare different (
even if I played tricks to make them compare equal I would need to
locate the original, potentially lua-customised object ). So I need
some kind of table keyed by some kind of value-type ( I always use the
registry for it, and have a rule of only using lightuserdata pointing
to the object-owned memory, normally this ).

Each one has its advantages and problems, but the full userdata
approach needs some kind of value-typed key to be able to be located.

Francisco Olarte.






>
> —Tim
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Tim Hill
In reply to this post by 彭 书呆

> On Mar 23, 2017, at 8:26 AM, 彭 书呆 <[hidden email]> wrote:
>
>
> ... so I have to use full userdata, then store the pointer itself into the full userdata.
>

Isn’t that really the point here? For some reason you don’t want to create a full userdata to wrap a void* to some externally managed C struct? Why not?

—Tim



Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Coda Highland
On Thu, Mar 23, 2017 at 9:54 PM, Tim Hill <[hidden email]> wrote:
>
>> On Mar 23, 2017, at 8:26 AM, 彭 书呆 <[hidden email]> wrote:
>>
>>
>> ... so I have to use full userdata, then store the pointer itself into the full userdata.
>>
>
> Isn’t that really the point here? For some reason you don’t want to create a full userdata to wrap a void* to some externally managed C struct? Why not?
>

The "why not" was already explained: that means that you can have two
Lua objects pointing to the same C object, but the Lua objects compare
as unequal.

/s/ Adam

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Thomas Jericke
On 24.03.2017 15:34, Coda Highland wrote:

> On Thu, Mar 23, 2017 at 9:54 PM, Tim Hill <[hidden email]> wrote:
>>> On Mar 23, 2017, at 8:26 AM, 彭 书呆 <[hidden email]> wrote:
>>>
>>>
>>> ... so I have to use full userdata, then store the pointer itself into the full userdata.
>>>
>> Isn’t that really the point here? For some reason you don’t want to create a full userdata to wrap a void* to some externally managed C struct? Why not?
>>
> The "why not" was already explained: that means that you can have two
> Lua objects pointing to the same C object, but the Lua objects compare
> as unequal.
>
> /s/ Adam
>
You can, but you don't have to. To avoid creating two Lua objects
pointing to the same C objects is also possible with userdata, you just
need some care.
--
Thomas

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Coda Highland
On Fri, Mar 24, 2017 at 7:46 AM, Thomas Jericke <[hidden email]> wrote:

> On 24.03.2017 15:34, Coda Highland wrote:
>>
>> On Thu, Mar 23, 2017 at 9:54 PM, Tim Hill <[hidden email]> wrote:
>>>>
>>>> On Mar 23, 2017, at 8:26 AM, 彭 书呆 <[hidden email]> wrote:
>>>>
>>>>
>>>> ... so I have to use full userdata, then store the pointer itself into
>>>> the full userdata.
>>>>
>>> Isn’t that really the point here? For some reason you don’t want to
>>> create a full userdata to wrap a void* to some externally managed C struct?
>>> Why not?
>>>
>> The "why not" was already explained: that means that you can have two
>> Lua objects pointing to the same C object, but the Lua objects compare
>> as unequal.
>>
>> /s/ Adam
>>
> You can, but you don't have to. To avoid creating two Lua objects pointing
> to the same C objects is also possible with userdata, you just need some
> care.
> --
> Thomas

Well yes, of course, you make a weak-valued table with lightuserdata
keys and full userdata values and marshal your pointers through that,
but it's not immediately obvious and there are a number of possible
pitfalls in the implementation (for example, forgetting that
weak-valued tables are a thing makes this solution either ridiculously
leaky or annoyingly difficult to maintain).

This has come up a number of times before, so while I don't think that
"per-value metatables" is the right solution to the task, it does at
least indicate that the issue is fairly common.

/s/ Adam

Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Tim Hill
In reply to this post by Coda Highland

> On Mar 24, 2017, at 7:34 AM, Coda Highland <[hidden email]> wrote:
>
> On Thu, Mar 23, 2017 at 9:54 PM, Tim Hill <[hidden email]> wrote:
>>
>>> On Mar 23, 2017, at 8:26 AM, 彭 书呆 <[hidden email]> wrote:
>>>
>>>
>>> ... so I have to use full userdata, then store the pointer itself into the full userdata.
>>>
>>
>> Isn’t that really the point here? For some reason you don’t want to create a full userdata to wrap a void* to some externally managed C struct? Why not?
>>
>
> The "why not" was already explained: that means that you can have two
> Lua objects pointing to the same C object, but the Lua objects compare
> as unequal.
>
> /s/ Adam
>

Not if you have __eq in the metatable. And in any case this only happens if your C code actually creates more than one full userdata for a given C pointer, something that is in the control of the coder (it’s easy to use a weak table in the registry to lookup a full userdata by the encapsulated void* light userdata).

—Tim


Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Tim Hill
In reply to this post by Coda Highland

> On Mar 24, 2017, at 10:26 AM, Coda Highland <[hidden email]> wrote:
>
>
> Well yes, of course, you make a weak-valued table with lightuserdata
> keys and full userdata values and marshal your pointers through that,
> but it's not immediately obvious and there are a number of possible
> pitfalls in the implementation (for example, forgetting that
> weak-valued tables are a thing makes this solution either ridiculously
> leaky or annoyingly difficult to maintain).
>
> This has come up a number of times before, so while I don't think that
> "per-value metatables" is the right solution to the task, it does at
> least indicate that the issue is fairly common.
>


I don’t think is a particularly complex design pattern, but I do agree it might be one that is worth documenting somewhere (in the wiki perhaps?). In any case the OP was suggesting a core design change in Lua as a solution when I think this pattern does all he needs.

—Tim


Reply | Threaded
Open this post in threaded view
|

Re: Feature request: per-value uservalue

Coda Highland
On Fri, Mar 24, 2017 at 1:16 PM, Tim Hill <[hidden email]> wrote:

>
>> On Mar 24, 2017, at 10:26 AM, Coda Highland <[hidden email]> wrote:
>>
>>
>> Well yes, of course, you make a weak-valued table with lightuserdata
>> keys and full userdata values and marshal your pointers through that,
>> but it's not immediately obvious and there are a number of possible
>> pitfalls in the implementation (for example, forgetting that
>> weak-valued tables are a thing makes this solution either ridiculously
>> leaky or annoyingly difficult to maintain).
>>
>> This has come up a number of times before, so while I don't think that
>> "per-value metatables" is the right solution to the task, it does at
>> least indicate that the issue is fairly common.
>>
>
>
> I don’t think is a particularly complex design pattern, but I do agree it might be one that is worth documenting somewhere (in the wiki perhaps?). In any case the OP was suggesting a core design change in Lua as a solution when I think this pattern does all he needs.
>
> —Tim

And as it happens, I agree with you. I'm just explaining it instead of
dismissing it.

> Not if you have __eq in the metatable. And in any case this only happens if your C code actually creates more than one full userdata for a given C pointer, something that is in the control of the coder (it’s easy to use a weak table in the registry to lookup a full userdata by the encapsulated void* light userdata).

That works for normal equality comparisons but that doesn't help for
cases where raw equality is the only thing taken into consideration,
e.g. table indexing.

/s/ Adam

12