Make type respect __name and add rawtype

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Make type respect __name and add rawtype

DarkWiiPlayer
Greetings Lua-l!

A while ago I was having a look at the Lua source code and found that in
certain parts, a `__name` value in the metatable is used for providing
custom type names to objects.

I've held the opinion for years now that there should be a way to change
what type is reported for an object by the `type` function, so I took
the opportunity that most of this is already implemented and wrote a
quick patch that adds this behavior, along with a `rawtype` function
that essentially works as the `type` function normally does.

I posted the patch on github for easier reading, but I'm also appending
it at the bottom of the mail:
https://gist.github.com/DarkWiiPlayer/bb0d153827990b4781f4f743d2bc7a3a

I will admit that I didn't spend much time considering the performance
impact of such a change, but I imagine for most Lua code it would make
almost no difference.

While it would be a breaking change, the only code that would be
affected is where the `__name` field is already being used for something
else, which I doubt would be in too many places.

As an alternative proposal: if changing the behavior of `type` is
considered a bad idea, I'd love to at least see some other function that
behaves like this, although I'm unsure about what it could be called.
`type` really is the best name in my opinion, with `name` (in reference
to the `__name` field name) being the next best option I can think of.
`class` might also be easy to remember for non-programmers, but
misleading to people with a programming background. The only other thing
I can think of is `typename`, which is probably the easiest to remember
(Does what `type` does, except when `__name` exists).

Greetings and happy holidays!

—————

diff --unified -r lua-5.4.0/src/lapi.c lua-5.4.0-rawtype/src/lapi.c
--- lua-5.4.0/src/lapi.c    2020-06-18 16:25:53.000000000 +0200
+++ lua-5.4.0-rawtype/src/lapi.c    2020-11-27 09:31:30.787330086 +0100
@@ -267,6 +267,13 @@
 }
 
 
+LUA_API const char *lua_objtypename (lua_State *L, int t) {
+  api_check(L, LUA_TNONE <= t && t < LUA_NUMTYPES, "invalid type");
+  const TValue *o = index2value(L, t);
+  return luaT_objtypename(L, o);
+}
+
+
 LUA_API int lua_iscfunction (lua_State *L, int idx) {
   const TValue *o = index2value(L, idx);
   return (ttislcf(o) || (ttisCclosure(o)));
diff --unified -r lua-5.4.0/src/lbaselib.c lua-5.4.0-rawtype/src/lbaselib.c
--- lua-5.4.0/src/lbaselib.c    2020-06-18 16:25:53.000000000 +0200
+++ lua-5.4.0-rawtype/src/lbaselib.c    2020-11-27 09:38:19.813573132 +0100
@@ -240,7 +240,7 @@
 }
 
 
-static int luaB_type (lua_State *L) {
+static int luaB_rawtype (lua_State *L) {
   int t = lua_type(L, 1);
   luaL_argcheck(L, t != LUA_TNONE, 1, "value expected");
   lua_pushstring(L, lua_typename(L, t));
@@ -248,6 +248,16 @@
 }
 
 
+static int luaB_type (lua_State *L) {
+  int t = lua_type(L, 1);
+  luaL_argcheck(L, t != LUA_TNONE, 1, "value expected");
+  //lua_pushstring(L, lua_typename(L, t));
+  const char *s = lua_objtypename(L, -1);
+  lua_pushstring(L, s);
+  return 1;
+}
+
+
 static int luaB_next (lua_State *L) {
   luaL_checktype(L, 1, LUA_TTABLE);
   lua_settop(L, 2);  /* create a 2nd argument if there isn't one */
@@ -504,6 +514,7 @@
   {"tonumber", luaB_tonumber},
   {"tostring", luaB_tostring},
   {"type", luaB_type},
+  {"rawtype", luaB_rawtype},
   {"xpcall", luaB_xpcall},
   /* placeholders */
   {LUA_GNAME, NULL},
diff --unified -r lua-5.4.0/src/lua.h lua-5.4.0-rawtype/src/lua.h
--- lua-5.4.0/src/lua.h    2020-06-18 16:25:54.000000000 +0200
+++ lua-5.4.0-rawtype/src/lua.h    2020-11-27 09:33:22.347601208 +0100
@@ -186,6 +186,7 @@
 LUA_API int             (lua_isuserdata) (lua_State *L, int idx);
 LUA_API int             (lua_type) (lua_State *L, int idx);
 LUA_API const char     *(lua_typename) (lua_State *L, int tp);
+LUA_API const char     *(lua_objtypename) (lua_State *L, int tp);
 
 LUA_API lua_Number      (lua_tonumberx) (lua_State *L, int idx, int
*isnum);
 LUA_API lua_Integer     (lua_tointegerx) (lua_State *L, int idx, int
*isnum);


Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Egor Skriptunoff-2
On Thu, Dec 24, 2020 at 1:54 PM DarkWiiPlayer wrote:
if changing the behavior of `type` is considered a bad idea

IMO, it's a bad idea to break compatibility with widespread check:
if type(v) == "table" then
 
I'd love to at least see some other function that behaves like this

The similar problem with numeric subtype was solved by introducing math.type()
So, it might be a good decision to introduce table.type()
If there is no "__name" metafield, table.type() may return "table"/"userdata"

You may ask:
Why should this function be located in the "table" library while it should work with both tables and userdata?
My answer:  Why not?
We already have functions in the "math" library working with strings: math.min(), math.max()
math.min("11", "2") == "11"  -- contra-mathematical behavior
So, why not have a function in the "table" library working with userdata?  :-)
 
Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

DarkWiiPlayer
Yes, in a way I understand why you wouldn't want `type(something) ==
"table"` to break, but at the same time, that's kind of the point of the
whole thing: to have a mechanism to tell other code that something is
*not* a table as far as it is concerned. That code should then be
expected to treat it as an opaque object, just as it would with
userdata. Just as with table indexing, and `__index`, where the indexing
code also just takes the returned value without asking questions. If
you'd have to use first `type`, then `table.type` if it's a table, that
really defeats the purpose of metaprogramming: at that point you can
just as well expect others to read `getmetatable(something).__name`
manually.

On 24/12/2020 14:45, Egor Skriptunoff wrote:

> On Thu, Dec 24, 2020 at 1:54 PM DarkWiiPlayer wrote:
>
>     if changing the behavior of `type` is considered a bad idea
>
>
> IMO, it's a bad idea to break compatibility with widespread check:
> if type(v) == "table" then
>  
>
>     I'd love to at least see some other function that behaves like this
>
>
> The similar problem with numeric subtype was solved by introducing
> math.type()
> So, it might be a good decision to introduce table.type()
> If there is no "__name" metafield, table.type() may return
> "table"/"userdata"
>
> You may ask:
> Why should this function be located in the "table" library while it
> should work with both tables and userdata?
> My answer:  Why not?
> We already have functions in the "math" library working with strings:
> math.min(), math.max()
> math.min("11", "2") == "11"  -- contra-mathematical behavior
> So, why not have a function in the "table" library working with
> userdata?  :-)
>  
Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Spar
In reply to this post by Egor Skriptunoff-2
Having a type metamethod is great, especially if LuaJIT implements it.
As example a game Garry's Mod had to make their own type function to respect __MetaName field for userdatas.
I agree with Egor this will break compatibility, but in the other hand every library have to name their own type meta field if they want to treat some objects differently. For example there is a JSON library with __jsontype field to handle different table encoding.
On 24 Dec 2020, 16:46 +0300, Egor Skriptunoff <[hidden email]>, wrote:
On Thu, Dec 24, 2020 at 1:54 PM DarkWiiPlayer wrote:
if changing the behavior of `type` is considered a bad idea

IMO, it's a bad idea to break compatibility with widespread check:
if type(v) == "table" then
 
I'd love to at least see some other function that behaves like this

The similar problem with numeric subtype was solved by introducing math.type()
So, it might be a good decision to introduce table.type()
If there is no "__name" metafield, table.type() may return "table"/"userdata"

You may ask:
Why should this function be located in the "table" library while it should work with both tables and userdata?
My answer:  Why not?
We already have functions in the "math" library working with strings: math.min(), math.max()
math.min("11", "2") == "11"  -- contra-mathematical behavior
So, why not have a function in the "table" library working with userdata?  :-)
 
Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Spar
In reply to this post by DarkWiiPlayer
table.type sounds better, the problem with getmetable().__name doesn't look that bad, __name was added recently, and I'm sure some people on older versions were doing something worse
On 24 Dec 2020, 17:28 +0300, DarkWiiPlayer <[hidden email]>, wrote:
Yes, in a way I understand why you wouldn't want `type(something) ==
"table"` to break, but at the same time, that's kind of the point of the
whole thing: to have a mechanism to tell other code that something is
*not* a table as far as it is concerned. That code should then be
expected to treat it as an opaque object, just as it would with
userdata. Just as with table indexing, and `__index`, where the indexing
code also just takes the returned value without asking questions. If
you'd have to use first `type`, then `table.type` if it's a table, that
really defeats the purpose of metaprogramming: at that point you can
just as well expect others to read `getmetatable(something).__name`
manually.

On 24/12/2020 14:45, Egor Skriptunoff wrote:
On Thu, Dec 24, 2020 at 1:54 PM DarkWiiPlayer wrote:

if changing the behavior of `type` is considered a bad idea


IMO, it's a bad idea to break compatibility with widespread check:
if type(v) == "table" then
 

I'd love to at least see some other function that behaves like this


The similar problem with numeric subtype was solved by introducing
math.type()
So, it might be a good decision to introduce table.type()
If there is no "__name" metafield, table.type() may return
"table"/"userdata"

You may ask:
Why should this function be located in the "table" library while it
should work with both tables and userdata?
My answer:  Why not?
We already have functions in the "math" library working with strings:
math.min(), math.max()
math.min("11", "2") == "11"  -- contra-mathematical behavior
So, why not have a function in the "table" library working with
userdata?  :-)
 
Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Egor Skriptunoff-2
In reply to this post by DarkWiiPlayer
On Thu, Dec 24, 2020 at 5:28 PM DarkWiiPlayer wrote:
If you'd have to use first `type`, then `table.type` if it's a table, that
really defeats the purpose of metaprogramming: at that point you can
just as well expect others to read `getmetatable(something).__name`
manually.

You don't need to invoke type() before table.type().
Use the following check:
if table.type(v) == "MyClass" then

Reading "__name" manually is not a good solution for two reasons:
1)
If v is a number or boolean then table.type(v) would return nil
(the same way as math.type() returns nil on non-numeric values)
But getmetatable(42).__name will raise an exception,
so you should check if metatable exists first.
2)
If metatable is protected by the "__metatable" field,
you are unable to read its "__name" field directly.
But of course "__name" is not a secret field, there should be
some way to read it even for protected metatables.
And table.type() would be that way.

Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Spar
I used __type with overridden type function for my scripts

On Thu, Dec 24, 2020 at 6:25 PM Egor Skriptunoff <[hidden email]> wrote:
On Thu, Dec 24, 2020 at 5:28 PM DarkWiiPlayer wrote:
If you'd have to use first `type`, then `table.type` if it's a table, that
really defeats the purpose of metaprogramming: at that point you can
just as well expect others to read `getmetatable(something).__name`
manually.

You don't need to invoke type() before table.type().
Use the following check:
if table.type(v) == "MyClass" then

Reading "__name" manually is not a good solution for two reasons:
1)
If v is a number or boolean then table.type(v) would return nil
(the same way as math.type() returns nil on non-numeric values)
But getmetatable(42).__name will raise an exception,
so you should check if metatable exists first.
2)
If metatable is protected by the "__metatable" field,
you are unable to read its "__name" field directly.
But of course "__name" is not a secret field, there should be
some way to read it even for protected metatables.
And table.type() would be that way.

Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Andrew Starks-2


On Thu, Dec 24, 2020 at 10:55 AM Spar <[hidden email]> wrote:
I used __type with overridden type function for my scripts

 

I've used similar magic and as has been pointed out, this has been done by many libraries.

To me, the border between "a good idea" and "a good idea for your code, but not for the language" is:

What's required for interoperability? In this case, is the situation improved for the user if the language offers this mechanism for subtyping tables?

Drawing this border is tricky. As it is, if I know how a certain library works, and I have chosen a different way, I can pretty easily patch their method into my own and deal with it appropriately, if not somewhat clumsily. Perhaps adding broader support for `__name` and a `table.type` method is a good idea. Can library authors take that mechanism and build a more complete class or type system? Would the existence of that feature make those systems more interoperable?

Or would it cause confusion, owing to the different ideas and requirements of different users and library authors, who would now be using `__name` and extending it in different ways?

I don't know the answer to this. Just asking questions. :)

Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Spar
As example when " library authors took that mechanism and built a more complete class or type system " happened was __pairs metamethod in my opinion.

On Thu, Dec 24, 2020 at 9:33 PM Andrew Starks <[hidden email]> wrote:


On Thu, Dec 24, 2020 at 10:55 AM Spar <[hidden email]> wrote:
I used __type with overridden type function for my scripts

 

I've used similar magic and as has been pointed out, this has been done by many libraries.

To me, the border between "a good idea" and "a good idea for your code, but not for the language" is:

What's required for interoperability? In this case, is the situation improved for the user if the language offers this mechanism for subtyping tables?

Drawing this border is tricky. As it is, if I know how a certain library works, and I have chosen a different way, I can pretty easily patch their method into my own and deal with it appropriately, if not somewhat clumsily. Perhaps adding broader support for `__name` and a `table.type` method is a good idea. Can library authors take that mechanism and build a more complete class or type system? Would the existence of that feature make those systems more interoperable?

Or would it cause confusion, owing to the different ideas and requirements of different users and library authors, who would now be using `__name` and extending it in different ways?

I don't know the answer to this. Just asking questions. :)

Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Sean Conner
In reply to this post by Andrew Starks-2
It was thus said that the Great Andrew Starks once stated:

> On Thu, Dec 24, 2020 at 10:55 AM Spar <[hidden email]> wrote:
>
> > I used *__type *with overridden *type *function for my scripts
>
> I've used similar magic and as has been pointed out, this has been done by
> many libraries.
>
> To me, the border between "a good idea" and "a good idea for your code, but
> not for the language" is:
>
> What's required for interoperability? In this case, is the situation
> improved for the user if the language offers this mechanism for subtyping
> tables?

  I don't quite understand the rational for this.  But then again, I'm a fan
of offensive programming [1] and would prefer for the program to "crash
fast, crash hard" to root out bugs.

  For a project, I had need to emulate Lua's file object.  So I have this
table that has a read, write, close, seek, setvbuf, flush and seek
functions.  Any code that explicitly checks for a type of 'userdata' or an
io.type of "file" will fail, yet if such checks are removed, it should run
fine.

  -spc

[1] https://interrupt.memfault.com/blog/defensive-and-offensive-programming

        This article describes my position.
Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

frank@kastenholz.org

The Principle of Least Astonishment leads one to the position that type() ought not change in a way that is not backwardly compatible. One could make “newtype()” or, perhaps, have type() return two values (one of the existing values plus, optionally,  the __name field from the meta table)

That said, I do like the idea in principle & have done hacks similar to those described

Frank

Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Dibyendu Majumdar
On Thu, 24 Dec 2020 at 22:23, [hidden email] <[hidden email]> wrote:
>
>
> The Principle of Least Astonishment leads one to the position that type() ought not change in a way that is not backwardly compatible. One could make “newtype()” or, perhaps, have type() return two values (one of the existing values plus, optionally,  the __name field from the meta table)
>
> That said, I do like the idea in principle & have done hacks similar to those described
>

In Ravi I added a new function that does this. I agree that leaving
the current type() function unchanged is better.

Regards
Reply | Threaded
Open this post in threaded view
|

Re: Make type respect __name and add rawtype

Thijs Schreijer


On 25 Dec 2020, at 02:19, Dibyendu Majumdar <[hidden email]> wrote:

On Thu, 24 Dec 2020 at 22:23, [hidden email] <[hidden email]> wrote:


The Principle of Least Astonishment leads one to the position that type() ought not change in a way that is not backwardly compatible. One could make “newtype()” or, perhaps, have type() return two values (one of the existing values plus, optionally,  the __name field from the meta table)

That said, I do like the idea in principle & have done hacks similar to those described


In Ravi I added a new function that does this. I agree that leaving
the current type() function unchanged is better.

Regards

Definitely with the “don’t break existing type” folks.

I’ve also implemented type replacements to deal with custom objects in the past. I think a generic form of math.type() would be nice (better than adding table.type). Eg. Something like “subtype()” that would incorporate math.type() as well as a meta-method/field to detect a type.

Alternatively extend type() in a compatible way with a second parameter to indicate you want the sub-type.

My 2cts
Thijs