The 'I want nils in tables' problem (or is it?), random thougths and an entirely different proposal

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

The 'I want nils in tables' problem (or is it?), random thougths and an entirely different proposal

Sergey Zakharchenko
Hello,

I've seen quite many discussions on the topic, and due to the recent
5.4 work1 announcement and the caps-lock posts that followed I thought
I could throw in my two copecks. The opinion and/or proposal may be
stupid but I haven't seen them yet so for completeness, here they go.

Background: I must admit that I've never needed nils in my tables. I'm
quite fine with table packing with {n=select("#",...),...} and
unpacking/iterating respecting n for argument-passing needs. I'm still
using (heavily patched) Lua 5.1.

In any type system, there tends to be an hierarchy of 'definedness'
for values. E.g. in current Lua, a value which is true could mean
'OK', false could mean 'not OK', and nil could mean 'I don't know what
you're talking about'. Some people want to fit 'I know what you're
talking about but I am not decided' in the picture. These people also
tend to be familiar with JavaScript, or at least JSON. What they're
looking for is something *more*, not *less* defined than nil. That's
why the 'undef' proposal looks like a bit backwards to them. You're
(not literally, insert all caps mentioned above) trying to add
'undefined' from JavaScript to Lua, but nil already *is* the undefined
you're looking for.

There are Lua libraries in the wild which work with JSON, and they
face the 'nil cannot be a real table value' problem. The solutions
include one which I'm in favor of: presenting JSON null as a light
userdata with value NULL (the C NULL pointer, which usually happens to
be 0). There is little other use for such a light userdata (e.g.
nobody in their right mind would use it for some sort of a token while
they could just point to some private function/variable of theirs).
What's missing from this picture is that this userdata remains true
when treated as a boolean. So what I propose is to add such a light
userdata to the globals (as e.g. 'null'?) and allow it to be false in
a boolean context.

OT: nils as values are peanuts, wait until they want nils as *keys*...

Best regards,

--
DoubleF

Reply | Threaded
Open this post in threaded view
|

Re: The 'I want nils in tables' problem (or is it?), random thougths and an entirely different proposal

Axel Kittenberger
You're right that the JS-counterpart of 'nil' is 'undefined'. It's the result if a key is queried of a table ("object" in JS) is not existent.

null in JS or somewhat isn't the issue, a good counterpart in Lua is created simply by doing defining the global "null = { }".  Maybe add a metatable for it, to hinter key adding/reading of it. The only difference by itself it defaults to false in JS, while {} defaults to true. Anyway, writing a lot of Javascript, I can tell you "null" isn't the issue and one shouldn't use it in Javascript ever anyway (except some backward API demands it).

However, the difference is, Javascript does have a state of keys in objects that is "this is a known key of the object, but it is set to undefined". Normally one wouldn't notice, unless you either use the Object.keys( o ) function to get all keys, where it will show up, or use the "'key' in o" operator or use for( let key in Object ) etc. I suppose in some implementations it might also make a performance difference due to virtual classing.

I can tell you only of a recent experience of mine
  sync{ table.optiom, key1 = 'foo', key2 = 'bar' }
now "optiom" is clearly a spelling mistake, it thus defaulted to nil and I had no way in sync() to check for it and it simply resulted in assuming it wasn't given at all. I think, I'll add a metatable to 'table' to error on non-existing keys as workaround, but anyway, that was a typical paper cut due to the language used.

And yes I'm aware a calling syntax like this wouldn't be available in JS anyway. There I would have to do it like
  sync( table.option, 'key1', 'foo', 'key2', 'bar' ).. or sync( [ table.option ], { key1: 'foo', key2: 'bar' } )
so yes it's a complaint about an issue that couldn't arise in JS anyway due to less elegant calling syntax to start with.

In Lua upvalues can be the 'it is there but nil', as well function parameters and function return values, I always considered it inconsistent that it wasn't available for tables.

PS: I'm still happy with the Lua 5.x line having booleans. Especially since Lua 5.3, the question would otherwise be, would "not nil" result in an integer or floating 1? :-) And yes it also helps greatly for people having to work with JSON.

On Fri, Mar 16, 2018 at 9:57 AM, Sergey Zakharchenko <[hidden email]> wrote:
Hello,

I've seen quite many discussions on the topic, and due to the recent
5.4 work1 announcement and the caps-lock posts that followed I thought
I could throw in my two copecks. The opinion and/or proposal may be
stupid but I haven't seen them yet so for completeness, here they go.

Background: I must admit that I've never needed nils in my tables. I'm
quite fine with table packing with {n=select("#",...),...} and
unpacking/iterating respecting n for argument-passing needs. I'm still
using (heavily patched) Lua 5.1.

In any type system, there tends to be an hierarchy of 'definedness'
for values. E.g. in current Lua, a value which is true could mean
'OK', false could mean 'not OK', and nil could mean 'I don't know what
you're talking about'. Some people want to fit 'I know what you're
talking about but I am not decided' in the picture. These people also
tend to be familiar with JavaScript, or at least JSON. What they're
looking for is something *more*, not *less* defined than nil. That's
why the 'undef' proposal looks like a bit backwards to them. You're
(not literally, insert all caps mentioned above) trying to add
'undefined' from JavaScript to Lua, but nil already *is* the undefined
you're looking for.

There are Lua libraries in the wild which work with JSON, and they
face the 'nil cannot be a real table value' problem. The solutions
include one which I'm in favor of: presenting JSON null as a light
userdata with value NULL (the C NULL pointer, which usually happens to
be 0). There is little other use for such a light userdata (e.g.
nobody in their right mind would use it for some sort of a token while
they could just point to some private function/variable of theirs).
What's missing from this picture is that this userdata remains true
when treated as a boolean. So what I propose is to add such a light
userdata to the globals (as e.g. 'null'?) and allow it to be false in
a boolean context.

OT: nils as values are peanuts, wait until they want nils as *keys*...

Best regards,

--
DoubleF


Reply | Threaded
Open this post in threaded view
|

Re: The 'I want nils in tables' problem (or is it?), random thougths and an entirely different proposal

Rena
On Fri, Mar 16, 2018, 08:37 Axel Kittenberger, <[hidden email]> wrote:
You're right that the JS-counterpart of 'nil' is 'undefined'. It's the result if a key is queried of a table ("object" in JS) is not existent.

null in JS or somewhat isn't the issue, a good counterpart in Lua is created simply by doing defining the global "null = { }".  Maybe add a metatable for it, to hinter key adding/reading of it. The only difference by itself it defaults to false in JS, while {} defaults to true. Anyway, writing a lot of Javascript, I can tell you "null" isn't the issue and one shouldn't use it in Javascript ever anyway (except some backward API demands it).

However, the difference is, Javascript does have a state of keys in objects that is "this is a known key of the object, but it is set to undefined". Normally one wouldn't notice, unless you either use the Object.keys( o ) function to get all keys, where it will show up, or use the "'key' in o" operator or use for( let key in Object ) etc. I suppose in some implementations it might also make a performance difference due to virtual classing.

I can tell you only of a recent experience of mine
  sync{ table.optiom, key1 = 'foo', key2 = 'bar' }
now "optiom" is clearly a spelling mistake, it thus defaulted to nil and I had no way in sync() to check for it and it simply resulted in assuming it wasn't given at all. I think, I'll add a metatable to 'table' to error on non-existing keys as workaround, but anyway, that was a typical paper cut due to the language used.

And yes I'm aware a calling syntax like this wouldn't be available in JS anyway. There I would have to do it like
  sync( table.option, 'key1', 'foo', 'key2', 'bar' ).. or sync( [ table.option ], { key1: 'foo', key2: 'bar' } )
so yes it's a complaint about an issue that couldn't arise in JS anyway due to less elegant calling syntax to start with.

In Lua upvalues can be the 'it is there but nil', as well function parameters and function return values, I always considered it inconsistent that it wasn't available for tables.

PS: I'm still happy with the Lua 5.x line having booleans. Especially since Lua 5.3, the question would otherwise be, would "not nil" result in an integer or floating 1? :-) And yes it also helps greatly for people having to work with JSON.

On Fri, Mar 16, 2018 at 9:57 AM, Sergey Zakharchenko <[hidden email]> wrote:
Hello,

I've seen quite many discussions on the topic, and due to the recent
5.4 work1 announcement and the caps-lock posts that followed I thought
I could throw in my two copecks. The opinion and/or proposal may be
stupid but I haven't seen them yet so for completeness, here they go.

Background: I must admit that I've never needed nils in my tables. I'm
quite fine with table packing with {n=select("#",...),...} and
unpacking/iterating respecting n for argument-passing needs. I'm still
using (heavily patched) Lua 5.1.

In any type system, there tends to be an hierarchy of 'definedness'
for values. E.g. in current Lua, a value which is true could mean
'OK', false could mean 'not OK', and nil could mean 'I don't know what
you're talking about'. Some people want to fit 'I know what you're
talking about but I am not decided' in the picture. These people also
tend to be familiar with JavaScript, or at least JSON. What they're
looking for is something *more*, not *less* defined than nil. That's
why the 'undef' proposal looks like a bit backwards to them. You're
(not literally, insert all caps mentioned above) trying to add
'undefined' from JavaScript to Lua, but nil already *is* the undefined
you're looking for.

There are Lua libraries in the wild which work with JSON, and they
face the 'nil cannot be a real table value' problem. The solutions
include one which I'm in favor of: presenting JSON null as a light
userdata with value NULL (the C NULL pointer, which usually happens to
be 0). There is little other use for such a light userdata (e.g.
nobody in their right mind would use it for some sort of a token while
they could just point to some private function/variable of theirs).
What's missing from this picture is that this userdata remains true
when treated as a boolean. So what I propose is to add such a light
userdata to the globals (as e.g. 'null'?) and allow it to be false in
a boolean context.

OT: nils as values are peanuts, wait until they want nils as *keys*...

Best regards,

--
DoubleF

The problem with `null = {}` is it's not standard. Every library has its own null that isn't equal to other libraries' null (and which, confusingly, evaluates to true).

Using a light userdata with value NULL is a good idea, but I think there's no way to create that without some additional C code, and it would still evaluate to true.

For what it's worth, I like the new `undef` proposal. I just don't like the syntax. I'd much rather `table.delete` or `delete k in t` for removing and something like `table.haskey` or `k in t` for checking.