Undefined variables returning nil

classic Classic list List threaded Threaded
64 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Undefined variables returning nil

William Sumner
Lua's behavior of silently returning nil for undefined variables can lead to bugs that propagate undetected. Has thought been given to changing this in a major version update, or are there reasons for retaining this behavior? I'm aware of strict.lua, but it only provides safety for global/module scope.

Preston
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Petite Abeille

On Mar 19, 2013, at 9:23 PM, William Sumner <[hidden email]> wrote:

> Lua's behavior of silently returning nil for undefined variables can lead to bugs that propagate undetected. Has thought been given to changing this in a major version update, or are there reasons for retaining this behavior? I'm aware of strict.lua, but it only provides safety for global/module scope.

Something like this perhaps?

do
  local error = error
  local print = print
  local _ENV = setmetatable( {}, { __index = function( self, aKey ) error( ( 'Undefined %q' ):format( aKey ) ) end } )

  foo = 1
  print( 1, foo )
  print( 2, bar )
end


Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

William Sumner
On Mar 19, 2013, at 2:51 PM, Petite Abeille <[hidden email]> wrote:

> On Mar 19, 2013, at 9:23 PM, William Sumner <[hidden email]> wrote:
>
>> Lua's behavior of silently returning nil for undefined variables can lead to bugs that propagate undetected. Has thought been given to changing this in a major version update, or are there reasons for retaining this behavior? I'm aware of strict.lua, but it only provides safety for global/module scope.
>
> Something like this perhaps?
>
> do
>  local error = error
>  local print = print
>  local _ENV = setmetatable( {}, { __index = function( self, aKey ) error( ( 'Undefined %q' ):format( aKey ) ) end } )
>
>  foo = 1
>  print( 1, foo )
>  print( 2, bar )
> end

Like strict.lua, this only covers undefined globals.

Preston
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Kaj Eijlers
>>Like strict.lua, this only covers undefined globals.

Well, if a local isn't defined it's supposed to look for the global. If that wasn't the case you couldn't use globals anymore (except prefixing em all),,,,,,so the global metatable seems to be the mst useful mechanism for detecting undefined variables.
If you didn't define a local, and a global exists, it's technically not undefined.

Or do you mean to trap nil on table members (ie in a class system)? Then you gotta set the metatable on that table.

But having it look for a local and not falling back to globals seems rather un-Lua to me.

Kaj

Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Petite Abeille
In reply to this post by William Sumner

On Mar 19, 2013, at 10:37 PM, William Sumner <[hidden email]> wrote:

> Like strict.lua, this only covers undefined globals.

Indeed. What else is there? Undefined local? What would that be?


Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

William Sumner
In reply to this post by Kaj Eijlers
On Mar 19, 2013, at 3:52 PM, Kaj Eijlers <[hidden email]> wrote:

> >>Like strict.lua, this only covers undefined globals.
>
> Well, if a local isn't defined it's supposed to look for the global. If that wasn't the case you couldn't use globals anymore (except prefixing em all),,,,,,so the global metatable seems to be the mst useful mechanism for detecting undefined variables.
> If you didn't define a local, and a global exists, it's technically not undefined.
>
> Or do you mean to trap nil on table members (ie in a class system)? Then you gotta set the metatable on that table.
>
> But having it look for a local and not falling back to globals seems rather un-Lua to me.
>
> Kaj
>

I'm talking about ALL attempted accesses of undefined variables, including table/array/class elements. I was wondering if addressing this in a major version update of the language has been considered before and if there are reasons for retaining the current behavior.

Preston
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Petite Abeille

On Mar 19, 2013, at 11:05 PM, William Sumner <[hidden email]> wrote:

> I'm talking about ALL attempted accesses of undefined variables, including table/array/class elements.

Lua has only tables. As shown, you can trap __index and decide what you want to do. It's up to you.


Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

William Sumner
On Mar 19, 2013, at 4:22 PM, Petite Abeille <[hidden email]> wrote:

>
> On Mar 19, 2013, at 11:05 PM, William Sumner <[hidden email]> wrote:
>
>> I'm talking about ALL attempted accesses of undefined variables, including table/array/class elements.
>
> Lua has only tables. As shown, you can trap __index and decide what you want to do. It's up to you.

I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient. I'm suggesting that, unless there's a good reason for the current behavior that I'm unaware of, relying on user-implemented solutions is undesirable and that the language should raise an error because the current default behavior is unsafe.

Preston
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Kevin Martin

On 19 Mar 2013, at 22:32, William Sumner wrote:

> I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient. I'm suggesting that, unless there's a good reason for the current behavior that I'm unaware of, relying on user-implemented solutions is undesirable and that the language should raise an error because the current default behavior is unsafe.

By raising an error , you are removing the language's method of determining if a table contains a key (if table[key] == nil then … end). Surely you agree this is a fairly critical operation? How do suggest this is done instead?

Kev


Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Petite Abeille
In reply to this post by William Sumner

On Mar 19, 2013, at 11:32 PM, William Sumner <[hidden email]> wrote:

> I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient.

Non sequitur. What else is there?

Perhaps if you spell out what specific problem/scenario you are trying to sort out, then someone could suggest something constructive. Generalities are not going to get you very far.

> I'm suggesting that, unless there's a good reason for the current behavior that I'm unaware of, relying on user-implemented solutions is undesirable

I'm a user, and I'm just fine as it is. But thanks for your concern.

> and that the language should raise an error because the current default behavior is unsafe.

Hmm, thanks, but no thanks. Feel free to change your own copy of Lua in any way, shape, or form, you see fit.


Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Geoff Leyland
In reply to this post by William Sumner
On 20/03/2013, at 11:32 AM, William Sumner <[hidden email]> wrote:

> I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient. I'm suggesting that, unless there's a good reason for the current behavior that I'm unaware of, relying on user-implemented solutions is undesirable and that the language should raise an error because the current default behavior is unsafe.

Please, no.  "if table[key] then something() end" works.  Otherwise we'd have to invent new syntax to check whether keys existed.

Giving the user enough rope to implement solutions to their particular problems is a big part of what makes Lua what it is: relying on user-implemented solutions *is* desirable.


Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

William Sumner
In reply to this post by Kevin Martin
On Mar 19, 2013, at 4:40 PM, Kevin Martin <[hidden email]> wrote:

>
> On 19 Mar 2013, at 22:32, William Sumner wrote:
>
>> I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient. I'm suggesting that, unless there's a good reason for the current behavior that I'm unaware of, relying on user-implemented solutions is undesirable and that the language should raise an error because the current default behavior is unsafe.
>
> By raising an error , you are removing the language's method of determining if a table contains a key (if table[key] == nil then … end). Surely you agree this is a fairly critical operation? How do suggest this is done instead?
>
> Kev

One solution might be to introduce an operator for determining if a table contains an element with a given key.

t = { a = "apple" }
if "a" in t then print "evaluates to true" end
if "b" in t then print "never prints" end

Another solution might be a table.contains() function:

t = { a = "apple" }
if t:contains "a" then print "evaluates to true" end

Preston
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Petite Abeille

On Mar 19, 2013, at 11:53 PM, William Sumner <[hidden email]> wrote:

> One solution might be to introduce an operator for determining if a table contains an element with a given key.

That's what table[ key ] is for.  No point moving the issue around.

__index = error + rawget and Bob's your uncle.  Alternatively, judicious use of assert may have its charms as well.

And you can always use you very own access function, say KABOOM = function( aTable, aKey ) return aTable[ aKey ] or error( 'KABOOM' ) end

print( KABOOM( { a = 1 }, 'b' ) )

In short, the choice is really *yours*.



Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

William Sumner
In reply to this post by Petite Abeille
On Mar 19, 2013, at 4:43 PM, Petite Abeille <[hidden email]> wrote:

> On Mar 19, 2013, at 11:32 PM, William Sumner <[hidden email]> wrote:
>
>> I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient.
>
> Non sequitur. What else is there?

I'm not sure what you're calling a non sequitur, but in case you misunderstood me, silently getting nil back when accessing a non-existent table element is unsafe, particularly because tables are used for so much, so simply catching undefined global accesses and not undefined elements is insufficient.

> Perhaps if you spell out what specific problem/scenario you are trying to sort out, then someone could suggest something constructive. Generalities are not going to get you very far.

I don't know how to be more specific. The behavior of silently returning nil for accesses of undefined variables is unsafe, so I was wondering if changing this behavior has been considered for a major version update of the language, and if there are reasons for retaining the current behavior. I'm aware of user-implemented solutions, but it would seem to me preferable to make the safe behavior the default since there isn't a valid reason I can think of that one would want an uncaught bug to be standard behavior.

>> I'm suggesting that, unless there's a good reason for the current behavior that I'm unaware of, relying on user-implemented solutions is undesirable
>
> I'm a user, and I'm just fine as it is. But thanks for your concern.
>
>> and that the language should raise an error because the current default behavior is unsafe.
>
> Hmm, thanks, but no thanks. Feel free to change your own copy of Lua in any way, shape, or form, you see fit.

Well, I don't understand your defensiveness; I'm just making a suggestion.

Preston
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

William Sumner
In reply to this post by Geoff Leyland
On Mar 19, 2013, at 4:43 PM, Geoff Leyland <[hidden email]> wrote:

> On 20/03/2013, at 11:32 AM, William Sumner <[hidden email]> wrote:
>
>> I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient. I'm suggesting that, unless there's a good reason for the current behavior that I'm unaware of, relying on user-implemented solutions is undesirable and that the language should raise an error because the current default behavior is unsafe.
>
> Please, no.  "if table[key] then something() end" works.  Otherwise we'd have to invent new syntax to check whether keys existed.
>
> Giving the user enough rope to implement solutions to their particular problems is a big part of what makes Lua what it is: relying on user-implemented solutions *is* desirable.

Why is it desirable to rely on user-implemented workarounds for unsafe behavior? Lua is targeted at non-professional programmers and is often exposed to them through scripting interfaces; this seems like an undue burden unless there is a technical reason for it.

Preston
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Kaj Eijlers
In reply to this post by William Sumner
The problem seems, to me, that you can't properly define this solution.

What if a certain metatable gives something back for only certain non-existing members. Should these then be a non-existing entry, or not?

By doing the null-check yourself you can 
a) operate optimal when you don't care
b) do something very specific when you do.

So the definition of the problem you're trying to solve is use-case specific. This is why metatables are cool and a fundamental concept. 
Wrap all your table creation with a metatable and you can have what you seem to desire.

Kaj
Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Kevin Martin
In reply to this post by William Sumner

On 19 Mar 2013, at 22:53, William Sumner wrote:

It was meant as a rhetorical question, to ask yourself. However, to comment on your suggestions:

1) The contains function is clearly a bad idea. How would you implement it?
2) The in operator is better-ish, the first question that comes to mind is how you would modify the C API, as lua_gettable/etc can no longer push nil.

However, I'm just trying to give you things to think about, I'm not interested in engaging in a serious discussion.

If you really want to make this change, you are quite welcome as someone has already said.

I guess my main objection is that I don't actually think there's a problem. I don't see the unsafeness. I also don't think you have explained why it is unsafe, only stated that it is unsafe.

1) nil literally means no-value. Is it not reasonable for a table access to return no-value when there is no value to return?
2) Any attempt to do anything with a nil value yields an error, so if getting the nil value was a mistake as your error is trying to catch, the error will be produced as soon as you try and do anything with the value, see below. Is this so far from you error?
3) Not handling the case of no-value is definitely an error, but that is bad programming, nothing to do with the language.

Kev

-------------------------------------------

> do
>> local x = nil
>> print(x+5)
>> end
stdin:3: attempt to perform arithmetic on local 'x' (a nil value)
stack traceback:
        stdin:3: in main chunk
        [C]: in ?
>

> do
>> local x = nil
>> x()
>> end
stdin:3: attempt to call local 'x' (a nil value)
stack traceback:
        stdin:3: in main chunk
        [C]: in ?
>

> do
>> local x = nil
>> print(x.a)
>> end
stdin:3: attempt to index local 'x' (a nil value)
stack traceback:
        stdin:3: in main chunk
        [C]: in ?
>


Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

Petite Abeille
In reply to this post by William Sumner

On Mar 20, 2013, at 12:16 AM, William Sumner <[hidden email]> wrote:

> but in case you misunderstood me, silently getting nil back when accessing a non-existent table element is unsafe

Hmmm… this is a feature though. There is nothing wrong with accessing a key which doesn't exist. Table access has nothing to say about 'safety'.

On the other hand… are you a Python person perhaps? Looking for something like KeyError? If so, it's trivial to implement your own.

So, as mentioned, feel free to shape your own Lua whichever way you see fit.




Reply | Threaded
Open this post in threaded view
|

Re: Undefined variables returning nil

petah
In reply to this post by William Sumner
>> On Mar 19, 2013, at 11:32 PM, William Sumner <[hidden email]> wrote:
>>
>>> I know Lua only has tables. Because they're used for arrays, classes, and more, catching only undefined globals is insufficient.
>>
>> Non sequitur. What else is there?
>
>I'm not sure what you're calling a non sequitur, but in case you misunderstood me, silently getting nil back when accessing a non-existent table element is unsafe, particularly because tables are used for so much, so simply catching undefined global accesses and not undefined elements is insufficient.

Insufficient to /you/ but absurd to other Lua users, hence a non-sequitur. Most of us don't want tab[rnd()] to return anything but nil.

It seems you're coming from a higher-level language to claim what is sufficient and what isn't -- maybe ActionScript which IIRC differentiates between uninitialized, nil and value -- but all of this is subjective and ultimately a trade-off.

Lua aims for simplicity and speed: nil means "is not there" unless you want to tack on a higher-level meaning, as suggested earlier.

That's already a step up (abstraction-wise) from C/C++; if you don't appreciate Lua's simplicity and want more hand-holding then stick to a higher-level language. Or better yet, embed Lua where you think it benefits you and use other languages where you think it doesn't.

>> Hmm, thanks, but no thanks. Feel free to change your own copy of Lua in any way, shape, or form, you see fit.
>
>Well, I don't understand your defensiveness; I'm just making a suggestion.

Don't read ulterior motives in this remark: you REALLY are welcome to change the language as you see fit.

cheers,

-- p



Reply | Threaded
Open this post in threaded view
|

RE: Undefined variables returning nil

Ivo Beltchev-2
In reply to this post by Kevin Martin
I've recently struggled with this problem.

I want to create inheritance of environments. I set the parent environment
as __index for the child environment. So when you access a variable from the
child environment that's not there, it will look at the parent. Pretty
standard stuff - kind of like scopes and namespaces work in C++.

Let's say the parent has {a=5, b=10}. The child has {a="five", c=true}. If I
access "a" I will get "five" instead of 5. So far so good. The two a's are
two very different variables. But if I set the child's a to nil, I will
start using the one from the parent because there is no way to distinguish
between "not defined" and "defined, but currently nil":
print(a) -> five
a=nil
print(a) -> 5

In contrast local variables work differently. If I have a global 'a' and a
local 'a', then if the local variable is nil using 'a' will still refer to
the local variable and not the global.

So my solution for now is to use local variables in the global chunk. But
there are limitations: there's a limit for how many local variables I can
have in a chunk, how many upvalues a function can have, and also I can't
have multiple source files sharing the same environment (because each
becomes its own chunk). Also there is a problem with using a solution like
Pluto for persisting the state. Since many functions have upvalues, they
have to be saved together with their bytecode, which seriously bloats the
size of the state.

Ivo

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Kevin Martin
Sent: Tuesday, March 19, 2013 4:29 PM
To: Lua mailing list
Subject: Re: Undefined variables returning nil


On 19 Mar 2013, at 22:53, William Sumner wrote:

It was meant as a rhetorical question, to ask yourself. However, to comment
on your suggestions:

1) The contains function is clearly a bad idea. How would you implement it?
2) The in operator is better-ish, the first question that comes to mind is
how you would modify the C API, as lua_gettable/etc can no longer push nil.

However, I'm just trying to give you things to think about, I'm not
interested in engaging in a serious discussion.

If you really want to make this change, you are quite welcome as someone has
already said.

I guess my main objection is that I don't actually think there's a problem.
I don't see the unsafeness. I also don't think you have explained why it is
unsafe, only stated that it is unsafe.

1) nil literally means no-value. Is it not reasonable for a table access to
return no-value when there is no value to return?
2) Any attempt to do anything with a nil value yields an error, so if
getting the nil value was a mistake as your error is trying to catch, the
error will be produced as soon as you try and do anything with the value,
see below. Is this so far from you error?
3) Not handling the case of no-value is definitely an error, but that is bad
programming, nothing to do with the language.

Kev

-------------------------------------------

> do
>> local x = nil
>> print(x+5)
>> end
stdin:3: attempt to perform arithmetic on local 'x' (a nil value) stack
traceback:
        stdin:3: in main chunk
        [C]: in ?
>

> do
>> local x = nil
>> x()
>> end
stdin:3: attempt to call local 'x' (a nil value) stack traceback:
        stdin:3: in main chunk
        [C]: in ?
>

> do
>> local x = nil
>> print(x.a)
>> end
stdin:3: attempt to index local 'x' (a nil value) stack traceback:
        stdin:3: in main chunk
        [C]: in ?
>




1234