Embedded 'nil' values - a suggestion

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Embedded 'nil' values - a suggestion

Gavin Kistner
One of Lua's greatest assets is the simplicity of the core. No Arrays  
and Hashes and Queues, we have simply Tables.

The merging of all of these concepts into one atomic type, however,  
has the collision of embedded nils - tables-as-hashes need a way to  
remove a key entirely; tables-as-arrays need a way to store an  
'empty' value.

For the purposes of this discussion, let's assume that it makes sense  
to have both nil (nothingness, no value) and false (a value  
indicating non-truth) in the Lua language. We could go one step  
farther, like JavaScript, and add a third, similar value:  
'undefined'. (That's what it's called in JavaScript land; we could  
call it whatever we want.)

In JavaScript, this:
        function foo( a, b ){
           alert( a + ':' + b )
        }
        foo( null )
reports:
        "null:undefined"

In JavaScript this:
        var myHash = { foo:null }
        alert( myHash.foo + ':' + myHash.bar )
reports:
        "null:undefined"

And to remove a key from a hash, you use the delete operator:
        delete myHash.foo

In JavaScript, null is not equal to false, but it IS equal to  
undefined. They are two shades of a very similar concept.


My proposal is to use a concept similar to JavaScript (3 'dead'  
values) but in a slightly different way.
        * false is a value that means "I am not true!"
        * undefined (or other name) is a 'non-value' that means "This key  
exists, but has no useful value!"
        * nil is the current non-value that means 'emptiness', and removes  
any keys that have it as a value.

Calling a vararg function would transfer nil values into undefined  
values in the table:
        function foo( ... )
           for i,v in ipairs( {...} ) do print( i,v,v==nil ) end
        end
        foo( nil, 2, nil )
        --> 1 undefined true
        --> 2 2 false
        --> 3 undefined true

I'm sure my idea is not without it's flaws, but hopefully there is  
some merit in it. In my humble, Lua-newb opinion, the embedded-nil  
topic has been brought up enough to warrant (yet another?) attempt to  
fix it. I understand the clean design that brought Lua to its current  
point, but it _seems_ to me that people need a true solution.

I think the solution has to be:
        a) something like the above, allowing embedded nils; or
        b) internal tracking of the true end of an array with array-specific  
iterators for tables; or
        c) a whole new array type of object that sits alongside tables

The 'solution' cannot be "track the internal size yourself and  
iterate it yourself" specifically because it is impossible in 5.1 to  
pass nil values to vararg functions and know that the user explicitly  
passed an empty value.
Reply | Threaded
Open this post in threaded view
|

RE: Embedded 'nil' values - a suggestion

Couwenberg, Wim

> The 'solution' cannot be "track the internal size yourself
> and iterate it yourself" specifically because it is

That works fine for me...

> impossible in 5.1 to pass nil values to vararg functions and
> know that the user explicitly passed an empty value.

It is possible.  See also recent posts about this on this list:

  http://lua-users.org/lists/lua-l/2006-02/msg00537.html

Specifically the "select("#", ...)" construct.

--
Wim

This message and attachment(s) are intended solely for the use of the addressee and may contain information that is privileged, confidential or otherwise exempt from disclosure under applicable law.  
If you are not the intended recipient or agent thereof responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited.  
If you have received this communication in error, please notify the sender immediately by telephone and with a "reply" message.  
Thank you for your cooperation.


Reply | Threaded
Open this post in threaded view
|

Re: Embedded 'nil' values - a suggestion

Gavin Kistner
On Mar 7, 2006, at 8:33 AM, Couwenberg, Wim wrote:
>> impossible in 5.1 to pass nil values to vararg functions and
>> know that the user explicitly passed an empty value.
>
> It is possible.  See also recent posts about this on this list:
>
>   http://lua-users.org/lists/lua-l/2006-02/msg00537.html
>
> Specifically the "select("#", ...)" construct.

Why thank you! Odd, but workable.

Reply | Threaded
Open this post in threaded view
|

Re: Embedded 'nil' values - a suggestion

Gavin Kistner
In reply to this post by Couwenberg, Wim
On Mar 7, 2006, at 8:33 AM, Couwenberg, Wim wrote:
> It is possible.  See also recent posts about this on this list:
>
>   http://lua-users.org/lists/lua-l/2006-02/msg00537.html
>
> Specifically the "select("#", ...)" construct.

I've added this information to
http://lua-users.org/wiki/MigratingToFiveOne
under "Using vararg expressions rather than 'arg'"

Reply | Threaded
Open this post in threaded view
|

Re: Embedded 'nil' values - a suggestion

Petite Abeille

On Mar 07, 2006, at 17:48, Gavin Kistner wrote:

> I've added this information to
> http://lua-users.org/wiki/MigratingToFiveOne
> under "Using vararg expressions rather than 'arg'"

In addition, you might want to consider using table.maxn() when
unpacking multiple return values:

function test()
     return "a", nil, "b", nil
end

local someResults = { test() }

print( "#", #someResults, unpack( someResults ) )
print( "maxn", table.maxn( someResults ), unpack( someResults, 1,
table.maxn( someResults ) ) )

 > #       1       a
 > maxn    3       a       nil     b

Not perfect as one doesn't seem to be able to catch the very last nil
value, but good enough for all practical purposes :)

Cheers

--
PA, Onnay Equitursay
http://alt.textdrive.com/

Reply | Threaded
Open this post in threaded view
|

Re: Embedded 'nil' values - a suggestion

Wim Couwenberg-2
In reply to this post by Gavin Kistner
>> It is possible.  See also recent posts about this on this list:
>>
>>   http://lua-users.org/lists/lua-l/2006-02/msg00537.html
>>
>> Specifically the "select("#", ...)" construct.

> Why thank you! Odd, but workable.

As another fun example, define the following functions:


local unpack, select = unpack, select

function pack(...)
  local v = {...}
  local n = select("#", ...)

  return function(f, from, to)
    from = from == nil and 1 or
      from < 0 and n + 1 + from or
      from

    to = to == nil and n or
      to < 0 and n + 1 + to or
      to

    return f(v, from, to)
  end
end

function upairs(v, from, to)
  return function(v, i)
    i = i and i + 1 or from
    if i <= to then
      return i, v[i]
    end
  end, v
end


Examples:

> p = pack("aap", "noot", nil)
> =p(unpack)
aap  noot  nil

> =p(unpack, 1, -2)
aap  noot

> =p(unpack, -2)
noot nil

> for i, v in p(upairs) do print(i, v) end
1  aap
2  noot
3  nil

> for i, v in p(upairs, -2) do print(i, v) end
2  noot
3  nil


--
Wim

Reply | Threaded
Open this post in threaded view
|

Re: Embedded 'nil' values - a suggestion

Hugh O'Byrne
In reply to this post by Gavin Kistner
Gavin Kistner wrote:

> One of Lua's greatest assets is the simplicity of the core. No Arrays  
> and Hashes and Queues, we have simply Tables.
>
> The merging of all of these concepts into one atomic type, however,  has
> the collision of embedded nils - tables-as-hashes need a way to  remove
> a key entirely; tables-as-arrays need a way to store an  'empty' value.
>
> For the purposes of this discussion, let's assume that it makes sense  
> to have both nil (nothingness, no value) and false (a value  indicating
> non-truth) in the Lua language. We could go one step  farther, like
> JavaScript, and add a third, similar value:  'undefined'. (That's what
> it's called in JavaScript land; we could  call it whatever we want.)
>
> In JavaScript, this:
>     function foo( a, b ){
>        alert( a + ':' + b )
>     }
>     foo( null )
> reports:
>     "null:undefined"
>
> In JavaScript this:
>     var myHash = { foo:null }
>     alert( myHash.foo + ':' + myHash.bar )
> reports:
>     "null:undefined"
>
> And to remove a key from a hash, you use the delete operator:
>     delete myHash.foo
>
> In JavaScript, null is not equal to false, but it IS equal to  
> undefined. They are two shades of a very similar concept.

This I don't get.  Your demonstrations show there is a difference
between what Java calls 'null' and what it calls 'undefined'.  They are
not the same thing.  What do you mean by saying they are 'equal'?

> My proposal is to use a concept similar to JavaScript (3 'dead'  values)
> but in a slightly different way.
>     * false is a value that means "I am not true!"
>     * undefined (or other name) is a 'non-value' that means "This key  
> exists, but has no useful value!"
>     * nil is the current non-value that means 'emptiness', and removes  
> any keys that have it as a value.
>
> Calling a vararg function would transfer nil values into undefined  
> values in the table:
>     function foo( ... )
>        for i,v in ipairs( {...} ) do print( i,v,v==nil ) end
>     end
>     foo( nil, 2, nil )
>     --> 1    undefined    true
>     --> 2    2    false
>     --> 3    undefined    true

It looks like you have 'nil' going in, and 'undefined' coming out...
being generous, I'll say there is some implicit casting happening here,
but I don't see where or why.  Also, it looks like (undefined == true).
  As previously, how can you say two things are equal and different?

> I'm sure my idea is not without it's flaws, but hopefully there is  some
> merit in it. In my humble, Lua-newb opinion, the embedded-nil  topic has
> been brought up enough to warrant (yet another?) attempt to  fix it. I
> understand the clean design that brought Lua to its current  point, but
> it _seems_ to me that people need a true solution.
>
> I think the solution has to be:
>     a) something like the above, allowing embedded nils; or
>     b) internal tracking of the true end of an array with
> array-specific  iterators for tables; or
>     c) a whole new array type of object that sits alongside tables
>
> The 'solution' cannot be "track the internal size yourself and  iterate
> it yourself" specifically because it is impossible in 5.1 to  pass nil
> values to vararg functions and know that the user explicitly  passed an
> empty value.

Late on the bandwagon...

This is actually the most appealing idea on 'shades of false' I have
seen so far.

I used to be horrified, but I thought about it and now see the sense.
In a nutshell, the idea "nil is the representation of logical-false" was
too deeply ingrained in my head.  Though Lua *had* no *direct* boolean
value representations, it still used boolean values.  The guard on an if
statement is a boolean value, however it may be disguised.  Whatever was
in the place of the guard was (conceptually) automatically and silently
cast to a boolean type, just like "2" in "2"+3 is cast to a numeric
type.  Logical operators aside (they *are* different), the real boolean
value had no existence, except as manifested by the behaviour of if,
while, and the like.  As simple as the mapping, nil -> false, other ->
true, is, it is not an equivalence.  Values take on a new meaning when
used as guards, the meaning logical-false or logical-true.  Now those
meanings have direct representations in Lua.

if (a) then ...

If 'a' is boolean logical-false, then the branch is not executed.
If 'a' is boolean logical-true, then the branch is executed.
If 'a' is not a boolean value, it must be cast to a boolean value, or
the language needs to throw an exception or an error or something.
This third way used to be the *only* working way to use an 'if'
statement.  That was actually not clean.  The clean if-statements are
the first two.  I like the introduction of the boolean value type.

The purist in me dislikes syntactic sugar.  (Tired of typing
parentheses?  Give me a break!  Be a man!)  I love Lua because it has so
little, but I find I'd like it better if it had even less.  That part of
me thinks that if the 'a' above is not a boolean value, perhaps it
should be cast to logical-false.  (Some reasons provided below.)  But,
with legacy considerations, the previous casting (nil -> false, other
(non-boolean!) -> true) has a better chance of winning out in the end.

One aspect of all this that hasn't gotten as much attention as it
deserves is the behaviour of 'not'.  Is 'not' the same as "(value) ~=
true"?  Is it the same as "(value) == false"?  Is it the same as "(cast
value to boolean) ~= true"?  The last one is the only one that makes
sense to me, but I imagine there will be noise on the alternatives.  (It
is equivalent to the first, if all non-boolean values are cast to
logical-false, and is an argument for casting that way.  Casting to
boolean is exactly the expression "(value) == true".  Can't be more
direct than that.)

One more point: I like representing sets in Lua by using the index to
represent an element, and non-nil to represent membership of that
element to the set.  No non-nil value in particular is (or, used to be)
more suited to this task than any other.  Assigning 'true' for
membership is slightly distasteful, because the opposite is assigning
'false', and a representation of non-membership is not as clean (or
symmetrical, with empty table representing empty set) as the absence of
a representation of membership.  A single value-type which represents
'this table entry is not empty, but I don't want to say anything more
about it', sounds like your 'undefined', but in a different meaning.  I
don't know if the two should, or could, be made one.  And if they are
bundled together as one symbol, I don't know if it should be cast as
boolean true or false.

Hugh O'Byrne.