lists with nil play nice for Lua 5.2

classic Classic list List threaded Threaded
63 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

lists with nil play nice for Lua 5.2

John Belmonte
Recently I began using Lua again in a real product, my last serious use
being at the time of version 4.0.  From the perspective of 4.0 to 5.1,
lists have gone from tolerable to bad with regard to nil element
support.  As is well-documented and discussed frequently, a nil will
foil any of the following:

    #t                 -- non-deterministic
    unpack(t)          --       ""
    ipairs(t)          --       ""

Common ways in which such lists are created include:

    { 2, nil, 'bar' }  -- literal list with nil
    { f() }            -- capture function return vals which include nil
    { ... }            -- capture function/chunk args which include nil

Programming in Lua refers to nil in a list as a "hole", but that is from
the perspective of Lua's table representation which cannot distinguish
nil-valued from nonexistent keys.  From the view of the list creator,
why should either of the following be considered as having holes?

    x = { 2, nil, 'bar' }
    t = { 2 }; table.insert(t, nil); table.insert(t, 'bar')

Clearly they are not sparse as constructed.  Stepping back, a list is
generally defined a series of values.  Lua nil is a first class value--
an incredibly useful one-- which should not render lists practically
useless.

I don't have a proposal for solving this in some efficient and
backwards-compatible way, only a bare request: make operations on lists
with nil work out of the box in the obvious way.  The current behavior
is a trap for the Lua novice and sends the expert straight to his bag of
tricks for a workaround.  The experts come up with different solutions,
causing interoperability problems between pieces of code.  This
situation isn't acceptable for a data type as fundamental as a list.

Like most in the community, I will have my pet list of small changes I'd
like to see in 5.2, but honestly if this was the only issue addressed I
would be happy.

Regards,
--John


Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Ketmar Dark-2
On Sat, 28 Jul 2007 16:38:39 -0400
John Belmonte <[hidden email]> wrote:

> Common ways in which such lists are created include:
> 
>     { 2, nil, 'bar' }  -- literal list with nil
>     { f() }            -- capture function return vals which include
> nil { ... }            -- capture function/chunk args which include
> nil
a strange way, indeed. why not change `nil' to `false'? Lua is
weak-typed, so one can use some unique value (string, for example) to
mark `unidefined, but present' values.

Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

David Given
In reply to this post by John Belmonte
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Belmonte wrote:
[...]
> Clearly they are not sparse as constructed.  Stepping back, a list is
> generally defined a series of values.  Lua nil is a first class value--
> an incredibly useful one-- which should not render lists practically
> useless.

The thing is, though, a Lua table is not a list. A table is a dictionary, a
collection of key->value pairs. What you're referring to as a list is merely
an Lua array: a table with a contiguous set of integer keys starting at 1.

The reason why the documentation refers to nil values as 'holes' is because an
*unset* item in the dictionary is considered to have a value of nil. You
remove a mapping by setting its value to nil. So if you try to define a table
with a contiguous set of integer keys, etc, with one value as nil, then you've
contradicted yourself: the keys are no longer contiguous because one item is
unset. Since all the array operators are defined to *only* work on tables that
are valid arrays, then of course they'll fail.

I realise it's not an awful lot of help to you, but I think the key to your
problem is that the abstraction you've based your code design around, the
list, is not the same one that Lua is providing. So of course you're going to
run into problems.

What is it exactly you're trying to achieve?

- --
âââ ïïïïïïïïïïïïïï âââ http://www.cowlark.com âââââââââââââââââââ
â
â "There does not now, nor will there ever, exist a programming language in
â which it is the least bit hard to write bad programs." --- Flon's Axiom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGq7gXf9E0noFvlzgRAhvVAJ9aQhwbJz54zF9h9oqzhhyzAB3SswCfXM7E
BaEYRG2e0vLxJORJX2Ix4qM=
=lS16
-----END PGP SIGNATURE-----



Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Gavin Wraith
In reply to this post by John Belmonte
In message <46ABA94F.9030905@...> you wrote:

> Programming in Lua refers to nil in a list as a "hole", but that is from
> the perspective of Lua's table representation which cannot distinguish
> nil-valued from nonexistent keys.  From the view of the list creator,
> why should either of the following be considered as having holes?
>
>     x = { 2, nil, 'bar' }
>     t = { 2 }; table.insert(t, nil); table.insert(t, 'bar')
>
> Clearly they are not sparse as constructed.  Stepping back, a list is
> generally defined a series of values.  Lua nil is a first class value--
> an incredibly useful one-- which should not render lists practically
> useless.

There is a long history of confusion, even in the purely theoretical
considerations of the semantics of type theories, about bottom
elements denoting undefined data, exceptions etc. The difficulty seems
to be that nil is being stretched to rather a lot of uses. The
introduction of boolean values in Lua freed nil from having to represent
false. Is it not conceivable that the user may need lots of different
kinds of nil? For example

nil (= undefined),
nil (= you can go ahead and garbage collect),
nil (= error condition),
nil (= please keep this slot ready till I get round to filling it)

might in some circumstance need to be discriminated, perhaps?

-- 
Gavin Wraith ([hidden email])
Home page: http://www.wra1th.plus.com/

Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

D Burgess-4
In reply to this post by John Belmonte
John Belmonte wrote:
> ..  Lua nil is a first class value--
> an incredibly useful one-- which should not render lists practically
> useless. ..

I agree with point that John makes. I also agree that using false
instead of nil goes someway to resolving the issue. I have even
used lightuserdata (char*)NULL to represent non-existence in
tables. However, I think Lua should move to providing a length
(#) operator that always "works" for arrays and iteration that always
"works" for arrays (sparse or otherwise). Is is tough to explain
the current behaviour to the novice Lua user.

db

Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

David Given
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

David Burgess wrote:
[...]
> I agree with point that John makes. I also agree that using false
> instead of nil goes someway to resolving the issue. I have even
> used lightuserdata (char*)NULL to represent non-existence in
> tables. However, I think Lua should move to providing a length
> (#) operator that always "works" for arrays and iteration that always
> "works" for arrays (sparse or otherwise). Is is tough to explain
> the current behaviour to the novice Lua user.

This is rather against the Lua Way, which is to provide the fundamental tools
necessary to achieve the functionality, but leave it up to the user to use the
tools to achieve the desired goal.

table as it stands provides an efficient associative map that's also efficient
for the special case when you're using the table as an array. That doesn't
make it any less of a map, though, so you still have be aware of the
semantics. Change the way table works and you end up changing the core feature
of the language --- very dangerous.

If you *really* want a true array, wrap it in a structure that keeps manual
track of the length, or just do 'unset = {}' and use 'unset' as a placeholder.

It's all very well saying this behaviour is confusing to new users, but
efficient use of tables is an absolutely key aspect to writing Lua programs.
It's something that just has to be learnt. (See signature!)

(Although I will admit that this is far too easy to get it wrong, and break an
array and have to suffer the weird bugs you get that way; but rather than
change the language, I'd much rather have debugging tools to let you know when
you get it wrong. A strict mode, for example, that makes the array-aware tools
check to make sure that the table they're working on is a valid array...

- --
âââ ïïïïïïïïïïïïïï âââ http://www.cowlark.com âââââââââââââââââââ
â
â "There does not now, nor will there ever, exist a programming language in
â which it is the least bit hard to write bad programs." --- Flon's Axiom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGq+IRf9E0noFvlzgRAmN1AKCIJt8Jqw/0+3u3ZE3j2KZYkaEzigCgtUck
r9y1upe5pQKZQ4CwLSiShLg=
=+eoQ
-----END PGP SIGNATURE-----



Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

David Manura
In reply to this post by John Belmonte
John Belmonte <john <at> neggie.net> writes:
>     { f() }            -- capture function return vals which include nil

The problem with the construct { f() }, though appealing in tersity, is that it
loses information (trailing nils), at least if the number of trailing nils is
variable.  We can overcome that by save(f()), where save is defined as "function
save(...) return {n=select('#', ...), ...} end", which is correct but involves
needless memory copying.  This page was recently updated on this concern:
http://lua-users.org/wiki/StoringNilsInTables .  

A similar concern is how "x and y or z" works as a condition -except- when y is
nil. -- http://lua-users.org/wiki/TernaryOperator

The more fundamental problem I think is the restricted ability in Lua to
manipulate lists (varargs "..." and multiple return values) for functional
programming.  The typical workaround, as above, is to stuff it in a table, which
involves an extra memory allocation and leads to the above concerns, as well as
the additional concern about holes in the table, which the Lua standard library
doesn't handle.  We can avoid the table with tricks using additional function
and tail calls, but it is awkward and not necessarily efficient.  C functions
have more flexibility here because they can manipulate the stack on which these
lists are stored, but pure Lua functions don't have that kind of access. 
Ideally, we would want a way for the quite common task of manipulating stack
lists to be written in a way that is simple, efficient and correct (without
trade-off).

The "Proxy Table of Local Variables, _L" example in
http://lua-users.org/wiki/LuaHacks maybe can be adapted here to provide a table
interface, with "n" field, in C that acts as a proxy to a list on the Lua stack
(rather than as a copy into a table as I believe is the case for "arg"), but we
still have the issue with handling of holes by the Lua standard library.


Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Thomas Lauer-3
In reply to this post by John Belmonte
John Belmonte <[hidden email]> wrote:

> Programming in Lua refers to nil in a list as a "hole", but that is from
> the perspective of Lua's table representation which cannot distinguish
> nil-valued from nonexistent keys.  From the view of the list creator,
> why should either of the following be considered as having holes?
> 
>     x = { 2, nil, 'bar' }
>     t = { 2 }; table.insert(t, nil); table.insert(t, 'bar')
> 
> Clearly they are not sparse as constructed.  Stepping back, a list is
> generally defined a series of values.  Lua nil is a first class value--
> an incredibly useful one-- which should not render lists practically
> useless.

I agree. nil as a perfectly sensible value in its own right as compared
to nil as meaning undefined is a certain source of confusion.

However, as others have remarked, I am not sure whether there's a
elegant solution that's also backward compatible.

-- 
cheers  thomasl

web : http://thomaslauer.com/start


Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

John Belmonte
In reply to this post by John Belmonte
Following are further notes on this.

The list implementation uses nil as an end sentinel.  This value works
out nicely since it is the value of a nonexistent table key, and hence
the end sentinel need not be explicitly managed.  For example, when
appending to list { 'a', 'b' }, we need only set index 3 to the new
value, rather than having to both set the value to index 3 and set the
sentinel to index 4.

The issue is that some people (at least me) feel that nil is too useful
a value within lists to be relegated to the job of sentinel.  Lists are
commonly used to capture function input and output value sequences,
deterring use of nil in those cases as well.

At the very least, this being Lua, there should be a straightforward way
for users to implement support for nil values in lists, if that is
desired.  However, the list constructor syntax (e.g. {2, nil, 'bar',
nil}) drops the sequence length information from the start, so support
there would at least require a core change.  A less attractive approach
is to give up the native constructor syntax, and use something like
Array(2, nil, 'bar', nil), since an accurate vararg length is available
via select.  However, besides being less efficient, a plain table could
not be used in the implementation, since that type does not support the
len metamethod.  Using of another type (e.g. userdata, function closure)
would necessarily require C coding.

Prior to Lua version 5.0, nil values in a list were supported via the
"n" field, an optional length field honored by length and iteration
operations, and set by the core in the case of a vararg list.


Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Tony Finch
In reply to this post by Gavin Wraith
On Sat, 28 Jul 2007, Gavin Wraith wrote:
>
> There is a long history of confusion, even in the purely theoretical
> considerations of the semantics of type theories, about bottom
> elements denoting undefined data, exceptions etc. The difficulty seems
> to be that nil is being stretched to rather a lot of uses.

It's worth comparing Lua with Javascript in this area. JS has both
undefined (which is similar to Lua's nil since it is the vaue used for
unset variables and elements) and null (which is like null in Java: it
only applies to objects not native types - typeof(null) == "object"). This
was somewhat confusing to me when I first learned JS, since I was
expecting something more like Lua's semantics. (I learned Lua after JS and
I think it's cleaner in almost every way.)

Tony.
-- 
f.a.n.finch  <[hidden email]>  http://dotat.at/
IRISH SEA: SOUTHERLY, BACKING NORTHEASTERLY FOR A TIME, 3 OR 4. SLIGHT OR
MODERATE. SHOWERS. MODERATE OR GOOD, OCCASIONALLY POOR.

Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Asko Kauppi
In reply to this post by David Given

I agree with John about the annoyance of this non-intuitive nil handling. And I do my set of tricks around it, as John pointed many of us do...

As a solution, how about just making #tbl etc. _fail_ if they are applied on a table that somehow is known to have holes in it. Then again, if that can be efficiently found out it wouldn't take much more to fallback to a less efficient hopping-over-holes algorithm for those failing cases. The annoyance is the unexpected behaviour, this certainly is one of Lua's "traps" (which every language has some).

To David:

Fixing the issue with #, unpack() etc. would not in my opinion hurt the general use of tables, in any way. It might cause these operators to run a bit slower, though (which is the real problem as far as I know?).

-asko


David Given kirjoitti 29.7.2007 kello 0:41:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Belmonte wrote:
[...]
Clearly they are not sparse as constructed.  Stepping back, a list is
generally defined a series of values. Lua nil is a first class value--
an incredibly useful one-- which should not render lists practically
useless.

The thing is, though, a Lua table is not a list. A table is a dictionary, a collection of key->value pairs. What you're referring to as a list is merely an Lua array: a table with a contiguous set of integer keys starting at 1.

The reason why the documentation refers to nil values as 'holes' is because an *unset* item in the dictionary is considered to have a value of nil. You remove a mapping by setting its value to nil. So if you try to define a table with a contiguous set of integer keys, etc, with one value as nil, then you've contradicted yourself: the keys are no longer contiguous because one item is unset. Since all the array operators are defined to *only* work on tables that
are valid arrays, then of course they'll fail.

I realise it's not an awful lot of help to you, but I think the key to your problem is that the abstraction you've based your code design around, the list, is not the same one that Lua is providing. So of course you're going to
run into problems.

What is it exactly you're trying to achieve?

- --
âââ ïïïïïïïïïïïïïï âââ http://www.cowlark.com âââââââââââââââââââ
â
â "There does not now, nor will there ever, exist a programming language in â which it is the least bit hard to write bad programs." --- Flon's Axiom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGq7gXf9E0noFvlzgRAhvVAJ9aQhwbJz54zF9h9oqzhhyzAB3SswCfXM7E
BaEYRG2e0vLxJORJX2Ix4qM=
=lS16
-----END PGP SIGNATURE-----




Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Norman Ramsey-2
Though I shouldn't, I can't resist weighing in on this thread.
The purpose of nill is to be a unique value that cannot appear 
in any table or list.  If you want a unique value for some other purpose.
Lua provides you a trivial way to make one:

  hole = {}

  l = {1, hole, 3, 'beeblebrox'}

Of course, you must do the extra work of explaining and documenting
this value.  Perhaps Lua 5.2 can address the policy disorder by
providing a value 'table.hole' whose sole purpose is to be distinct
from every other value (including nil) :-)



Norman


Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

David Given
In reply to this post by Asko Kauppi
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Asko Kauppi wrote:
> I agree with John about the annoyance of this non-intuitive nil
> handling. And I do my set of tricks around it, as John pointed many of
> us do...

I don't like it, either. I'm just not sure this can be easily fixed without
changing a fundamental core semantic.

Having the array operators fail when applied to invalid tables is certainly
one way around this, but I'm not sure if it can be done efficiently ---
wouldn't checking a table for valid array-ness require looking at every
numeric element in the table? This is potentially useful in a hypothetical
strict mode (I'd certainly use it!), but I wouldn't want it all the time. (I
can also think of a fairly straightforward approach for efficiently ensuring
that #t on a hole-y array becomes deterministic and is always the index
immediately preceding the first unused slot, which is another approach.)

This also doesn't help John's original problem in that he wants to be able to
use nil as a valid element in an array. I'm not sure this can be achieved
without changing the behaviour of table in a non-backward-compatible way.
Given an array {[1]=1, [2]=2, [3]=nil, [4]=4}, then does ipairs() visit
element 3 or not? If it *does*, it's violating table behaviour; if it
*doesn't*, it's violating (the desired) array behaviour.

Given that it's fairly trivial to construct an efficient Array 'class' with
specific array semantics, I'd be rather inclined not to risk changing anything:

function Array(...)
	local data = {...}
	local length = select(..., "#")
	local o = {
		...accessor functions here...
	}
	setmetable(o, o)
	return o
end

That way you get a nice OO interface that conforms to ideal semantics for
arrays, and you're not forced into backwards compatibility with tables.
Alternatively, simply don't use nils to mean 'unset' --- it's trivial to
create a value specifically for doing this ('unset = {}'). After all, if it
hurts when you do this, don't do this...

- --
âââ ïïïïïïïïïïïïïï âââ http://www.cowlark.com âââââââââââââââââââ
â
â "There does not now, nor will there ever, exist a programming language in
â which it is the least bit hard to write bad programs." --- Flon's Axiom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGrPTIf9E0noFvlzgRAsTXAJ95W8JVOkctlm/Vn5QPbbK659evSACgsn6r
G70NG4EnG2OgpWAKHYl5jSc=
=750M
-----END PGP SIGNATURE-----



Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Wesley Smith
You kow you could always make your own table, unpack, etc.
functionality and replace the existing Lua ones and your scripts would
work as expected without having to change a thing in Lua or wait for
some hypothetical future version.

wes




On 7/29/07, David Given <[hidden email]> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Asko Kauppi wrote:
> > I agree with John about the annoyance of this non-intuitive nil
> > handling. And I do my set of tricks around it, as John pointed many of
> > us do...
>
> I don't like it, either. I'm just not sure this can be easily fixed without
> changing a fundamental core semantic.
>
> Having the array operators fail when applied to invalid tables is certainly
> one way around this, but I'm not sure if it can be done efficiently ---
> wouldn't checking a table for valid array-ness require looking at every
> numeric element in the table? This is potentially useful in a hypothetical
> strict mode (I'd certainly use it!), but I wouldn't want it all the time. (I
> can also think of a fairly straightforward approach for efficiently ensuring
> that #t on a hole-y array becomes deterministic and is always the index
> immediately preceding the first unused slot, which is another approach.)
>
> This also doesn't help John's original problem in that he wants to be able to
> use nil as a valid element in an array. I'm not sure this can be achieved
> without changing the behaviour of table in a non-backward-compatible way.
> Given an array {[1]=1, [2]=2, [3]=nil, [4]=4}, then does ipairs() visit
> element 3 or not? If it *does*, it's violating table behaviour; if it
> *doesn't*, it's violating (the desired) array behaviour.
>
> Given that it's fairly trivial to construct an efficient Array 'class' with
> specific array semantics, I'd be rather inclined not to risk changing anything:
>
> function Array(...)
>         local data = {...}
>         local length = select(..., "#")
>         local o = {
>                 ...accessor functions here...
>         }
>         setmetable(o, o)
>         return o
> end
>
> That way you get a nice OO interface that conforms to ideal semantics for
> arrays, and you're not forced into backwards compatibility with tables.
> Alternatively, simply don't use nils to mean 'unset' --- it's trivial to
> create a value specifically for doing this ('unset = {}'). After all, if it
> hurts when you do this, don't do this...
>
> - --
> ┌── dg@cowlark.com ─── http://www.cowlark.com ───────────────────
> │
> │ "There does not now, nor will there ever, exist a programming language in
> │ which it is the least bit hard to write bad programs." --- Flon's Axiom
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFGrPTIf9E0noFvlzgRAsTXAJ95W8JVOkctlm/Vn5QPbbK659evSACgsn6r
> G70NG4EnG2OgpWAKHYl5jSc=
> =750M
> -----END PGP SIGNATURE-----
>
>

Reply | Threaded
Open this post in threaded view
|

RE: lists with nil play nice for Lua 5.2

Jerome Vuarand-2
In reply to this post by John Belmonte
John Belmonte wrote:
> Programming in Lua refers to nil in a list as a "hole", but that is
> from the perspective of Lua's table representation which cannot
> distinguish nil-valued from nonexistent keys.  From the view of the
> list creator, why should either of the following be considered as
> having holes?    

I think part of the problem comes from the fact that some people
distinguish nil-valued from non-existent keys. But the concept of a
non-existent key is not applicable to Lua tables. In Lua a table is an
infinite associative array, with all possible keys having a value. By
default all keys have a nil value. These nils are not explicitly stored,
but that's an implementation detail.

In that context the # operator is not well defined. I see two good
(though not perfect) solutions: either make the table # operator
deterministic (for example return the biggest integer key), or allow
users to override the # operator (that's my prefered solution, since it
would be useful for other problems, like returning the norm of a
vector).


Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Wesley Smith
The # operator could be overridden with a token filter I beleive.
wes

On 7/30/07, Jerome Vuarand <[hidden email]> wrote:
> John Belmonte wrote:
> > Programming in Lua refers to nil in a list as a "hole", but that is
> > from the perspective of Lua's table representation which cannot
> > distinguish nil-valued from nonexistent keys.  From the view of the
> > list creator, why should either of the following be considered as
> > having holes?
>
> I think part of the problem comes from the fact that some people
> distinguish nil-valued from non-existent keys. But the concept of a
> non-existent key is not applicable to Lua tables. In Lua a table is an
> infinite associative array, with all possible keys having a value. By
> default all keys have a nil value. These nils are not explicitly stored,
> but that's an implementation detail.
>
> In that context the # operator is not well defined. I see two good
> (though not perfect) solutions: either make the table # operator
> deterministic (for example return the biggest integer key), or allow
> users to override the # operator (that's my prefered solution, since it
> would be useful for other problems, like returning the norm of a
> vector).
>

Reply | Threaded
Open this post in threaded view
|

RE: lists with nil play nice for Lua 5.2

Patrick Donnelly-2
In reply to this post by John Belmonte

> In that context the # operator is not well defined. I see two good
> (though not perfect) solutions: either make the table # operator
> deterministic (for example return the biggest integer key), or allow
> users to override the # operator (that's my prefered solution, since it
> would be useful for other problems, like returning the norm of a
> vector).

http://www.lua.org/manual/5.1/manual.html#2.8

__len??

-Patrick Donnelly

"One of the lessons of history is that nothing is often a good thing to do and always a clever thing to say."

-Will Durant
Reply | Threaded
Open this post in threaded view
|

RE: lists with nil play nice for Lua 5.2

Jerome Vuarand-2
Patrick Donnelly  wrote: 
> > In that context the # operator is not well defined. I see two good
> > (though not perfect) solutions: either make the table # operator
> > deterministic (for example return the biggest integer key), or allow
> > users to override the # operator (that's my prefered solution, since
it
> > would be useful for other problems, like returning the norm of a
> > vector).
> 
> http://www.lua.org/manual/5.1/manual.html#2.8
> 
> __len??

I thought so initially, but it's not overridable for tables :-)


Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Asko Kauppi
In reply to this post by Wesley Smith

Well...

anything can be overridden with a token filter. Except for 'continue', that is.


Wesley Smith kirjoitti 30.7.2007 kello 18:15:

The # operator could be overridden with a token filter I beleive.
wes

On 7/30/07, Jerome Vuarand <[hidden email]> wrote:
John Belmonte wrote:
Programming in Lua refers to nil in a list as a "hole", but that is
from the perspective of Lua's table representation which cannot
distinguish nil-valued from nonexistent keys.  From the view of the
list creator, why should either of the following be considered as
having holes?

I think part of the problem comes from the fact that some people
distinguish nil-valued from non-existent keys. But the concept of a
non-existent key is not applicable to Lua tables. In Lua a table is an
infinite associative array, with all possible keys having a value. By
default all keys have a nil value. These nils are not explicitly stored,
but that's an implementation detail.

In that context the # operator is not well defined. I see two good
(though not perfect) solutions: either make the table # operator
deterministic (for example return the biggest integer key), or allow
users to override the # operator (that's my prefered solution, since it
would be useful for other problems, like returning the norm of a
vector).



Reply | Threaded
Open this post in threaded view
|

Re: lists with nil play nice for Lua 5.2

Irayo
In reply to this post by Norman Ramsey-2
Norman Ramsey wrote:
Perhaps Lua 5.2 can address the policy disorder by
providing a value 'table.hole' whose sole purpose is to be distinct
from every other value (including nil) :-)

This makes the most sense to me out of any of the other solutions.

Nil in tables is not "nothingness"--it means, technically, that the key is non-existent. So, asking Lua if a specific table element is nil is, in essence, asking if that element is non-existent. Setting a table element to nil specifically is just *telling* Lua that the element doesn't exist. Lua doesn't handle non-existent elements because they *don't exist*. So you can't expect to be able to tell Lua that an element doesn't exist and then have it know that you meant that the element existed, but was blank.

People use the value 'nil' in such a way as to say "nothing". 'blah = nothing'. But if you're writing that with a nil, you're telling Lua that blah doesn't exist. A "hole" or "blank" value, as Norman suggested, addresses this by being able to differentiate between

t[1] = nil;  -- a nonexistant element, and
t[2] = table.hole; -- an element that exists, but has no properly defined value

#t = 2, because 'hole' is has a value. Of course, I rather think we could come up with a more friendly name than "hole"--like maybe "blank". I dunno.

Users can continue using nil if they want, and use all the work-arounds to have iterators work in sparse tables, which means it's fully backward-compatible. Nothing changes except the addition of another element to the table library (as far as I know). Other users can properly use new functions.

It may also be useful to define a function in such a way that f(a) is "(a==nil) or (a==table.hole)", so that you can check whether a value is nil *or* a hole.

As always, I'm not a Lua expert and I don't know what most people in the community are looking for; this just seems to be a very simple and elegant solution to a long-lasting problem. Feel free to get all up in my grill if you disagree--I want to see what others think.

--
Irayo


1234