A very basic thing I don't get

classic Classic list List threaded Threaded
51 messages Options
123
Reply | Threaded
Open this post in threaded view
|

A very basic thing I don't get

Thijs Koerselman-2
I feel really stupid for asking this.

local pos = {unpack({1,2,3}), 4}
print("pos size ".. #pos)

I would expect #pos to return 4, but instead its 2. A more real world situation would be to convert a vec3 to vec4 like this: 

        local vec4 = {unpack(vec3), 1}

What the heck am I missing?

Thijs
Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Wesley Smith
For functions with multiple return values like pack, only 1 return
value is kept in a list of operations.  It has confused me in the past
as well.  Here's a more straightforward example:

-- will only return 2 values, not #x+1 values
function test(x)
   return unpack(x), 1
end

Your example with {unpack(x), 1} is the same.  Only the first value
form unpack is kept.  The easiest way to do what you want is:

local vec4 = {unpack(vec3)}
vec4[4] = 1

wes





On Sat, Oct 1, 2011 at 10:25 AM, Thijs Koerselman
<[hidden email]> wrote:
> I feel really stupid for asking this.
> local pos = {unpack({1,2,3}), 4}
> print("pos size ".. #pos)
> I would expect #pos to return 4, but instead its 2. A more real world
> situation would be to convert a vec3 to vec4 like this:
>         local vec4 = {unpack(vec3), 1}
> What the heck am I missing?
> Thijs

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Peter Cawley
In reply to this post by Thijs Koerselman-2
>From §2.5.7 of the Lua 5.1 reference manual, which is about table constructors:
"If the last field in the list has the form exp and the expression is
a function call or a vararg expression, then all values returned by
this expression enter the list consecutively (see §2.5.8)."

The message to take away from this is that things are truncated to
exactly 1 result iff they are not the last thing in an expression
list.

On Sat, Oct 1, 2011 at 6:25 PM, Thijs Koerselman
<[hidden email]> wrote:
> I feel really stupid for asking this.
> local pos = {unpack({1,2,3}), 4}
> print("pos size ".. #pos)
> I would expect #pos to return 4, but instead its 2. A more real world
> situation would be to convert a vec3 to vec4 like this:
>         local vec4 = {unpack(vec3), 1}
> What the heck am I missing?
> Thijs

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

steve donovan
In reply to this post by Thijs Koerselman-2
On Sat, Oct 1, 2011 at 7:25 PM, Thijs Koerselman
<[hidden email]> wrote:
> What the heck am I missing?

The special magic you require only works at the _end_ of a table constructor.

{unpack{1,2},3} ends up being {1,3}, since the extra return value is discarded.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Stefan Reich
In reply to this post by Peter Cawley
On Sat, Oct 1, 2011 at 7:32 PM, Peter Cawley <[hidden email]> wrote:

> The message to take away from this is that things are truncated to
> exactly 1 result iff they are not the last thing in an expression
> list.

Why is that done? Strikes me as rather confusing. (The OP obviously
fell for it too.)

-Stefan

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Michal Kottman
On 1 October 2011 19:43, Stefan Reich
<[hidden email]> wrote:
> On Sat, Oct 1, 2011 at 7:32 PM, Peter Cawley <[hidden email]> wrote:
>
>> The message to take away from this is that things are truncated to
>> exactly 1 result iff they are not the last thing in an expression
>> list.
>
> Why is that done? Strikes me as rather confusing. (The OP obviously
> fell for it too.)

The other way round is also confusing sometimes - some functions
return 'nil, error' in case they fail. This is a common idiom in
places where you do not want to throw an error (like io.open()).

Let's imagine a situation that you want to open several files, and
store them in array:

-- works in standard Lua
openFiles = {io.open('file1'), io.open('file2'), io.open('file3')}
for i=1,3 do
    if openFiles[i] then -- either a file handle or nil
        print(openFiles[i]:read('*a')) -- may fail here if also the
errors were stored in the array
    end
end

If the array constructor worked as you think it should, in case a file
cannot be opened, a {..., nil, string, ...} would be stored in the
array, causing an error in your loop.

Also this would also fail if you used the result of io.open() as a
parameter of a function, i.e.: processFile(io.open('file'), other,
params). In case it failed, 'other' and 'params' would be shifted by
one.

I'm not saying that this is definitely the reason multiple values
currently work, but your way would IMHO cause much more confusion - it
is harder to debug when parameters suddenly shift positions and values
end up in different variables :). When the 'expansion' only happens at
the end, it is "safer", because it cannot overwrite any parameters.

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Stefan Reich
Well... I would generally say that I am not totally happy with the way
arrays and lists in Lua work. It's probably the only thing I would
change right now if I was to change something in the language.

Putting lists and maps in one structure is a little weird. It tends to
favor the maps I'd say. Arrays, technically, are usually things that
take a linear space in memory. Processors understand arrays easily.
However, the Lua structure for arrays is more like a map with
integers. It's a really different thing, and sometimes it shows in its
handling.

I find the relation between nil and lists also confusing.

Java lists are very clear. They have a length, you can put anything in
any position, including nil, and there are no "gaps" anywhere. (Except
if you consider "nil" a gap...)

Either way, it is very simple and straightforward. Lua arrays kinda
keep confusing me. Their size can be different from their length which
I find quite strange to say the least.

Also, I saw a confusing implementation of the length operator in kahlua:

        public final int len() {
                int high = keys.length;
                int low = 0;
                while (low < high) {
                        int middle = (high + low + 1) >> 1;
                        Double key = LuaState.toDouble(middle);
                        Object value = rawget(key);
                        if (value == null) {
                                high = middle - 1;
                        } else {
                                low = middle;
                        }
                }
                return low;
        }

It's obviously a binary search to find the highest key. But does it
even work? If there are nils in the table, it begins to act randomly.
It may, among others, return the key before the lowest nil - or the
highest key in the array.

I have to say: Arrays in Lua are indeed a little confusing.

-Stefan

On Sat, Oct 1, 2011 at 7:55 PM, Michal Kottman <[hidden email]> wrote:
> On 1 October 2011 19:43, Stefan Reich
> <[hidden email]> wrote:
>> Why is that done? Strikes me as rather confusing. (The OP obviously
>> fell for it too.)
>
> The other way round is also confusing sometimes - some functions
> return 'nil, error' in case they fail. This is a common idiom in
> places where you do not want to throw an error (like io.open()).

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Petite Abeille

On Oct 1, 2011, at 8:43 PM, Stefan Reich wrote:

> I have to say: Arrays in Lua are indeed a little confusing.

As heard is Austin, Texas:

Keep Lua weird!


Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Stefan Reich
On Sat, Oct 1, 2011 at 8:49 PM, Petite Abeille <[hidden email]> wrote:
>
> On Oct 1, 2011, at 8:43 PM, Stefan Reich wrote:
>
>> I have to say: Arrays in Lua are indeed a little confusing.
>
> As heard is Austin, Texas:
>
> Keep Lua weird!

What? No... make Lua easy!!

Also, Texas is a weird place and I don't like weird places =)

.Stefan

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

joao lobato
In reply to this post by Stefan Reich
On 10/1/11, Stefan Reich <[hidden email]> wrote:

> It's obviously a binary search to find the highest key. But does it
> even work? If there are nils in the table, it begins to act randomly.
> It may, among others, return the key before the lowest nil - or the
> highest key in the array.

That's why the length operator is undefined for non-sequences.

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Kristofer Karlsson
In reply to this post by Stefan Reich

Also, I saw a confusing implementation of the length operator in kahlua:

       public final int len() {
               int high = keys.length;
               int low = 0;
               while (low < high) {
                       int middle = (high + low + 1) >> 1;
                       Double key = LuaState.toDouble(middle);
                       Object value = rawget(key);
                       if (value == null) {
                               high = middle - 1;
                       } else {
                               low = middle;
                       }
               }
               return low;
       }

It's obviously a binary search to find the highest key. But does it
even work? If there are nils in the table, it begins to act randomly.
It may, among others, return the key before the lowest nil - or the
highest key in the array.


Sorry about that. That is a bug in an old version of Kahlua.
The latest version starts with a binary search and falls back to linear probing if it fails. Sort of like regular Lua.
This is the latest version for Kahlua 1: http://code.google.com/p/kahlua/source/browse/trunk/src/se/krka/kahlua/vm/LuaTableImpl.java
For Kahlua 2, see this the method len in https://github.com/krka/kahlua2/blob/master/core/src/se/krka/kahlua/vm/KahluaUtil.java

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Stefan Reich
In reply to this post by joao lobato
Hm. Well. Undefined is not very cool. Maybe your program happens to
rely on 'undefined' (it might not notice as there is not really an
error message) - and then we try to run it in, say, Kahlua and then it
doesn't work anymore.

Isn't it a lot smarter to have well-defined operators for all common
operations? I mean, we're talking about the length of a list here. Not
something exotic or so. You use that thing every day, all the time.
One nil in there, and the result is undefined? Is that safe?

I don't really think so.

And if it isn't - why don't we join forces and solve that little problem?


On Sat, Oct 1, 2011 at 8:55 PM, joao lobato <[hidden email]> wrote:

> On 10/1/11, Stefan Reich <[hidden email]> wrote:
>
>> It's obviously a binary search to find the highest key. But does it
>> even work? If there are nils in the table, it begins to act randomly.
>> It may, among others, return the key before the lowest nil - or the
>> highest key in the array.
>
> That's why the length operator is undefined for non-sequences.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Petite Abeille

On Oct 1, 2011, at 9:15 PM, Stefan Reich wrote:

> And if it isn't - why don't we join forces and solve that little problem?

Two points:

[1] Herding cats

[2] There is no spoon


[1] http://www.youtube.com/watch?v=Pk7yqlTMvp8
[2] http://www.youtube.com/watch?v=dzm8kTIj_0M

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

David Kastrup
In reply to this post by Stefan Reich
Stefan Reich <[hidden email]> writes:

> On Sat, Oct 1, 2011 at 8:49 PM, Petite Abeille <[hidden email]> wrote:
>>
>> On Oct 1, 2011, at 8:43 PM, Stefan Reich wrote:
>>
>>> I have to say: Arrays in Lua are indeed a little confusing.
>>
>> As heard is Austin, Texas:
>>
>> Keep Lua weird!
>
> What? No... make Lua easy!!

It already is.  It makes it easy to choose the right data structure for
any task: there is only one.

--
David Kastrup


Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Dirk Laurie
In reply to this post by Stefan Reich
On Sat, Oct 01, 2011 at 08:43:02PM +0200, Stefan Reich wrote:
> Well... I would generally say that I am not totally happy with the way
> arrays and lists in Lua work. It's probably the only thing I would
> change right now if I was to change something in the language.
>
...
> I find the relation between nil and lists also confusing.
>
...
> I have to say: Arrays in Lua are indeed a little confusing.
>

Only while you still think in some other language.

Think of it this way:

  1. Arrays and lists are not part of the language.
  2. Lua has only one data structuring mechanism: `table`.  
  3. A table is a map from "keys" to "values".  
  4. All Lua values except `nil` and `NaN` are keys.
  5. Any Lua value can be a "value".
  6. If you have not assigned a value to a key, the value `nil` is
     associated with that key.
  7. Whether a particular nil is actually stored, is an implementation
     detail that does not concern the user.
  8. The function `pairs` iterates over those (key,value) pairs for
     which the value is not nil, in some undefined order.
  9. Since the particular set of keys (1,2,3,...,n) occurs very often in
     applications, Lua offers extra support for those.
     - the length operator `#`
     - the function `ipairs` which iterates over the (key,value) pairs
       in the well-defined order 1,2,3,...
     - the table library
     - the abbreviated table constructor with implicit keys
     It is convenient to call such tables "lists" or "arrays".
 10. Those functions do not actually fail on tables that are not lists,
     but their results are not as useful in those cases.
 10. A set of keys like (1,2,3,5,7,8) is not in any way special, although
     the abbreviated table constructor allows you to define such a set
     with the aid of nil values.
 11. Whether a particular nil is actually stored, is an implementation
     detail that does not concern the user.  (We've said this before,
     but it is worth repeating.)
 12. Whether the implementation of a particular table takes advantage
     of the fact that it is in fact a list, is a detail that does not
     concern the user.
   
Dirk
 

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Mark Hamburg
In reply to this post by Stefan Reich
On Oct 1, 2011, at 12:15 PM, Stefan Reich wrote:

> Hm. Well. Undefined is not very cool. Maybe your program happens to
> rely on 'undefined' (it might not notice as there is not really an
> error message) - and then we try to run it in, say, Kahlua and then it
> doesn't work anymore.
>
> Isn't it a lot smarter to have well-defined operators for all common
> operations? I mean, we're talking about the length of a list here. Not
> something exotic or so. You use that thing every day, all the time.
> One nil in there, and the result is undefined? Is that safe?
>
> I don't really think so.
>
> And if it isn't - why don't we join forces and solve that little problem?

Boundedly undefined behavior is a common choice in processor designs and programming languages. It allows greater latitude in implementation and optimizations. For example, there are plenty of pieces of C that are implementation specific -- e.g., when exactly increment and decrement operators are evaluated relative to the expression in which they occur. The policy is essentially that the language (or processor or...) isn't designed to support particular usage patterns but it won't blow up if those patterns are used.

The # operator is actually quite well defined in that you can count on particular properties of its result. It's just that in an array with holes the value isn't uniquely defined.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Thijs Koerselman-2
In reply to this post by David Kastrup
Thanks for the clear explanations. I feel less stupid now. I'm glad I now understand how it works and more important why it works like that. I'm not going to mingle in the discussion about whether it is good or bad thing. I'll leave that to the pros:)

Cheers,
Thijs
Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Petite Abeille
In reply to this post by Dirk Laurie

On Oct 1, 2011, at 9:52 PM, Dirk Laurie wrote:

>  2. Lua has only one data structuring mechanism: `table`.  

Furthermore, if one is so inclined, one should feel free to build any type of specialized data structure out of the basic building blocks provided by the language. In some circle, this is called "programming" :)


Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Lorenzo Donati-2
In reply to this post by Dirk Laurie
On 01/10/2011 21.52, Dirk Laurie wrote:

> On Sat, Oct 01, 2011 at 08:43:02PM +0200, Stefan Reich wrote:
>> Well... I would generally say that I am not totally happy with the way
>> arrays and lists in Lua work. It's probably the only thing I would
>> change right now if I was to change something in the language.
>>
> ...
>> I find the relation between nil and lists also confusing.
>>
> ...
>> I have to say: Arrays in Lua are indeed a little confusing.
>>
>
> Only while you still think in some other language.
>
> Think of it this way:

Very good summary of an often recurring topic!

Some additions for the archive:

[...]
>    8. The function `pairs` iterates over those (key,value) pairs for
>       which the value is not nil, in some undefined order.
Which is a sort of wrapper for the true iterator function "next"
>    9. Since the particular set of keys (1,2,3,...,n) occurs very often in
>       applications, Lua offers extra support for those.
>       - the length operator `#`
>       - the function `ipairs` which iterates over the (key,value) pairs
>         in the well-defined order 1,2,3,...
>       - the table library
>       - the abbreviated table constructor with implicit keys
>       It is convenient to call such tables "lists" or "arrays".
In Lua 5.2 the new terminology for those ones are "sequences" (see
section 3.4.6 of refman)

>   10. Those functions do not actually fail on tables that are not lists,
>       but their results are not as useful in those cases.
In Lua 5.1 `#' was well defined also for non-sequences, albeit it didn't
return a generally useful value. In 5.2 `#' is defined only for
sequences. Although the old behaviour is still there for non-sequences,
now is to be considered an implementation detail.

Note: as the aforementioned section 3.4.6 states, non-integer keys
doesn't interfere with a table being a sequence, thus the following *is*
a sequence:

{ 10, 20, 30, 'a', 'b', foo = 1, bar = 'hello!'}

(this may come up as unexpected for newbies or for people not reading
the manual carefully, since the meaning of the term "sequence" in Lua is
slightly different from the usual math meaning).

[...]

> Dirk
>
>
>
>

Cheers.
-- Lorenzo

Reply | Threaded
Open this post in threaded view
|

Re: A very basic thing I don't get

Lorenzo Donati-2
In reply to this post by Petite Abeille
On 01/10/2011 22.25, Petite Abeille wrote:
>
> On Oct 1, 2011, at 9:52 PM, Dirk Laurie wrote:
>
>>   2. Lua has only one data structuring mechanism: `table`.
>
> Furthermore, if one is so inclined, one should feel free to build any type of
 >specialized data structure out of the basic
 >building blocks provided by the language. [...]

Very, true.

This is IMO a big newbie gotcha: Lua tables are so "high-level", i.e.
they can be easily used to implement lists, trees, maps, stacks, etc.,
that one can easily think they should provide a much fatter interface
(the common problem of getting into Lua's spirit).

It's not easy to grasp initially that one can use OOP techniques to turn
a generic table into, say, a "stack object", with usual push/pop methods.


>
>

Cheers.
-- Lorenzo

123