Lua, LuaJIT2 and differences with the length operator

classic Classic list List threaded Threaded
36 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Lua, LuaJIT2 and differences with the length operator

Daniele Alessandri
Hi everyone,

I just noticed a subtle difference between using the length operator
on tables containing nil values on Lua and LuaJIT2 and I would like to
get a clarification:

Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> t = { 'foo', nil, 'bar' }
> print(#t)
3

LuaJIT 2.0.0-beta5 -- Copyright (C) 2005-2010 Mike Pall. http://luajit.org/
JIT: OFF CMOV SSE2 SSE4.1 fold cse dce fwd dse narrow loop abc fuse
> t = { 'foo', nil, 'bar' }
> print(#t)
1

Which one is right? I guess Lua, even if
http://www.lua.org/manual/5.1/manual.html#2.5.5 states that "If the
array has holes (that is, nil values between other non-nil values),
then #t can be any of the indices that directly precedes a nil value
(that is, it may consider any such nil value as the end of the
array).".

I would expect at least the same behavior, or am I just doing it wrong
by trying to rely on the current one showed by Lua 5.1?

Thanks,
Daniele

--
Daniele Alessandri
http://clorophilla.net/
http://twitter.com/JoL1hAHN

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Peter Cawley
On Wed, Dec 29, 2010 at 3:18 PM, Daniele Alessandri
<[hidden email]> wrote:
> Which one is right? I guess Lua, even if
> http://www.lua.org/manual/5.1/manual.html#2.5.5 states that "If the
> array has holes (that is, nil values between other non-nil values),
> then #t can be any of the indices that directly precedes a nil value
> (that is, it may consider any such nil value as the end of the
> array).".
>
> I would expect at least the same behavior, or am I just doing it wrong
> by trying to rely on the current one showed by Lua 5.1?

They are both right. As the manual states, in cases like these, there
are multiple "right" lengths, and implementations are free to give you
any of the "right" lengths. Hence relying on all implementations
giving the same result is bad practice, as this isn't guaranteed.

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Luiz Henrique de Figueiredo
In reply to this post by Daniele Alessandri
> I just noticed a subtle difference between using the length operator
> on tables containing nil values on Lua and LuaJIT2 and I would like to
> get a clarification:

Let's not get into this discussion again: #t does not return anything useful
if t is "an array with holes".

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Duncan Cross
In reply to this post by Daniele Alessandri
On Wed, Dec 29, 2010 at 3:18 PM, Daniele Alessandri
<[hidden email]> wrote:

> Hi everyone,
>
> I just noticed a subtle difference between using the length operator
> on tables containing nil values on Lua and LuaJIT2 and I would like to
> get a clarification:
>
> Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
>> t = { 'foo', nil, 'bar' }
>> print(#t)
> 3
>
> LuaJIT 2.0.0-beta5 -- Copyright (C) 2005-2010 Mike Pall. http://luajit.org/
> JIT: OFF CMOV SSE2 SSE4.1 fold cse dce fwd dse narrow loop abc fuse
>> t = { 'foo', nil, 'bar' }
>> print(#t)
> 1
>
> Which one is right? I guess Lua, even if
> http://www.lua.org/manual/5.1/manual.html#2.5.5 states that "If the
> array has holes (that is, nil values between other non-nil values),
> then #t can be any of the indices that directly precedes a nil value
> (that is, it may consider any such nil value as the end of the
> array).".
>
> I would expect at least the same behavior, or am I just doing it wrong
> by trying to rely on the current one showed by Lua 5.1?

You are doing it wrong by trying to rely on that. As the manual
states, "#t can be any of the indices" - *any* of them, it is
deliberately undefined which. You should treat it as if it is random,
even though it may seem to be consistent.

In fact, it is not even consistent in Lua 5.1. For example, these two
ways of populating a table should get us identical tables but they get
different length results (at least, they do for me, it might even vary
from machine to machine):

> t1 = { 'foo', nil, 'bar' }
> print(#t1)
3
> t2 = {}
> t2[1] = 'foo'
> t2[2] = nil
> t2[3] = 'bar'
> print(#t2)
1

-Duncan

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Daniele Alessandri
In reply to this post by Luiz Henrique de Figueiredo
On Wed, Dec 29, 2010 at 16:31, Luiz Henrique de Figueiredo
<[hidden email]> wrote:

> Let's not get into this discussion again: #t does not return anything useful
> if t is "an array with holes".

Yeah I know, that's exactly why I've pointed out that link from the
docs, it's just that I indirectly hit that difference by doing an
unpack on a table with (actually unexpected) holes and unpack
automatically uses #t to get the length of the table and was surprised
that Lua and LuaJIT were behaving differently.

--
Daniele Alessandri
http://clorophilla.net/
http://twitter.com/JoL1hAHN

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Renato Maia
In reply to this post by Daniele Alessandri

On 29 Dec 2010, at 13:18, Daniele Alessandri wrote:
> Which one is right?

Both.

> I would expect at least the same behavior, or am I just doing it wrong
> by trying to rely on the current one showed by Lua 5.1?


You should rely on what the manual says, not on the observable  
behavior of a specific implementation of Lua (even if it is the  
official one).

--
Renato Maia
Computer Scientist
Tecgraf/PUC-Rio
__________________________
http://www.inf.puc-rio.br/~maia/


Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Daniele Alessandri
In reply to this post by Duncan Cross
On Wed, Dec 29, 2010 at 16:39, Duncan Cross <[hidden email]> wrote:

> You are doing it wrong by trying to rely on that. As the manual
> states, "#t can be any of the indices" - *any* of them, it is
> deliberately undefined which. You should treat it as if it is random,
> even though it may seem to be consistent.
>
> In fact, it is not even consistent in Lua 5.1. For example, these two

OK, this confirms to me that a piece of my code was working on Lua 5.1
by pure coincidence :-)
I'll handle it differently instead of just passing the user-provided
table to unpack (see my other reply to Luiz).

Thanks!

--
Daniele Alessandri
http://clorophilla.net/
http://twitter.com/JoL1hAHN

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Matthew Wild
On 29 December 2010 16:07, Daniele Alessandri <[hidden email]> wrote:

> On Wed, Dec 29, 2010 at 16:39, Duncan Cross <[hidden email]> wrote:
>
>> You are doing it wrong by trying to rely on that. As the manual
>> states, "#t can be any of the indices" - *any* of them, it is
>> deliberately undefined which. You should treat it as if it is random,
>> even though it may seem to be consistent.
>>
>> In fact, it is not even consistent in Lua 5.1. For example, these two
>
> OK, this confirms to me that a piece of my code was working on Lua 5.1
> by pure coincidence :-)
> I'll handle it differently instead of just passing the user-provided
> table to unpack (see my other reply to Luiz).
>

A common idiom (blessed by an official table.pack() in 5.2):

function pack(...)
    return { n = select("#", ...), ... }; -- Save the number of items in 'n'
end

-- Usage:
mytable = pack(1, nil, 3);

print(unpack(mytable, 1, mytable.n)) -- Use number of items instead of '#'

Regards,
Matthew

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Leo Razoumov
In reply to this post by Luiz Henrique de Figueiredo
On Wed, Dec 29, 2010 at 10:31, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
> Let's not get into this discussion again: #t does not return anything useful
> if t is "an array with holes".

There is a good reason why this question persistently comes up again and again.
Lua provides no good way to test whether a given array has holes or
not. Most of the time one relies upon leap of faith to trust #t
values. In that regard Lua quite successfully bridged a gap between
Coding and Religion:-)

--Leo--

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Henning Diedrich
On 1/4/11 2:18 PM, Leo Razoumov wrote:
On Wed, Dec 29, 2010 at 10:31, Luiz Henrique de Figueiredo
[hidden email] wrote:
Let's not get into this discussion again: #t does not return anything useful
if t is "an array with holes".
There is a good reason why this question persistently comes up again and again.
Lua provides no good way to test whether a given array has holes or
not. Most of the time one relies upon leap of faith to trust #t
values. In that regard Lua quite successfully bridged a gap between
Coding and Religion:-)

--Leo-- 

Wow. You know what will happen now, yes?

*leaning back, waiting*

Henning

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

KHMan
In reply to this post by Leo Razoumov
On 1/4/2011 9:18 PM, Leo Razoumov wrote:
> On Wed, Dec 29, 2010 at 10:31, Luiz Henrique de Figueiredo wrote:
>> Let's not get into this discussion again: #t does not return anything useful
>> if t is "an array with holes".
>
> There is a good reason why this question persistently comes up again and again.
> Lua provides no good way to test whether a given array has holes or
> not. Most of the time one relies upon leap of faith to trust #t
> values. In that regard Lua quite successfully bridged a gap between
> Coding and Religion:-)

Faulty applied programming. When did programming involve a leap of
faith? Try telling that to developers of aviation software.

Use #t conservatively, according to its specifications.

This advice has been repeated by many others in recent past postings.

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Dirk Laurie
In reply to this post by Leo Razoumov
On Tue, Jan 04, 2011 at 03:18:29PM +0200, Leo Razoumov wrote:
> Lua provides no good way to test whether a given array has holes or
> not.
Not Lua's job.  Your job.

You, as programmer, make a conscious decision as to whether your
arrays are allowed to contain holes.  If so, you don't rely on
the default #, table.insert, table.remove.

Trying very hard, this early 2011, to be a nice person,
Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Leo Razoumov
On Tue, Jan 4, 2011 at 08:41, Dirk Laurie <[hidden email]> wrote:
> On Tue, Jan 04, 2011 at 03:18:29PM +0200, Leo Razoumov wrote:
>> Lua provides no good way to test whether a given array has holes or
>> not.
> Not Lua's job.  Your job.
>
> You, as programmer, make a conscious decision as to whether your
> arrays are allowed to contain holes.  If so, you don't rely on
> the default #, table.insert, table.remove.
>

table.insert does not help. Please, read the manual.

-- From the Lua 5.1.4 manual
table.insert (table, [pos,] value)
Inserts element value at position pos in table, shifting up other
elements to open space, if necessary. The default value for pos is
n+1, where n is the length of the table (see §2.5.5), so that a call
table.insert(t,x) inserts x at the end of table t.

table.insert uses the same #len operator (aux_getn to be exact) and
will gladly insert your new value into the first available hole.

>Trying very hard, this early 2011, to be a nice person,
>Dirk

Try harder!

--Leo--

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Luiz Henrique de Figueiredo
In reply to this post by Dirk Laurie
> Trying very hard, this early 2011, to be a nice person,

Always a laudable intention.

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Henning Diedrich
On 1/4/11 2:55 PM, Luiz Henrique de Figueiredo wrote:
Trying very hard, this early 2011, to be a nice person,
Always a laudable intention.

And I think you did succeed, Dirk, you are wrong of course*, but you succeeded in being nice.

Henning

*) effectively self-deprecating irony. Hint provided courtesy New Year's resolution Lua mailing list betterment movement.
Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Axel Kittenberger
In reply to this post by Leo Razoumov
> table.insert uses the same #len operator (aux_getn to be exact) and
> will gladly insert your new value into the first available hole.

a) if by the "first available" you do not mean the lowest index
(otherwise again someone following into this trap? :-), but the first
showing up on binary search, then its true.

b) it might or might not shift indexes after the hole:

See: similar holds true for insert:
http://lua-users.org/lists/lua-l/2010-12/msg00595.html

And see response:
http://lua-users.org/lists/lua-l/2010-12/msg00596.html

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Leo Razoumov
On Tue, Jan 4, 2011 at 09:07, Axel Kittenberger <[hidden email]> wrote:
>> table.insert uses the same #len operator (aux_getn to be exact) and
>> will gladly insert your new value into the first available hole.
>
> a) if by the "first available" you do not mean the lowest index
> (otherwise again someone following into this trap? :-), but the first
> showing up on binary search, then its true.
>
> b) it might or might not shift indexes after the hole:
>

I was responding to a comment by Dirk who suggested to use
table.insert to append to the end of an array.
I simply wanted to point out that table.insert(t,v) suffers from the
same problems as t[#t+1]=v

Yes, "first available hole" should be replaced by "first found hole".
It does make a difference.

--Leo--

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Henning Diedrich
On 1/4/11 3:13 PM, Leo Razoumov wrote:
Yes, "first available hole" should be replaced by "first found hole".
It does make a difference.

Still not clear enough, but I think "first available" was coined for the irony.

In passing, I understood it's "any hole or after the end"?

Henning
Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Dirk Laurie
In reply to this post by Leo Razoumov
On Tue, Jan 04, 2011 at 04:13:29PM +0200, Leo Razoumov wrote:
>
> I was responding to a comment by Dirk who suggested to use
> table.insert to append to the end of an array.

May I quote my post?

| You, as programmer, make a conscious decision as to whether your
| arrays are allowed to contain holes.  If so, you don't rely on
| the default #, table.insert, table.remove.

The abbreviation "don't" stands for "do not".

Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Lua, LuaJIT2 and differences with the length operator

Leo Razoumov
In reply to this post by Dirk Laurie
On Tue, Jan 4, 2011 at 08:41, Dirk Laurie <[hidden email]> wrote:
> On Tue, Jan 04, 2011 at 03:18:29PM +0200, Leo Razoumov wrote:
>> Lua provides no good way to test whether a given array has holes or
>> not.
> Not Lua's job.  Your job.
>

And how exactly I am supposed to do this job? E.g., how can I test
that an array  t coming my way from someone else's library has no
holes? If you have a good way to test this condition, please, by all
means, share your expertise. If the actual purpose of your replies is
just ranting, please, spare your time and time of others.

--Leo--

12