Lua 5.3 `ipairs` considered perilous

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Lua 5.3 `ipairs` considered perilous

Dirk Laurie-2
It seemed such a good idea at the time, `ipairs` respecting the
__index metamethod. After burning my fingers a few times with
it, though, I find it simpler not to use `ipairs` at all, instead of
pondering each time whether ipairs will behave.

Example 1. XML tables. These are lists in which the items
are either scalars or XML tables, and which also have some
"attributes", i.e. string-valued keys associated with scalar values.
The context is very often one in which the items take default
attributes from the containing value by specifying it as __index.

Unfortunately numerical keys are also inherited. If you traverse
a subtable via 'ipairs', it will look for an item in the parent table
when the subtable is exhausted. And all the way back to the
top-level table.

Example 2. Plugging holes in lists of numbers.
    x = setmetatable({5,9,nil,6,nil,nil,3,1},
    {__index = load"return 0/0",
     __len=load"return 8"})
    print(table.concat(x," "))
5 9 -nan 6 -nan -nan 3 1
Without that "index", you can't concatenate the table.

However, try this:
> for k,v in ipairs(y) do print(k,v) end
and see what happens.

If only we still had __ipairs, neat solutions to the problem
would have been possible. As it stands, the straightforward
numeric `for` is the only answer.

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Sergey Zakharchenko
Hello Dirk,

2016-02-16 18:19 GMT+03:00 Dirk Laurie <[hidden email]>:
> Example 2. Plugging holes in lists of numbers.
>     x = setmetatable({5,9,nil,6,nil,nil,3,1},
>     {__index = load"return 0/0",
>      __len=load"return 8"})
>     print(table.concat(x," "))

> However, try this:
>> for k,v in ipairs(y) do print(k,v) end
> and see what happens.

Don't the supplied __len and __index explicitly supply contradictory data?

Best regards,

--
DoubleF

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Hisham
In reply to this post by Dirk Laurie-2
On 16 February 2016 at 13:19, Dirk Laurie <[hidden email]> wrote:

> It seemed such a good idea at the time, `ipairs` respecting the
> __index metamethod. After burning my fingers a few times with
> it, though, I find it simpler not to use `ipairs` at all, instead of
> pondering each time whether ipairs will behave.
>
> Example 1. XML tables. These are lists in which the items
> are either scalars or XML tables, and which also have some
> "attributes", i.e. string-valued keys associated with scalar values.
> The context is very often one in which the items take default
> attributes from the containing value by specifying it as __index.
>
> Unfortunately numerical keys are also inherited. If you traverse
> a subtable via 'ipairs', it will look for an item in the parent table
> when the subtable is exhausted. And all the way back to the
> top-level table.

That of course depends on the design of your XML-handling library. I
see it as a problem of the library and not of ipairs. (Out of
curiosity, which library does this? TBH it would surprise me when
reading a subtag if it inherited its parents attributes at all.)

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

David Favro
In reply to this post by Dirk Laurie-2
On 02/16/2016 10:19 AM, Dirk Laurie wrote:

> Example 1. XML tables. These are lists in which the items
> are either scalars or XML tables, and which also have some
> "attributes", i.e. string-valued keys associated with scalar values.
> The context is very often one in which the items take default
> attributes from the containing value by specifying it as __index.

I've never seen this practice, and wouldn't desire it myself.


> Example 2. Plugging holes in lists of numbers.
>      x = setmetatable({5,9,nil,6,nil,nil,3,1},
>      {__index = load"return 0/0",
>       __len=load"return 8"})
>      print(table.concat(x," "))
> 5 9 -nan 6 -nan -nan 3 1
> Without that "index", you can't concatenate the table.
>
> However, try this:
>> for k,v in ipairs(y) do print(k,v) end
> and see what happens.

It's a little more code, but it works:
tlen = 8;
x = setmetatable({5,9,nil,6,nil,nil,3,1},
     {__index = function(t,k)
             if type(k)=="number" and k<=tlen then return 0/0; end
             end;
      __len = function() return tlen; end;
      });

Furthermore, if the # operator respects __index (section 3.4.7 is a little
ambiguous on this, neither specifying that it does nor that it doesn't),
neither the upvalue tlen nor the __len() function are necessary.  In your
example, __len also isn't necessary, i.e. this works although I don't think
it can be relied upon based on the manual:
x = setmetatable({5,9,nil,6,nil,nil,3,1},
     {__index = function(t,k)
             if type(k)=="number" and k<=8 then return 0/0; end
             end;
      });


> If only we still had __ipairs, neat solutions to the problem
> would have been possible.

Solutions ("neat" is a matter of opinion) are still possible, see above example.

You didn't specify what would be your alternative "neat" solution if
__ipairs() still existed. If you don't specify *both* __ipairs *and*
__index, you would not have filled the holes, since x[3] would still return
nil, while `for i,v in ipairs(x)` would produce 3,nan.  So my guess is that
your "neat" solution would have at least as much code as my "unkempt" one.


> As it stands, the straightforward
> numeric `for` is the only answer.

No it isn't [see above example] but even if it were, what's wrong with
numeric `for`s?

While I have no big problem with __ipairs, I do think that the default
ipairs() should respect __index.  Given that it does, there are only a tiny
minority of cases when it is simpler to use __ipairs than __index, so it
seems cleaner to remove it.

-- David


Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Luiz Henrique de Figueiredo
In reply to this post by Hisham
> TBH it would surprise me when reading a subtag if it inherited its
> parents attributes at all.)

SVG is based on XML *and* inheritance of attributes.

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Mason Bogue
>neat solutions to the problem would have been possible.

It doesn't even need to be that much longer:

function __index(self, k) return type(k) == "number" and k <= #self
and 0/0 or nil end

I don't think it's too much of a burden to write what you actually
mean in the __index metamethod. If the __index() returns non-nil
values for positions that should be nil, it's because your code is
wrong, not Lua.

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Daurnimator
On 18 February 2016 at 12:48, Mason Bogue <[hidden email]> wrote:
> and 0/0 or nil

^^ I'm not sure what that was meant to be, as 0/0 is always truthy, so
the `or nil` is never hit.

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Jonathan Goble
On Wed, Feb 17, 2016 at 9:19 PM, Daurnimator <[hidden email]> wrote:
> On 18 February 2016 at 12:48, Mason Bogue <[hidden email]> wrote:
>> and 0/0 or nil
>
> ^^ I'm not sure what that was meant to be, as 0/0 is always truthy, so
> the `or nil` is never hit.

It's the classic idiom "test and value_if_true or value_if_false",
just extended with two tests. For reference, the function posted by
Mason (plus some line breaks and indentation to eliminate hard line
wraps) was:

function __index(self, k)
    return type(k) == "number" and k <= #self and 0/0 or nil
end

First, 'type(k) == "number"' is tested. If false, then the binary
operation 'type(k) == "number" and k <= #self' short-circuits and
returns its first argument, the boolean false. Then 'false and 0/0'
short-circuits, returning its first argument (false). That leaves
'false or nil', which returns its second argument, nil, as the result
of the whole chain of operations.

If type(k) is a number, though, then 'type(k) == "number" and k <=
#self' returns the result of 'k <= #self'. If that is false, then the
next 'and' operation short-circuits as above, and nil is again the
final result. If true, then you get 'true and 0/0', which results in
the truthy '0/0', which then short-circuits the final op.

Does that all make sense now?

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Daurnimator
On 18 February 2016 at 13:33, Jonathan Goble <[hidden email]> wrote:

> On Wed, Feb 17, 2016 at 9:19 PM, Daurnimator <[hidden email]> wrote:
>> On 18 February 2016 at 12:48, Mason Bogue <[hidden email]> wrote:
>>> and 0/0 or nil
>>
>> ^^ I'm not sure what that was meant to be, as 0/0 is always truthy, so
>> the `or nil` is never hit.
>
> It's the classic idiom "test and value_if_true or value_if_false",
> just extended with two tests. For reference, the function posted by
> Mason (plus some line breaks and indentation to eliminate hard line
> wraps) was:
>
> function __index(self, k)
>     return type(k) == "number" and k <= #self and 0/0 or nil
> end
>
> First, 'type(k) == "number"' is tested. If false, then the binary
> operation 'type(k) == "number" and k <= #self' short-circuits and
> returns its first argument, the boolean false. Then 'false and 0/0'
> short-circuits, returning its first argument (false). That leaves
> 'false or nil', which returns its second argument, nil, as the result
> of the whole chain of operations.
>
> If type(k) is a number, though, then 'type(k) == "number" and k <=
> #self' returns the result of 'k <= #self'. If that is false, then the
> next 'and' operation short-circuits as above, and nil is again the
> final result. If true, then you get 'true and 0/0', which results in
> the truthy '0/0', which then short-circuits the final op.
>
> Does that all make sense now?
>

Yep. sorry, had a brain fail :)

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.3 `ipairs` considered perilous

Daniel Silverstone
On Thu, Feb 18, 2016 at 13:36:40 +1100, Daurnimator wrote:

> > It's the classic idiom "test and value_if_true or value_if_false",
> > just extended with two tests. For reference, the function posted by
> > Mason (plus some line breaks and indentation to eliminate hard line
> > wraps) was:
> >
> > function __index(self, k)
> >     return type(k) == "number" and k <= #self and 0/0 or nil
> > end
> >
> > First, 'type(k) == "number"' is tested. If false, then the binary
> > operation 'type(k) == "number" and k <= #self' short-circuits and
> > returns its first argument, the boolean false. Then 'false and 0/0'
> > short-circuits, returning its first argument (false). That leaves
> > 'false or nil', which returns its second argument, nil, as the result
> > of the whole chain of operations.
> >
> > If type(k) is a number, though, then 'type(k) == "number" and k <=
> > #self' returns the result of 'k <= #self'. If that is false, then the
> > next 'and' operation short-circuits as above, and nil is again the
> > final result. If true, then you get 'true and 0/0', which results in
> > the truthy '0/0', which then short-circuits the final op.
> >
> > Does that all make sense now?
> >
>
> Yep. sorry, had a brain fail :)

And this is why the keyboard gods put parentheses into our keymaps and
Roberto let us use them to disambiguate our code.

D.

--
Daniel Silverstone                         http://www.digital-scurf.org/
PGP mail accepted and encouraged.            Key Id: 3CCE BABE 206C 3B69