Lua life after integers

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Lua life after integers

Tuom Larsen
Dear Lua developers!

I have to immediately apologize for bringing up this topic again.
However, the introduction of integer type made me curious whether
there was any change in your stance on other features as well. After
all, I can imagine if someone proposed integers few years ago it would
be rejected in the name of simplicity [1].

Please, what is your opinion about:

- splitting table into hash-table-based dictionary and resizable
vector/array/list?

- indexing from zero, instead of one?

I don't want to bother you with my reasoning, I bet you heard many
similar explanations before.

Thanks a lot in advance, I hope you don't mind such questions too much!

T.

[1] "Lua has no integer type, as it does not need it."
http://www.lua.org/pil/2.3.html

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Dirk Laurie-2
2014-12-08 17:10 GMT+02:00 Tuom Larsen <[hidden email]>:

> I have to immediately apologize for bringing up this topic again.
> However, the introduction of integer type made me curious whether
> there was any change in your stance on other features as well. After
> all, I can imagine if someone proposed integers few years ago it would
> be rejected in the name of simplicity [1].

Someone did propose integers a few years ago. Also struct and UTF-8.
All of these are now in Lua 5.3 beta.

[1] www.lua.org/wshop12/Ierusalimschy.pdf

> - splitting table into hash-table-based dictionary and resizable
> vector/array/list?

This is the current implementation, exploitable in the API
through `lua_createtable` but not at the Lua script level.

> - indexing from zero, instead of one?

Zot.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Hisham
In reply to this post by Tuom Larsen
On 8 December 2014 at 13:10, Tuom Larsen <[hidden email]> wrote:
> Dear Lua developers!
>
> I have to immediately apologize for bringing up this topic again.
> However, the introduction of integer type made me curious whether
> there was any change in your stance on other features as well. After
> all, I can imagine if someone proposed integers few years ago it would
> be rejected in the name of simplicity [1].
[...]
> [1] "Lua has no integer type, as it does not need it."
> http://www.lua.org/pil/2.3.html

It wasn't that much that their stance has changed; the world has
changed instead since PiL 1st edition with regard to integers. Lua
5.0, back in 2003 in a world ruled by 32-bit machines, did not need a
separate integer type. Lua 5.3 in this 64-bit world of 2014, does.

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Rena

On 2014-12-08 10:41 AM, "Hisham" <[hidden email]> wrote:
>
> On 8 December 2014 at 13:10, Tuom Larsen <[hidden email]> wrote:
> > Dear Lua developers!
> >
> > I have to immediately apologize for bringing up this topic again.
> > However, the introduction of integer type made me curious whether
> > there was any change in your stance on other features as well. After
> > all, I can imagine if someone proposed integers few years ago it would
> > be rejected in the name of simplicity [1].
> [...]
> > [1] "Lua has no integer type, as it does not need it."
> > http://www.lua.org/pil/2.3.html
>
> It wasn't that much that their stance has changed; the world has
> changed instead since PiL 1st edition with regard to integers. Lua
> 5.0, back in 2003 in a world ruled by 32-bit machines, did not need a
> separate integer type. Lua 5.3 in this 64-bit world of 2014, does.
>
> -- Hisham
>

Bit operators were something I missed ever since I discovered Lua, though. I'm glad they made it in. I think that's my favorite feature of 5.3.

If I were going to beg for another new feature/change, it'd be for metamethods to be honored in more cases, and an "is callable" and "is indexable" function. But I'm still pretty satisfied with Lua as it is now. Kudos to the authors for such a lean, flexible piece of kit.

--
Sent from my Game Boy.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Luiz Henrique de Figueiredo
In reply to this post by Dirk Laurie-2
> > - splitting table into hash-table-based dictionary and resizable
> > vector/array/list?
>
> This is the current implementation

This is only partially true: A sequence may be have parts in the array
part and parts in the hash part. Anyway, this should not matter.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Luiz Henrique de Figueiredo
In reply to this post by Rena
> If I were going to beg for another new feature/change, it'd be for
> metamethods to be honored in more cases

Lua 5.3 does honor more index metamethods in almost all functions.
What else do you have in mind?

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Rena
On Mon, Dec 8, 2014 at 11:04 AM, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
>> If I were going to beg for another new feature/change, it'd be for
>> metamethods to be honored in more cases
>
> Lua 5.3 does honor more index metamethods in almost all functions.
> What else do you have in mind?
>

If I'm understanding the manual correctly, lua_tolstring() doesn't
honour __tostring. Similarly lua_tonumberx() doesn't call __tonumber
(if such a metamethod even exists?). The manual also doesn't appear to
specify whether lua_call() will look at __call.

I was also wishing there was lua_geti() and lua_seti(), but it looks
like those did get added at some point. :-) (or were they there all
along and I'm just blind?)

--
Sent from my Game Boy.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Ryan
>indexing from zero, instead of one?

This would be a complete backwards-compatibility break with little gain.  If you want it, add 1 to your index or subtract 1 from Lua's index.

On Mon, Dec 8, 2014 at 4:20 PM, Rena <[hidden email]> wrote:
On Mon, Dec 8, 2014 at 11:04 AM, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
>> If I were going to beg for another new feature/change, it'd be for
>> metamethods to be honored in more cases
>
> Lua 5.3 does honor more index metamethods in almost all functions.
> What else do you have in mind?
>

If I'm understanding the manual correctly, lua_tolstring() doesn't
honour __tostring. Similarly lua_tonumberx() doesn't call __tonumber
(if such a metamethod even exists?). The manual also doesn't appear to
specify whether lua_call() will look at __call.

I was also wishing there was lua_geti() and lua_seti(), but it looks
like those did get added at some point. :-) (or were they there all
along and I'm just blind?)

--
Sent from my Game Boy.


Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Luiz Henrique de Figueiredo
In reply to this post by Rena
> If I'm understanding the manual correctly, lua_tolstring() doesn't
> honour __tostring. Similarly lua_tonumberx() doesn't call __tonumber
> (if such a metamethod even exists?). The manual also doesn't appear to
> specify whether lua_call() will look at __call.

All core metamethods are honored in the Lua API, except for those
functions that are named *raw*. Both the API functions and the VM
instructions ultimately use the same code. The core metamethods
are listed in http://www.lua.org/work/doc/manual.html#2.4 .

__tostring is not a core metamethod, nor are __pairs and __ipairs.

So, lua_tolstring does not honor __tostring. luaL_tolstring does.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Andrew Starks


On Monday, December 8, 2014, Luiz Henrique de Figueiredo <[hidden email]> wrote:
> If I'm understanding the manual correctly, lua_tolstring() doesn't
> honour __tostring. Similarly lua_tonumberx() doesn't call __tonumber
> (if such a metamethod even exists?). The manual also doesn't appear to
> specify whether lua_call() will look at __call.

All core metamethods are honored in the Lua API, except for those
functions that are named *raw*. Both the API functions and the VM
instructions ultimately use the same code. The core metamethods
are listed in http://www.lua.org/work/doc/manual.html#2.4 .

__tostring is not a core metamethod, nor are __pairs and __ipairs.

So, lua_tolstring does not honor __tostring. luaL_tolstring does.


Neither is __call then, I take it? It is not honored in xpcall, at least.  
Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Luiz Henrique de Figueiredo
> Neither is __call then, I take it? It is not honored in xpcall, at least.

__call is a core metamethod.

xpcall honors it (by not doing anything special about it):
        t = setmetatable({},{__call=print})
        print(xpcall(t,nil,10,20,30))

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Andrew Starks


On Monday, December 8, 2014, Luiz Henrique de Figueiredo <[hidden email]> wrote:
> Neither is __call then, I take it? It is not honored in xpcall, at least.

__call is a core metamethod.

xpcall honors it (by not doing anything special about it):
        t = setmetatable({},{__call=print})
        print(xpcall(t,nil,10,20,30))

I was referring to `msgh` 
Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Luiz Henrique de Figueiredo
> I was referring to `msgh`

In that case, yes, xpcall insists on a true function.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Soni "They/Them" L.

On 09/12/14 08:48 AM, Luiz Henrique de Figueiredo wrote:
>> I was referring to `msgh`
> In that case, yes, xpcall insists on a true function.
>
Why's that?

--
Disclaimer: these emails are public and can be accessed from <TODO: get a non-DHCP IP and put it here>. If you do not agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Rena
In reply to this post by Luiz Henrique de Figueiredo
On Mon, Dec 8, 2014 at 5:20 PM, Luiz Henrique de Figueiredo
<[hidden email]> wrote:

>> If I'm understanding the manual correctly, lua_tolstring() doesn't
>> honour __tostring. Similarly lua_tonumberx() doesn't call __tonumber
>> (if such a metamethod even exists?). The manual also doesn't appear to
>> specify whether lua_call() will look at __call.
>
> All core metamethods are honored in the Lua API, except for those
> functions that are named *raw*. Both the API functions and the VM
> instructions ultimately use the same code. The core metamethods
> are listed in http://www.lua.org/work/doc/manual.html#2.4 .
>
> __tostring is not a core metamethod, nor are __pairs and __ipairs.
>
> So, lua_tolstring does not honor __tostring. luaL_tolstring does.
>

Interesting. Why is __tostring not a core metamethod? The distinction
seems arbitrary to me.

As an aside, I notice the changes section in the 5.3 manual doesn't
list functions that were newly added, such as lua_geti. I think
listing new functions would be very helpful to keep people like me
from requesting things without noticing they were already added. :-)

--
Sent from my Game Boy.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Dirk Laurie-2
2014-12-10 16:44 GMT+02:00 Rena <[hidden email]>:

> Interesting. Why is __tostring not a core metamethod? The distinction
> seems arbitrary to me.

A core metamethod is a fallback. It is called when Lua does not
know what to do, as a last resort instead of throwing an error.
__tostring, __pairs etc are called _instead_ of what Lua knows
to do.

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Andrew Starks
On Wed, Dec 10, 2014 at 8:52 AM, Dirk Laurie <[hidden email]> wrote:

> 2014-12-10 16:44 GMT+02:00 Rena <[hidden email]>:
>
>> Interesting. Why is __tostring not a core metamethod? The distinction
>> seems arbitrary to me.
>
> A core metamethod is a fallback. It is called when Lua does not
> know what to do, as a last resort instead of throwing an error.
> __tostring, __pairs etc are called _instead_ of what Lua knows
> to do.
>

I don't make that distinction, even when I'm reminded that the authors do.

The reason is that, to me, it's meaningless. It works or it does not
work and remembering where it works and doesn't is not something that
I care to invest the time to memorize. At best, I know that whenever I
define `__ipairs`... sort of worry that it may not work, if my table
is handed to a library that may not honor it. To my way of thinking,
there are fields that are attached to metatables that begin with `__`
and end with the name of the method that they modify. Anything beyond
this, to paraphrase the Bible, is complex.

I don't know if this is important enough to bring up in another
thread, but I also don't like the deprecation of `__ipairs`. Whatever
the redundancy it may suffer, it is clear to the writer or reader of
the code that a sequence of key/value pairs, iterated in a consistent
order, is expected. If `ipairs` and `pairs` exist and we allow for a
way to override `pairs`, then consistency would require a
corresponding way to override ipairs.

I may be misunderstanding this issue and I may be mistaken about 5.3
deprecating it. If I am correct, then it appears to me that the
evolution of Lua is such that  eradicating redundancy is valued over
consistency. "Consistency" can be argued about (see math library
change).

I guess any person's definition of "simplicity" is their own.

-Andrew

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Sean Conner
It was thus said that the Great Andrew Starks once stated:

> On Wed, Dec 10, 2014 at 8:52 AM, Dirk Laurie <[hidden email]> wrote:
> > 2014-12-10 16:44 GMT+02:00 Rena <[hidden email]>:
> >
> >> Interesting. Why is __tostring not a core metamethod? The distinction
> >> seems arbitrary to me.
> >
> > A core metamethod is a fallback. It is called when Lua does not
> > know what to do, as a last resort instead of throwing an error.
> > __tostring, __pairs etc are called _instead_ of what Lua knows
> > to do.
> >
>
> I don't make that distinction, even when I'm reminded that the authors do.
>
> The reason is that, to me, it's meaningless. It works or it does not
> work and remembering where it works and doesn't is not something that
> I care to invest the time to memorize. At best, I know that whenever I
> define `__ipairs`... sort of worry that it may not work, if my table
> is handed to a library that may not honor it. To my way of thinking,
> there are fields that are attached to metatables that begin with `__`
> and end with the name of the method that they modify. Anything beyond
> this, to paraphrase the Bible, is complex.
>
> I don't know if this is important enough to bring up in another
> thread, but I also don't like the deprecation of `__ipairs`. Whatever
> the redundancy it may suffer, it is clear to the writer or reader of
> the code that a sequence of key/value pairs, iterated in a consistent
> order, is expected. If `ipairs` and `pairs` exist and we allow for a
> way to override `pairs`, then consistency would require a
> corresponding way to override ipairs.

  To recap:  in Lua 5.2, ipairs() will attempt to call __ipairs(), which
should return an iterator function, self and an initial value; otherwise,
ipairs() supplies the iterator function, self and an initial value.

  The default iterator function goes from index 1 until self[index] returns
nil.  I suspect that most (if not all) iterator functions returned from
__ipairs() will do the exact same thing, although it doesn't have to be:

        mt =                                
        {
          __ipairs = function(t)
            local function iter(s,k)
              if k == -1 then
                return 'one',"ONE"
              elseif k == 'one' then
                return 'two',"TWO"
              elseif k == 'two' then
                return 'three',"THREE"
              else
                return nil
              end      
            end
       
            return iter,t,-1  
          end  
        }

        x = setmetatable({},mt)

        for index,value in ipairs(x) do
          print(index,value)
        end

        one ONE
        two TWO
        three THREE

  At this point, you could use __pairs().  Okay, ipairs() could then check
to see if the initial value returned from __ipairs() is '0' (the default
value if __pairs() doesn't exist) but then, why even bother with __ipairs()?
I suppose it *could* check for an integer initial value, but then, is it
then a sequence as Lua even defines it?  Is that an issue? Consistent even?

  I know, there's signalling intent.  But the default iterator from ipairs()
supports __index, and in Lua, a sequence for ipairs() is defined as a
sequence of integer keys from 1 to n with non-nil values.  [1]

> I may be misunderstanding this issue and I may be mistaken about 5.3
> deprecating it. If I am correct, then it appears to me that the
> evolution of Lua is such that  eradicating redundancy is valued over
> consistency. "Consistency" can be argued about (see math library
> change).

  If that were true, then ipairs() would be removed as you can always do:

        for i = 1 , #t do ... end

  -spc (Viewing this from a userdata perspective ... )

[1] I would really expect Thiago, Rena and Coroutines [2] to be upset
        over the removal of __ipairs() as (to me) they seem like the type of
        people that love overriding how the language works.

[2] What ever happened to Coroutines anyway?

Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Soni "They/Them" L.

On 10/12/14 03:32 PM, Sean Conner wrote:

> It was thus said that the Great Andrew Starks once stated:
>> On Wed, Dec 10, 2014 at 8:52 AM, Dirk Laurie <[hidden email]> wrote:
>>> 2014-12-10 16:44 GMT+02:00 Rena <[hidden email]>:
>>>
>>>> Interesting. Why is __tostring not a core metamethod? The distinction
>>>> seems arbitrary to me.
>>> A core metamethod is a fallback. It is called when Lua does not
>>> know what to do, as a last resort instead of throwing an error.
>>> __tostring, __pairs etc are called _instead_ of what Lua knows
>>> to do.
>>>
>> I don't make that distinction, even when I'm reminded that the authors do.
>>
>> The reason is that, to me, it's meaningless. It works or it does not
>> work and remembering where it works and doesn't is not something that
>> I care to invest the time to memorize. At best, I know that whenever I
>> define `__ipairs`... sort of worry that it may not work, if my table
>> is handed to a library that may not honor it. To my way of thinking,
>> there are fields that are attached to metatables that begin with `__`
>> and end with the name of the method that they modify. Anything beyond
>> this, to paraphrase the Bible, is complex.
>>
>> I don't know if this is important enough to bring up in another
>> thread, but I also don't like the deprecation of `__ipairs`. Whatever
>> the redundancy it may suffer, it is clear to the writer or reader of
>> the code that a sequence of key/value pairs, iterated in a consistent
>> order, is expected. If `ipairs` and `pairs` exist and we allow for a
>> way to override `pairs`, then consistency would require a
>> corresponding way to override ipairs.
>    To recap:  in Lua 5.2, ipairs() will attempt to call __ipairs(), which
> should return an iterator function, self and an initial value; otherwise,
> ipairs() supplies the iterator function, self and an initial value.
>
>    The default iterator function goes from index 1 until self[index] returns
> nil.  I suspect that most (if not all) iterator functions returned from
> __ipairs() will do the exact same thing, although it doesn't have to be:
>
> mt =
> {
>  __ipairs = function(t)
>    local function iter(s,k)
>      if k == -1 then
>        return 'one',"ONE"
>      elseif k == 'one' then
>        return 'two',"TWO"
>      elseif k == 'two' then
>        return 'three',"THREE"
>      else
>        return nil
>      end
>    end
>
>    return iter,t,-1
>  end
> }
>
> x = setmetatable({},mt)
>
> for index,value in ipairs(x) do
>  print(index,value)
> end
>
> one ONE
> two TWO
> three THREE
>
>    At this point, you could use __pairs().  Okay, ipairs() could then check
> to see if the initial value returned from __ipairs() is '0' (the default
> value if __pairs() doesn't exist) but then, why even bother with __ipairs()?
> I suppose it *could* check for an integer initial value, but then, is it
> then a sequence as Lua even defines it?  Is that an issue? Consistent even?
>
>    I know, there's signalling intent.  But the default iterator from ipairs()
> supports __index, and in Lua, a sequence for ipairs() is defined as a
> sequence of integer keys from 1 to n with non-nil values.  [1]
>
>> I may be misunderstanding this issue and I may be mistaken about 5.3
>> deprecating it. If I am correct, then it appears to me that the
>> evolution of Lua is such that  eradicating redundancy is valued over
>> consistency. "Consistency" can be argued about (see math library
>> change).
>    If that were true, then ipairs() would be removed as you can always do:
>
> for i = 1 , #t do ... end
>
>    -spc (Viewing this from a userdata perspective ... )
>
> [1] I would really expect Thiago, Rena and Coroutines [2] to be upset
> over the removal of __ipairs() as (to me) they seem like the type of
> people that love overriding how the language works.
>
> [2] What ever happened to Coroutines anyway?
>
local dummy = {}
ipairs = function(t) -- if you really need __ipairs
   return (rawget(debug.getmetatable(t) or dummy, "__ipairs") or
(function(t) return function(t,k) k = k + 1 local v = rawget(t, k) if v
~= nil then return k, v end return nil end, t, 0 end))(t) -- I guess
actual ipairs() has some type checking and stuff, for example this
accepts __call-ables too and idk about actual ipairs()
end

--
Disclaimer: these emails are public and can be accessed from <TODO: get a non-DHCP IP and put it here>. If you do not agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Lua life after integers

Andrew Starks
In reply to this post by Sean Conner
On Wed, Dec 10, 2014 at 11:32 AM, Sean Conner <[hidden email]> wrote:

>   To recap:  in Lua 5.2, ipairs() will attempt to call __ipairs(), which
> should return an iterator function, self and an initial value; otherwise,
> ipairs() supplies the iterator function, self and an initial value.
>
>   The default iterator function goes from index 1 until self[index] returns
> nil.  I suspect that most (if not all) iterator functions returned from
> __ipairs() will do the exact same thing, although it doesn't have to be:
>
>         mt =
>         {
>           __ipairs = function(t)
>             local function iter(s,k)
>               if k == -1 then
>                 return 'one',"ONE"
>               elseif k == 'one' then
>                 return 'two',"TWO"
>               elseif k == 'two' then
>                 return 'three',"THREE"
>               else
>                 return nil
>               end
>             end
>
>             return iter,t,-1
>           end
>         }
>
>         x = setmetatable({},mt)
>
>         for index,value in ipairs(x) do
>           print(index,value)
>         end
>
>         one     ONE
>         two     TWO
>         three   THREE
>
>   At this point, you could use __pairs().  Okay, ipairs() could then check
> to see if the initial value returned from __ipairs() is '0' (the default
> value if __pairs() doesn't exist) but then, why even bother with __ipairs()?
> I suppose it *could* check for an integer initial value, but then, is it
> then a sequence as Lua even defines it?  Is that an issue? Consistent even?

This is a good point. If the intent is to override __ipairs, then it
*should* return an __ipairs-like result. Integers, preferably starting
at one and preferably values that are not nil.

>   I know, there's signalling intent.  But the default iterator from ipairs()
> supports __index, and in Lua, a sequence for ipairs() is defined as a
> sequence of integer keys from 1 to n with non-nil values.  [1]

Hmm. I see your (the author's presumably, as well) point. The same
effect may be had, except that there is no longer a way to continue
after the value is nil, except by using some other mechanism, such as
a while loop.

There may be other side effects.

>
>> I may be misunderstanding this issue and I may be mistaken about 5.3
>> deprecating it. If I am correct, then it appears to me that the
>> evolution of Lua is such that  eradicating redundancy is valued over
>> consistency. "Consistency" can be argued about (see math library
>> change).
>
>   If that were true, then ipairs() would be removed as you can always do:
>
>         for i = 1 , #t do ... end
>
>   -spc (Viewing this from a userdata perspective ... )



>
> [1]     I would really expect Thiago, Rena and Coroutines [2] to be upset
>         over the removal of __ipairs() as (to me) they seem like the type of
>         people that love overriding how the language works.

Perhaps you may be conflating monkey patches and metamethods?

Given my lack of appreciation for the fact that ipairs respects
`__index`, my post makes more sense. To my credit, I anticipated this
and peppered my post with "to me" and "if I understand this
correctly." :)

I remain skeptical, but given the above (which I failed to locate in
my very brief scan over prior emails), it seems like a very natural
and logical change.

It's just the sort of terse solution that reminds you that Lua is
authored by people who live in the world of mathematics.

-Andrew


> [2]     What ever happened to Coroutines anyway?
>

* I don't pretend to know, but I recognize the pattern in my own
behavior. Engage. Obsess. Go too far. Retreat. Repeat.

12