Upcoming changes in Lua 5.4

classic Classic list List threaded Threaded
48 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Upcoming changes in Lua 5.4

Dibyendu Majumdar
Hi,

I noticed that the Lua arithmetic operators in 5.4 will not coerce
strings to numbers ... is my understanding correct? I think this is
good for performance as it avoids a function call for converting
values but presumably this is a breaking change for existing code that
may be relying on this feature?

Is it likely that this change will remain?

Also noticed that var args will now be put into a table - wanted to
check if this is just an implementation detail - i.e. no user visible
impact on either Lua or the C API?

Finally noted the change to handling of upvalues - upvalues are o
longer reference counted and instead managed as regular GC objects.
Presumably this doesn't affect performance?

Thanks and Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Daurnimator
On 1 January 2018 at 23:49, Dibyendu Majumdar <[hidden email]> wrote:
> I noticed that the Lua arithmetic operators in 5.4 will not coerce
> strings to numbers ... is my understanding correct? I think this is
> good for performance as it avoids a function call for converting
> values but presumably this is a breaking change for existing code that
> may be relying on this feature?

See previous discussions around LUA_NOCVTS2N
e.g. http://lua-users.org/lists/lua-l/2015-01/msg00623.html

> Also noticed that var args will now be put into a table - wanted to
> check if this is just an implementation detail - i.e. no user visible
> impact on either Lua or the C API?

_ARG is user-visible.

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Paige DePol
In reply to this post by Dibyendu Majumdar
Dibyendu Majumdar <[hidden email]> wrote:

> Hi,
>
> I noticed that the Lua arithmetic operators in 5.4 will not coerce
> strings to numbers ... is my understanding correct? I think this is
> good for performance as it avoids a function call for converting
> values but presumably this is a breaking change for existing code that
> may be relying on this feature?
>
> Is it likely that this change will remain?
>
> Also noticed that var args will now be put into a table - wanted to
> check if this is just an implementation detail - i.e. no user visible
> impact on either Lua or the C API?
>
> Finally noted the change to handling of upvalues - upvalues are o
> longer reference counted and instead managed as regular GC objects.
> Presumably this doesn't affect performance?
>
> Thanks and Regards
> Dibyendu

Some of these changes seem in-line with some of the ideas I had for my own
fork of Lua (vargs and upvalues specifically), so it would be interesting
to see how these changes were implemented for myself as well.

Where are you finding information about Lua 5.4? I checked the Lua website
(under the /work/ directory) and did not see anything about a 5.4 version.

I also checked GitHub but that seems to be Lua 5.3.4 still, well at least it
is according to the macros in lua.h at any rate... is the GitHub repository
actually being updated without changing the Lua version?

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Dibyendu Majumdar
Hi,

On 1 January 2018 at 21:46, Paige DePol <[hidden email]> wrote:
> Where are you finding information about Lua 5.4? I checked the Lua website
> (under the /work/ directory) and did not see anything about a 5.4 version.
>
> I also checked GitHub but that seems to be Lua 5.3.4 still, well at least it
> is according to the macros in lua.h at any rate... is the GitHub repository
> actually being updated without changing the Lua version?
>

Changes are in github repository - I think version will get updated
prior to release. I am guessing this will be 5.4 seeing as it has the
generational GC.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Paige DePol
Dibyendu Majumdar <[hidden email]> wrote:

> Hi,
>
> On 1 January 2018 at 21:46, Paige DePol <[hidden email]> wrote:
>> Where are you finding information about Lua 5.4? I checked the Lua website
>> (under the /work/ directory) and did not see anything about a 5.4 version.
>>
>> I also checked GitHub but that seems to be Lua 5.3.4 still, well at least it
>> is according to the macros in lua.h at any rate... is the GitHub repository
>> actually being updated without changing the Lua version?
>
> Changes are in github repository - I think version will get updated
> prior to release. I am guessing this will be 5.4 seeing as it has the
> generational GC.
>
> Regards
> Dibyendu

Yes, it does look like GitHub has more up-to-date code than the last official
release of Lua. Not sure why I didn't notice this before... especially as the
commit comments definitely indicate changes... so I do apologise for the noise.

I did not realise that GitHub was updated with developmental Lua code, so that
is a nice discovery for the new year. Is there any expected timeline for the
first release candidate version of Lua 5.4?

The last update to GitHub was July 9th... I am going to guess that there are
probably some significant changes from the code currently on GitHub then!

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Soni "They/Them" L.
In reply to this post by Daurnimator


On 2018-01-01 10:53 AM, Daurnimator wrote:

> On 1 January 2018 at 23:49, Dibyendu Majumdar <[hidden email]> wrote:
>> I noticed that the Lua arithmetic operators in 5.4 will not coerce
>> strings to numbers ... is my understanding correct? I think this is
>> good for performance as it avoids a function call for converting
>> values but presumably this is a breaking change for existing code that
>> may be relying on this feature?
> See previous discussions around LUA_NOCVTS2N
> e.g. http://lua-users.org/lists/lua-l/2015-01/msg00623.html
>
>> Also noticed that var args will now be put into a table - wanted to
>> check if this is just an implementation detail - i.e. no user visible
>> impact on either Lua or the C API?
> _ARG is user-visible.
>

Ooh select()-free vararg indexing, this will be interesting!

I guess ... is just a glorified (read: faster) table.unpack?

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Roberto Ierusalimschy
In reply to this post by Dibyendu Majumdar
> I noticed that the Lua arithmetic operators in 5.4 will not coerce
> strings to numbers ... is my understanding correct? I think this is
> good for performance as it avoids a function call for converting
> values but presumably this is a breaking change for existing code that
> may be relying on this feature?

Coercion is still present through string metamethods. In practice, the
semantics is quite similar to the old one, or even better:

Lua 5.3:
> "10" + 1      --> 11.0

Lua 5.4:
> "10" + 1      --> 11


> Also noticed that var args will now be put into a table - wanted to
> check if this is just an implementation detail - i.e. no user visible
> impact on either Lua or the C API?

As already explained, you can use _ARG; but you can use also any other
name:

  function foo (...=a) print(a.n) end

_ARG is present both for compatibility (the above syntax will not even
compile in older versions) and for the main chunk (which does not have
a header to name this parameter).


> Finally noted the change to handling of upvalues - upvalues are o
> longer reference counted and instead managed as regular GC objects.
> Presumably this doesn't affect performance?

Upvalues were implemented in this way in Lua 5.0, 5.1, and 5.2. Only
Lua 5.3 changed to reference counting. This way (regular GC objects) is
simpler for generational collection, as upvalues also go through the
generations. We did not measure any performance difference.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Mike Nelson
On 1/2/2018 6:18 AM, Roberto Ierusalimschy wrote:  ...
> As already explained, you can use _ARG; but you can use also any other
> name:
>
>    function foo (...=a) print(a.n) end
>
> _ARG is present both for compatibility (the above syntax will not even
> compile in older versions) and for the main chunk (which does not have
> a header to name this parameter).
>
  I just love the new syntax. I assume select will continue to work as
usual, for old code and for use cases where it works better. The _ARG
name makes the parameter name for the main chunk consistent with Lua
_NAME practice, a good change. Any code broken by this last change
should be an easy fix: a simple search and replace.

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Soni "They/Them" L.


On 2018-01-02 03:21 PM, Michael Nelson wrote:

> On 1/2/2018 6:18 AM, Roberto Ierusalimschy wrote:  ...
>> As already explained, you can use _ARG; but you can use also any other
>> name:
>>
>>    function foo (...=a) print(a.n) end
>>
>> _ARG is present both for compatibility (the above syntax will not even
>> compile in older versions) and for the main chunk (which does not have
>> a header to name this parameter).
>>
>  I just love the new syntax. I assume select will continue to work as
> usual, for old code and for use cases where it works better. The _ARG
> name makes the parameter name for the main chunk consistent with Lua
> _NAME practice, a good change. Any code broken by this last change
> should be an easy fix: a simple search and replace.
>

Hmm... Would the following work (and, if so, would it be faster than the
C API/built-in table.unpack)?

table.unpack = function(t, i, j)
   assert(type(t) == "table", "first arg to table.unpack must be table")
   i = i or 1
   j = j or rawlen(t)
   local oldn = rawget(t, "n")
   rawset(t, "n", j)
   local _ARG = t
   local function helper(t, oldn, ...)
     rawset(t, "n", oldn)
     return ...
   end
   return helper(t, oldn, select(i, ...)) -- I guess this select() is
the limiting factor.
end

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Paige DePol
In reply to this post by Mike Nelson
Michael Nelson <[hidden email]> wrote:

> On 1/2/2018 6:18 AM, Roberto Ierusalimschy wrote:  ...
>> As already explained, you can use _ARG; but you can use also any other
>> name:
>>
>> function foo (...=a) print(a.n) end
>>
>> _ARG is present both for compatibility (the above syntax will not even
>> compile in older versions) and for the main chunk (which does not have
>> a header to name this parameter).
> I just love the new syntax. I assume select will continue to work as
> usual, for old code and for use cases where it works better. The _ARG
> name makes the parameter name for the main chunk consistent with Lua
> _NAME practice, a good change. Any code broken by this last change should
> be an easy fix: a simple search and replace.

I have to agree, the new syntax for vargs will be nicer... and is something
I have mapped out for my hard fork of Lua using my Index data type. So, it
will be nice that both vanilla Lua and my Lua variant will now have the same
syntax potentially for varg access!

One thing I am curious about, Roberto, is the creation of tables to hold the
vargs. Won't this add significant overhead for the table allocations vs just
leaving the vargs on the stack?

I have downloaded the latest GitHub version and will peruse the diff from
5.3.4 to check out the changes. However, given that many Lua optimisation
tips seem to be centered around not creating unnecessary tables I just
wanted to ask that question now.

Is there any timeline for the next GitHub update and/or RC1 of 5.4?
No rush or anything, I am just asking out of curiosity! :)

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Daurnimator
In reply to this post by Paige DePol
On 2 January 2018 at 10:05, Paige DePol <[hidden email]> wrote:
> The last update to GitHub was July 9th... I am going to guess that there are
> probably some significant changes from the code currently on GitHub then!

Latest history from Roberto has been pushed.
b1daa069..3ad33fde

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Paige DePol
Daurnimator <[hidden email]> wrote:

> On 2 January 2018 at 10:05, Paige DePol <[hidden email]> wrote:
>> The last update to GitHub was July 9th... I am going to guess that there are
>> probably some significant changes from the code currently on GitHub then!
>
> Latest history from Roberto has been pushed.
> b1daa069..3ad33fde

Thank you very much for that update, Daurnimator!

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Dibyendu Majumdar
In reply to this post by Daurnimator
On 3 January 2018 at 05:40, Daurnimator <[hidden email]> wrote:
> On 2 January 2018 at 10:05, Paige DePol <[hidden email]> wrote:
>> The last update to GitHub was July 9th... I am going to guess that there are
>> probably some significant changes from the code currently on GitHub then!
>
> Latest history from Roberto has been pushed.
> b1daa069..3ad33fde
>

Thank you. I notice that there are several implementation changes ...
I have been trying to merge the general GC changes to Ravi, but do not
want to pull additional changes just yet. Are there any GC bug fixes
amongst the changes since July?

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Roberto Ierusalimschy
In reply to this post by Paige DePol
> One thing I am curious about, Roberto, is the creation of tables to hold the
> vargs. Won't this add significant overhead for the table allocations vs just
> leaving the vargs on the stack?

That was the motivation for stack varargs. But "significant" is quite
relative. Moreover, stack varargs add an overhead (a quite small one, but
it is not zero) to all functions, even those not varargs. We do not see
vararg functions in critical paths often. Several (most?) vararg functions
have to create tables (or pay the price of 'select') anyway.

More important, table is the bread-and-butter of Lua. If we start to
demonise them, we are left with little else.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Roberto Ierusalimschy
In reply to this post by Dibyendu Majumdar
> Thank you. I notice that there are several implementation changes ...
> I have been trying to merge the general GC changes to Ravi, but do not
> want to pull additional changes just yet. Are there any GC bug fixes
> amongst the changes since July?

Not sure. Check the logs...

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Paige DePol
In reply to this post by Roberto Ierusalimschy
Roberto Ierusalimschy <[hidden email]> wrote:

>> One thing I am curious about, Roberto, is the creation of tables to hold the
>> vargs. Won't this add significant overhead for the table allocations vs just
>> leaving the vargs on the stack?
>
> That was the motivation for stack varargs. But "significant" is quite
> relative. Moreover, stack varargs add an overhead (a quite small one, but
> it is not zero) to all functions, even those not varargs. We do not see
> vararg functions in critical paths often. Several (most?) vararg functions
> have to create tables (or pay the price of 'select') anyway.
>
> More important, table is the bread-and-butter of Lua. If we start to
> demonise them, we are left with little else.
>
> -- Roberto

Very good point about tables, Roberto! I would guess that anyone looking
for performance probably wouldn't be using varargs on critical paths anyway.

I do like the much nicer syntax for using varargs vs select() as well!

~Paige


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Soni "They/Them" L.


On 2018-01-08 09:26 PM, Paige DePol wrote:

> Roberto Ierusalimschy <[hidden email]> wrote:
>
>>> One thing I am curious about, Roberto, is the creation of tables to hold the
>>> vargs. Won't this add significant overhead for the table allocations vs just
>>> leaving the vargs on the stack?
>> That was the motivation for stack varargs. But "significant" is quite
>> relative. Moreover, stack varargs add an overhead (a quite small one, but
>> it is not zero) to all functions, even those not varargs. We do not see
>> vararg functions in critical paths often. Several (most?) vararg functions
>> have to create tables (or pay the price of 'select') anyway.
>>
>> More important, table is the bread-and-butter of Lua. If we start to
>> demonise them, we are left with little else.
>>
>> -- Roberto
> Very good point about tables, Roberto! I would guess that anyone looking
> for performance probably wouldn't be using varargs on critical paths anyway.
>
> I do like the much nicer syntax for using varargs vs select() as well!
>
> ~Paige
>
>

Clearly you have not seen this yet.
http://blog.ionoclast.com/2015/05/the-future-of-firth-pre-alpha2-and-beyond/

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Jonathan Goble
On Mon, Jan 8, 2018 at 6:38 PM Soni "They/Them" L. <[hidden email]> wrote:
Clearly you have not seen this yet.
http://blog.ionoclast.com/2015/05/the-future-of-firth-pre-alpha2-and-beyond/

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.

MY EYES!

Seriously, that background makes the page completely unreadable.
Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Soni "They/Them" L.


On 2018-01-08 09:40 PM, Jonathan Goble wrote:

> On Mon, Jan 8, 2018 at 6:38 PM Soni "They/Them" L. <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Clearly you have not seen this yet.
>     http://blog.ionoclast.com/2015/05/the-future-of-firth-pre-alpha2-and-beyond/
>
>     --
>     Disclaimer: these emails may be made public at any given time,
>     with or without reason. If you don't agree with this, DO NOT REPLY.
>
>
> MY EYES!
>
> Seriously, that background makes the page completely unreadable.

Sorry. That page/website has seen better days.

Here you go:
http://web.archive.org/web/20160223150746/http://blog.ionoclast.com/2015/05/the-future-of-firth-pre-alpha2-and-beyond/

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Upcoming changes in Lua 5.4

Paige DePol
In reply to this post by Jonathan Goble
Jonathan Goble <[hidden email]> wrote:

> On Mon, Jan 8, 2018 at 6:38 PM Soni "They/Them" L. <[hidden email]> wrote:
> Clearly you have not seen this yet.
> http://blog.ionoclast.com/2015/05/the-future-of-firth-pre-alpha2-and-beyond/
>
> --
> Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.
>
> MY EYES!
>
> Seriously, that background makes the page completely unreadable.

Selecting all the text on the page made it considerably more readable!

However, I am unsure why you linked to that page in the first place?

The page author has quite an interesting project, but it appears they
are targeting LuaJIT... which if memory serves is already implementing
tables and other objects using memory arenas, which helps improve all
memory management quite a considerable amount.

~Paige


123