[ANN] Lua 5.4.0 (alpha-rc1) now available

classic Classic list List threaded Threaded
86 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Soni "They/Them" L.


On 2019-05-31 10:12 p.m., dyngeccetor8 wrote:

> On 30/05/2019 15.53, Rodrigo Azevedo wrote:
> > The performance (GC) is excellent again, thank you very much!
>
> Can confirm, garbage collector performance is good:
>
>      /tmp/lua-5.3.5/src $ time ./lua gc_benchmark.lua
>      Lua 5.3
>
>      real  0m44.687s
>      user  0m42.724s
>      sys  0m1.964s
>
>      /tmp/lua-5.4.0-alpha/src $ time ./lua gc_benchmark.lua
>      Lua 5.4
>
>      real  0m42.717s
>      user  0m40.920s
>      sys  0m1.800s
>
> -- gc_bechmark.lua (
> local num_iterations = 1e8
> local table_width = 1e8
>
> print(_VERSION)
>
> math.randomseed(os.time())
> -- collectgarbage('stop')
>
> local t = {}
> for i = 1, num_iterations do
>    t[math.random(table_width)] = {}
> end
> -- )
>
> -- Martin
>

is this measuring GC performance or RNG performance?

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Luiz Henrique de Figueiredo
> math.randomseed(os.time())

BTW, in 5.4, you don't need to do this anymore: the RNG is seeded
randomly when the math library is loaded.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

dyngeccetor8
In reply to this post by Soni "They/Them" L.
On 01/06/2019 04.31, Soni "They/Them" L. wrote:

> On 2019-05-31 10:12 p.m., dyngeccetor8 wrote:
> [...]
>> -- gc_bechmark.lua (
>> local num_iterations = 1e8
>> local table_width = 1e8
>>
>> print(_VERSION)
>>
>> math.randomseed(os.time())
>> -- collectgarbage('stop')
>>
>> local t = {}
>> for i = 1, num_iterations do
>>    t[math.random(table_width)] = {}
>> end
>> -- )
> [...]
>
> is this measuring GC performance or RNG performance?

GC performance. RNG execution times an orders faster.

-- Martin

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

nobody
In reply to this post by Soni "They/Them" L.
On 01/06/2019 03.31, Soni "They/Them" L. wrote:
>> local num_iterations = 1e8
>> local table_width = 1e8
> is this measuring GC performance or RNG performance?

Neither.

With these settings, roughly 75% of the slots get at most 1 assignment,
or only about 25% of the tables will become garbage. >98% of the slots
are assigned to less than 4 times – GC is bored.  And the RNG just gets
lost in the noise of 1e8 table creations and all that entails…

-- nobody

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

dyngeccetor8
On 01/06/2019 06.47, nobody wrote:

> On 01/06/2019 03.31, Soni "They/Them" L. wrote:
>>> local num_iterations = 1e8
>>> local table_width = 1e8
>> is this measuring GC performance or RNG performance?
>
> Neither.
>
> With these settings, roughly 75% of the slots get at most 1 assignment,
> or only about 25% of the tables will become garbage. >98% of the slots
> are assigned to less than 4 times – GC is bored.  And the RNG just gets lost in
> the noise of 1e8 table creations and all that entails…
>
> -- nobody
>

I'm using same code that were too slow for "5.4.0-work2".
Here is link to my message: https://marc.info/?l=lua-l&m=152985309714067

Point is this test took 76 seconds for Lua 5.4.0-work2
while on 5.3.4 it executed in 38 seconds.

Now execution times between "5.3.5" and "5.4.0-alpha"
almost match.

RNG here is used to avoid possible optimizations for
sequential indexes in virtual machine.

Also execution time spent not only for garbage collection
but also for tables allocation.

In other words: this is ad hoc test. No matter what it
does, it should not be slower in new Lua version.

-- Martin

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Andrew Gierth
In reply to this post by Roberto Ierusalimschy
>>>>> "Roberto" == Roberto Ierusalimschy <[hidden email]> writes:

 >> But I am working in an environment - embedding Lua into an existing
 >> large and complex program, namely the PostgreSQL backend - which has
 >> its own rules about error handling that simply do not allow this. In
 >> particular, I can't guarantee that closing a database cursor won't
 >> throw an error, and if it does, then I can't allow execution of
 >> arbitrary Lua code until after the subtransaction has been cleaned
 >> up (which requires propagating the error).

 Roberto> I fully agree that finalizers should not throw errors. If
 Roberto> closing a database cursor must throw an error and you can't
 Roberto> allow execution of arbitrary Lua code etc., then probably you
 Roberto> shouldn't close cursors during finalizers.

That's all very well, but that's just one example of a possible error. I
can't even free allocated memory without the (vanishingly rare, but
possible) chance of an error being thrown.

And then the other obvious problem is that I would have to prevent the
(untrusted) Lua code from creating its own finalizers.

 Roberto> As Gé pointed out, propagating a finalizer error is pointless.
 Roberto> You are interrupting code that has nothing to do with the
 Roberto> error, and cannot know how to handle it.

This. DOES. NOT. MATTER. to me. All that matters is that the error MUST
NOT BE IGNORED.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Lorenzo Donati-3
In reply to this post by Roberto Ierusalimschy
[snip]

>   --> lua: temp:3: attempt to close non-closable variable '(temporary)'
[snip]


Just a nitpick: although "closable" and "closeable" are reported by
English dictionaries as equivalent (I thought it was a spelling mistake
at first but I double checked before posting), I somewhat prefer
"closeable" and I feel it is more common. Maybe native speakers could
step-in and have their say.


> -- Roberto
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Rodrigo Azevedo
In reply to this post by Roberto Ierusalimschy
Em qui, 30 de mai de 2019 às 12:48, Roberto Ierusalimschy
<[hidden email]> escreveu:

>
> > If true, then why not consider to default (mark) *all* 'local' to be
> > collected as soon as possible (when out of scope), at the given
> > reversed order, by the garbage collector (call __gc)?
>
> function newfile (name)
>   local f = io.open(name)
>   return f
> end   -- oops; 'f' goes out of scope, file is closed.
>
> Again, what goes out of scope are variables, not values.
>
> -- Roberto

It's OK, I'm not arguing, only try to understand <toclose>.

Is <toclose> action '__toclose' triggered by an 'out of scope' event?
then it does not need necessarily 'to be closed' something, right?

--- #####################################################
-- aggregate.lua
-- BEGIN

local sum = setmetatable({0},{
  __close = function(t) t[1] = t[1] + 1 end,
  __tostring = function(t) return t[1] end
})

local s1 = function()
  local <toclose> s = sum
  s[1] = s[1] + 1
  return s[1]
end -- s out of scope

print(s1(),s1(),sum)
-- END

rodrigo@rodrigo-X510UAR:~$ lua aggregate.lua
1    3    4
rodrigo@rodrigo-X510UA
--- #######################################################

It is very interesting and useful in fact, but '<toclose>' seems a
confusing word here.

Moreover, <const> is easily grasped, but <toclose> is not clear at all
depending on the context, as above, it is difficult to understand a
programmer's mind :)

Could it be possible to change the word '<toclose>' to something more
related to the actual behavior?

Thanks again for the amazing work!

--
Rodrigo Azevedo Moreira da Silva

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Luiz Henrique de Figueiredo
In reply to this post by Luiz Henrique de Figueiredo
luac has been rewritten to account for the new VM instructions.
We also welcome feedback on the listings output by luac -l -l.
Thanks.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Kim Alvefur
On Sat, Jun 01, 2019 at 09:52:17AM -0300, Luiz Henrique de Figueiredo wrote:
> luac has been rewritten to account for the new VM instructions.
> We also welcome feedback on the listings output by luac -l -l.
> Thanks.
>
Nice! Listings of constants, locals and upvalues are very handy.

> 14 [55] CALL 5 3 2 ; 2 in 1 out

Listing argument and return value counts are also very nice.

--
Kim "Zash" Alvefur

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Luiz Henrique de Figueiredo
> > We also welcome feedback on the listings output by luac -l -l.

> Nice! Listings of constants, locals and upvalues are very handy.

This is not new.

> Listing argument and return value counts are also very nice.

This is new.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Pierre Chapuis
> > Listing argument and return value counts are also very nice.
>
> This is new.

Sadly I do not have much time to play with this release yet, but I wanted to say I am very happy to see this as well. It is something I have wished for a few times in the past.

I am very happy that we get <close> as well, of course. :)

--
Pierre Chapuis

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Patrick Donnelly
In reply to this post by Luiz Henrique de Figueiredo
On Wed, May 29, 2019 at 5:38 PM Luiz Henrique de Figueiredo
<[hidden email]> wrote:
>
> Lua 5.4.0 (alpha-rc1) is now available for testing at
>         http://www.lua.org/work/lua-5.4.0-alpha-rc1.tar.gz

\o/ toclose! (I'll echo that a more Luaish syntax would be preferred
but I trust the Author's judgement on this.)

I thought I'd also share that another useful local qualifier would be
<static>. This would be a variable scoped as normal but is initialized
once when the surrounding closure is created. i.e.:

function foo(...)
  local <static> cache = {}
end

is syntactic sugar for:

do
  local cache = {}
  function foo(...)
  end
end

--
Patrick Donnelly

Reply | Threaded
Open this post in threaded view
|

Re: Errors in finalizers (was: [ANN] Lua 5.4.0 (alpha-rc1) now available)

David Heiko Kolf-2
In reply to this post by Andrew Gierth
Andrew Gierth wrote:
>>>>>> "Roberto" == Roberto Ierusalimschy <[hidden email]> writes:

>  Roberto> As Gé pointed out, propagating a finalizer error is pointless.
>  Roberto> You are interrupting code that has nothing to do with the
>  Roberto> error, and cannot know how to handle it.
>
> This. DOES. NOT. MATTER. to me. All that matters is that the error MUST
> NOT BE IGNORED.

But this is exactly what could happen when you throw the error in a
finalizer: The code might currently be in a pcall which doesn't expect
your finalizer error and therefor also doesn't react to it -- the code
following the pcall would continue as usual, you just skipped some
random code.

An example from actual code I have written:

  pcall(function ()
      local debug = require "debug"
      -- I am now doing stuff with the debug library
  end)
  -- At this point I continue, assuming that the worst that could
  -- have happened might be that I didn't get access to the debug
  -- library. I don't even check the pcall result.

If an error during a finalizer occurs I would probably store it in some
list and regularly check that list (for example in library functions
that you know should not run as normal when a finalizer failed previously).

Best regards

David

Reply | Threaded
Open this post in threaded view
|

Re: Errors in finalizers

Andrew Gierth
>>>>> "David" == David Heiko Kolf <[hidden email]> writes:

 >> This. DOES. NOT. MATTER. to me. All that matters is that the error
 >> MUST NOT BE IGNORED.

 David> But this is exactly what could happen when you throw the error
 David> in a finalizer: The code might currently be in a pcall which
 David> doesn't expect your finalizer error and therefor also doesn't
 David> react to it -- the code following the pcall would continue as
 David> usual, you just skipped some random code.

Except that this can't happen because I already replaced the visible
pcall() function with one that does the necessary error recovery:

https://github.com/pllua/pllua-ng/blob/master/src/error.c#L718

My complaint about ignoring errors in finalizers is, like my earlier
complaint about function values for <toclose>, based on the fact that it
results in errors being caught in a way that the sandboxing code can't
control.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Lorenzo Donati-3
In reply to this post by Patrick Donnelly
On 04/06/2019 21:27, Patrick Donnelly wrote:

> On Wed, May 29, 2019 at 5:38 PM Luiz Henrique de Figueiredo
> <[hidden email]> wrote:
>>
>> Lua 5.4.0 (alpha-rc1) is now available for testing at
>>         http://www.lua.org/work/lua-5.4.0-alpha-rc1.tar.gz
>
> \o/ toclose! (I'll echo that a more Luaish syntax would be preferred
> but I trust the Author's judgement on this.)
>
> I thought I'd also share that another useful local qualifier would be
> <static>. This would be a variable scoped as normal but is initialized
> once when the surrounding closure is created. i.e.:
>
> function foo(...)
>   local <static> cache = {}
> end
>
> is syntactic sugar for:
>
> do
>   local cache = {}
>   function foo(...)
>   end
> end
>
Wow, that seems like a very nice idea [1].

I use that idiom quite frequently, not only for caches, but also for
implementation functions, for example:

local fun
do
   local function fun_impl(...)
     -- maybe this function is also recursive
   end
   --[[local]] function fun(...)
     -- ...
       fun_impl(...)
     -- ...
   end
end


This could be abbreviated simply by ([1]):

local function fun(...)
   local <static> fun_impl = function(...)
   end
   -- ...
     fun_impl(...)
   -- ...
end

Note that the outer function is local so in this case the inner <static>
local would define an upvalue fun_impl which is "private" for the
function, but the outer function "name" wouldn't need that verbose
additional local declaration outside a do-end block ([1]).


BTW, I'm still struggling to come up with a better syntax. A keyword
based approach would be nicer, but it causes ambiguities in the grammar,
especially when more "attributes" are used (as pointed out by
Luiz/Roberto somewhere else).

So I try to decide which is better between the proposed syntax:

local <toclose, static, nice, helpful, wonderful> resource = ....

or something like:

local @toclose @static @nice @helpful @wonderful resource = ....

I somewhat prefer the latter, although the @ sigil is a bit ugly. It has
the advantage that when you visually parse it it is clear that those
words are attributes, whereas the <...> syntax may become obscure when
you quickly skim on the code. The angle brackets may be mistaken for
operators and if the attribute list is long they tend to be swamped by
the words and commas, etc.

Moreover how would you split that thing if the need arise?
Compare:


local < toclose, static,
   nice, helpful, wonderful > resource = ....

vs.

local @toclose @static
   @nice @helpful @wonderful resource = ....

In the first case the possibility to introduce spaces "inside" the angle
brackets could be a source of confusion, too (not to mention spaces at
commas in different coding styles). And maybe also could render the
grammar ambiguous.

The sigil approach marks in an unequivocal ways the attributes,
regardless of indentation, spacing, etc..

The more I see it the more I'd prefer the second approach.



Cheers!

-- Lorenzo


[1] too late at night here to think about possible drawbacks or pitfalls ;-)


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Thijs Schreijer
In reply to this post by Patrick Donnelly


> On 4 Jun 2019, at 21:27, Patrick Donnelly <[hidden email]> wrote:
>
> On Wed, May 29, 2019 at 5:38 PM Luiz Henrique de Figueiredo
> <[hidden email]> wrote:
>>
>> Lua 5.4.0 (alpha-rc1) is now available for testing at
>>        http://www.lua.org/work/lua-5.4.0-alpha-rc1.tar.gz
>
> \o/ toclose! (I'll echo that a more Luaish syntax would be preferred
> but I trust the Author's judgement on this.)
>
> I thought I'd also share that another useful local qualifier would be
> <static>. This would be a variable scoped as normal but is initialized
> once when the surrounding closure is created. i.e.:
>
> function foo(...)
>  local <static> cache = {}
> end
>
> is syntactic sugar for:
>
> do
>  local cache = {}
>  function foo(...)
>  end
> end
>
> --
> Patrick Donnelly
>

What happens if I do:

function foo(…)
 print(cache)
 local <static> cache = “something”
 cache = cache .. ”.”
end

foo()
foo()

I prefer the traditional Lua idiom because it is more explicit.


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Thijs Schreijer
In reply to this post by Roberto Ierusalimschy


> On 30 May 2019, at 17:43, Roberto Ierusalimschy <[hidden email]> wrote:
>
>>
>>
>>> On 30 May 2019, at 02:38, Luiz Henrique de Figueiredo <[hidden email]> wrote:
>>>
>>> Lua 5.4.0 (alpha-rc1) is now available for testing at
>>> http://www.lua.org/work/lua-5.4.0-alpha-rc1.tar.gz
>>
>>> From the what I read about “to-be-closed” [1], the difference with an “__gc” method is that is doesn’t wait for the GC cycle to kick in, but it is invoked immediately when the variable goes out-of-scope. So it has deterministic timing over “__gc”.
>>
>> If this is the case (please correct me if I’m wrong). Then why isn’t it just a flag on a meta-table? The meta table must be there already (since a requirement is to have an __gc method). One could even consider extending “__mode” to carry an extra character; "__mode = ‘kvc’ “.
>
> What goes out of scope are variables, not values. What get metatables
> are values, not variables.
>
> -- Roberto
>

Thx for the explanation, that makes sense.
Thijs
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Hakki Dogusan-3
In reply to this post by Lorenzo Donati-3
Hi,

On 5.06.2019 00:27, Lorenzo Donati wrote:

> On 04/06/2019 21:27, Patrick Donnelly wrote:
>> On Wed, May 29, 2019 at 5:38 PM Luiz Henrique de Figueiredo
>> <[hidden email]> wrote:
>>>
>>> Lua 5.4.0 (alpha-rc1) is now available for testing at
>>>         http://www.lua.org/work/lua-5.4.0-alpha-rc1.tar.gz
>>
>> \o/ toclose! (I'll echo that a more Luaish syntax would be preferred
>> but I trust the Author's judgement on this.)
>>
>> I thought I'd also share that another useful local qualifier would be
>> <static>. This would be a variable scoped as normal but is initialized
>> once when the surrounding closure is created. i.e.:
>>
> [snip]
> BTW, I'm still struggling to come up with a better syntax. A keyword
> based approach would be nicer, but it causes ambiguities in the grammar,
> especially when more "attributes" are used (as pointed out by
> Luiz/Roberto somewhere else).
>
> So I try to decide which is better between the proposed syntax:
>
> local <toclose, static, nice, helpful, wonderful> resource = ....
>
> or something like:
>
> local @toclose @static @nice @helpful @wonderful resource = ....
>

What about:

local [scoped, static] resource = ...
[local] mylocalvar = ...
[ongc=myfunction] myresource = ...

Is it ambiguous?


> I somewhat prefer the latter, although the @ sigil is a bit ugly. It has
> the advantage that when you visually parse it it is clear that those
> words are attributes, whereas the <...> syntax may become obscure when
> you quickly skim on the code. The angle brackets may be mistaken for
> operators and if the attribute list is long they tend to be swamped by
> the words and commas, etc.
>
> Moreover how would you split that thing if the need arise?
> Compare:
>
>
> local < toclose, static,
>    nice, helpful, wonderful > resource = ....
>
> vs.
>
> local @toclose @static
>    @nice @helpful @wonderful resource = ....
>
> In the first case the possibility to introduce spaces "inside" the angle
> brackets could be a source of confusion, too (not to mention spaces at
> commas in different coding styles). And maybe also could render the
> grammar ambiguous.
>
> The sigil approach marks in an unequivocal ways the attributes,
> regardless of indentation, spacing, etc..
>
> The more I see it the more I'd prefer the second approach.
>
>
>
> Cheers!
>
> -- Lorenzo
>
>
> [1] too late at night here to think about possible drawbacks or pitfalls
> ;-)
>
>
>

--
Regards,
Hakki Dogusan

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha-rc1) now available

Patrick Donnelly
In reply to this post by Thijs Schreijer
On Wed, Jun 5, 2019 at 12:14 AM Thijs Schreijer <[hidden email]> wrote:

> > On 4 Jun 2019, at 21:27, Patrick Donnelly <[hidden email]> wrote:
> >
> > On Wed, May 29, 2019 at 5:38 PM Luiz Henrique de Figueiredo
> > <[hidden email]> wrote:
> >>
> >> Lua 5.4.0 (alpha-rc1) is now available for testing at
> >>        http://www.lua.org/work/lua-5.4.0-alpha-rc1.tar.gz
> >
> > \o/ toclose! (I'll echo that a more Luaish syntax would be preferred
> > but I trust the Author's judgement on this.)
> >
> > I thought I'd also share that another useful local qualifier would be
> > <static>. This would be a variable scoped as normal but is initialized
> > once when the surrounding closure is created. i.e.:
> >
> > function foo(...)
> >  local <static> cache = {}
> > end
> >
> > is syntactic sugar for:
> >
> > do
> >  local cache = {}
> >  function foo(...)
> >  end
> > end
> >
> > --
> > Patrick Donnelly
> >
>
> What happens if I do:
>
> function foo(…)
>  print(cache)
>  local <static> cache = “something”
>  cache = cache .. ”.”
> end
>
> foo()
> foo()
>
> I prefer the traditional Lua idiom because it is more explicit.

I suppose the cache variable would not be in scope at the point of
"print(cache)" which makes the "syntactic sugar" analogy weaker. This
can be caught during parsing pretty easily. The other thing to
consider is access to locals to the function by the expression
assigning to the static variable. e.g.

function foo(a)
  local <static> cache = a
end

^ That should clearly be invalid code and the compiler should complain
appropriately. OTOH, having access to upvalues to foo should be
acceptable:

local _ENV = {}
function foo(a)
  local <static> rab = _ENV.bar
end

^ OK.

--
Patrick Donnelly

12345