[ANN] Lua 5.4.0 (beta-rc2) now available

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] Lua 5.4.0 (beta-rc2) now available

Luiz Henrique de Figueiredo
Lua 5.4.0 (beta-rc2) is now available for testing at
        http://www.lua.org/work/lua-5.4.0-beta-rc2.tar.gz

The checksums are
        MD5 961e2692a10a4a3c6fe80086e4cbefd5  -
        SHA1 5d018de9b1f69b7e334dc3fa43b815669e103880  -

This is an beta version. Some details may change in the final version.

The main changes in Lua 5.4.0 are listed at
        http://www.lua.org/work/doc/#changes

An updated reference manual is included and also available at
        http://www.lua.org/work/doc

The complete diffs from beta-rc1 to beta-rc2 are available at
        http://www.lua.org/work/diffs-lua-5.4.0-beta-rc1-rc2.html
        http://www.lua.org/work/diffu-lua-5.4.0-beta-rc1-rc2.html

A test suite is available at
        http://www.lua.org/work/lua-5.4.0-beta-tests.tar.gz

If your platform is a common Unix-like platform, just do
        make guess
The Makefile will guess your platform using uname and build Lua for it.
We welcome feedback on this, which is new, especially more uname targets
with explicit rules and fixes for existing ones.

We also welcome feedback on the listings output by luac -l -l, because
luac has been rewritten to account for the new VM instructions.

All feedback welcome. Thanks.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Luiz Henrique de Figueiredo
Lua 5.4.0 (beta-rc2) remains available for testing at http://www.lua.org/work/

Please take a look before we freeze Lua 5.4.0 (beta), which will
happen next week if no glitches are reported.

All feedback is welcome. Thanks.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Paul Ducklin
The lua.1 man page in the doc/ directory hasn’t been updated. Notably, it doesn’t mention the -W option.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Zelo
In reply to this post by Luiz Henrique de Figueiredo
I wrote about a glitch last week, but it never got to the mailing list.
Don't know if this reply gets to you. :/

- Jörg

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Zelo
In reply to this post by Luiz Henrique de Figueiredo
Hi,

another problem/difference I found while testing Lua 5.4:
Loading modules on threads has now a very limited require-depth.
Requiring t1.lua which itself requires t2.lua and so on stops
after 5 levels with "C stack overflow". The number of frames when
the error happens is in my case 53 and so quite low.

In Lua 5.3 the limit was much higher (>100)

- Jörg
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Roberto Ierusalimschy
> another problem/difference I found while testing Lua 5.4:
> Loading modules on threads has now a very limited require-depth.
> Requiring t1.lua which itself requires t2.lua and so on stops
> after 5 levels with "C stack overflow". The number of frames when
> the error happens is in my case 53 and so quite low.

That is weird. The number of frames should be 2*n: for each module,
it needs one frame for 'require' and one frame for the module itself.
I tested with 10 nested modules (which used 20 frames) and did not
find any problem.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Zelo

That is weird. The number of frames should be 2*n: for each module,
it needs one frame for 'require' and one frame for the module itself.
I tested with 10 nested modules (which used 20 frames) and did not
find any problem.

The following bash script results in this output:

---
1
2
3
4
5
PANIC: unprotected error in call to Lua API (C stack overflow)
---

With Lua 5.3 it goes up to 197 when there are enough t*-files.

Bash-Script:

 LUA_DIR=...
 mkdir tmp
 cd tmp

 for no in $(seq 1 10); do
   cat > t$no.lua <<EOF
 print( "$no" )
 require "t$(($no+1))"
 EOF
 done

 cat > t.c <<EOF
 #include <lua.h>
 #include <lauxlib.h>

 int main( int argc, char** argv )
 {
   lua_State* Lg = luaL_newstate();
   luaL_openlibs( Lg );
   lua_State* L = lua_newthread( Lg );

   lua_rawgeti( L, LUA_REGISTRYINDEX, LUA_RIDX_GLOBALS );
   lua_getfield( L, -1, "require" );
   lua_pushstring( L, "t1" );
   lua_call( L, 1, 0 );
   return 0;
 }
 EOF

 gcc t.c -o t -I$LUA_DIR/include $LUA_DIR/lib/liblua.a -lm -ldl
 ./t
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Roberto Ierusalimschy
> > That is weird. The number of frames should be 2*n: for each module,
> > it needs one frame for 'require' and one frame for the module itself.
> > I tested with 10 nested modules (which used 20 frames) and did not
> > find any problem.
> >
>
> The following bash script results in this output:
>
> [...]

Neither the main thread nor coroutines has this behavior, but
independent threads are indeed being created with a very small
stack limit. The following change should fix the issue:

--- a/lstate.c
+++ b/lstate.c
@@ -288,3 +288,2 @@ static void preinit_thread (lua_State *L, global_State *g) {
   L->errorJmp = NULL;
-  L->nCcalls = CSTACKTHREAD;
   L->hook = NULL;
@@ -329,2 +328,3 @@ LUA_API lua_State *lua_newthread (lua_State *L) {
   preinit_thread(L1, g);
+  L1->nCcalls = getCcalls(L);
   L1->hookmask = L->hookmask;


-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

nobody
In reply to this post by Luiz Henrique de Figueiredo
On 11/10/2019 19.27, Luiz Henrique de Figueiredo wrote:
> Lua 5.4.0 (beta-rc2) remains available for testing at
> http://www.lua.org/work/
>
> Please take a look before we freeze Lua 5.4.0 (beta), which will
> happen next week if no glitches are reported.
>
> All feedback is welcome. Thanks.

I went through the whole manual… took a while (sorry), real life got in
the way.

(According to the checksums, there were no changes from beta-rc2 to
beta, so this should still be current?)

I probably won't have time to dig further anytime soon. =/

-- nobody

## 2.4 (metamethods) / __index, same for __newindex / __call

[WORDING]

> `__index`: The indexing access operation `table[key]`. This event
> happens when `table` is not a table or when `key` is not present in
> `table`. >>>The metamethod is looked up in `table`.<<<

That last bit is very confusing.  To me, this reads as if a failing
`t[k]` looks for `__index` in `t`, not the metatable.

Better: …is looked up in `table`'s metatable. / `func`'s metatable.

## 2.4 (metamethods) / __name

[OMISSION]

> (The entry __name, when it contains a string, is used by some
> error-reporting functions to build error messages.)

`tostring` also uses it, so maybe "…, is used by `tostring` and some…"

## 2.5.2 (generational garbage collection)

[OMISSION]

It's not clear whether the generational collections are also done
incrementally or all at once.  (This matters for interactive programs
like games.)  That this isn't mentioned at all (no parameters for step
size etc. apart from minor/major thresholds) makes me think that I'll
have to deal with random long pauses from major collections.

Adding an explicit statement one way or the other would help, I think.

## 3.3.7 (local declarations)

[IMPLEMENTATION]

> A list of variables can contain at most one to-be-closed variable.

This is somewhat surprising and makes it harder to work with functions
that return multiple close-worthy resources.  The example that I ran
into is input/output pairs like

    local i<close>, o<close> = pipe(…)

Dealing with this requires some extra shuffling and local variables, and
you can't stick those into an inner scope because they need to be
visible for the <close> variables and those can't be pre-declared… and
so you end up with

    local i<close>,_o = pipe(…) ; local o<close> = _o

With the current constraint, the only reasonable way I see for dealing
with multiple things from a function that need to be <close>d is to pack
everything together so that there's only one thing to <close> and then
unpack that into the parts that you actually work with…

    local p<close> = pipe(…)
    local i, o = p.i, p.o

That's okay-ish, but if the constraint of only one <close> per local can
be removed, that'd simplify things somewhat.

## 3.4.3 (coercions and conversions)

[WORDING]

> The string library uses metamethods that try to coerce strings to
> numbers in all arithmetic operations. If the conversion fails, the
> library calls the metamethod of the other operand (if present) or it
> raises an error.
Actually it only tries the metamethod of the right side, to prevent a
"ping-pong" loop with some other metamethod that applies the same logic.

It might be useful to be explicit about this so that other programmers
reading the manual understand the potential problem and also write their
metamethods in such a loop-preventing way.

## 3.4.11 (function definitions)

[TYPO]

> Then, we have the following mapping from arguments to parameters and
>  to the vararg expression:
>     CALL            PARAMETERS
>     f(3)             a=3, b=nil
>     …                …
'PARAMETERS' is one space to the left of the column.

## 4.8

[CSS]

With a narrow window, the [-o,+p,x] marker may overlap the function,
making both unreadable.

I don't know how to fix this – does anyone on the list have experience
with that?

### lua_call

[WORDING]

> To do a call you must use the following protocol: first, the value to
> be called is pushed onto the stack; then, the arguments to the call
> are pushed in direct order; that is, the first argument is pushed
> first. Finally you call lua_call; nargs is the number of arguments
> that you pushed onto the stack. When the function returns, all
> arguments and the function value are popped and the function results
> are pushed onto the stack. …

Suboptimal choice of words; suggestion:

first, the value to be called is pushed onto the stack
--> first, the function value is pushed onto the stack

(The purpose is IMHO clearer than "value to be called", and "function
value" instead of "function" still expresses the generality.)

and the function results are pushed onto the stack.
--> and the call results are pushed onto the stack.

(Functions don't do anything, calling functions does.)

## 6 (the standard library)

[COMMENT]

> Currently, fail is equal to nil, but that may change in future
> versions.

This would worsen confusion of values and in-band error signaling,
making it harder to write generic higher-order / glue code or DSLs.

When creating other computational substrates on / within Lua, these
usually don't contain the "Lua-ism" `nil` (and so that can freely be
used on the Lua / implementation side to signal errors).  Booleans
however are often needed and there's no alternative, as (apart from
`nil`) there's no false value other than `false`, which means you have
to reuse primitive booleans unless you're willing to get rid of
if/while/and/… in your new substrate (DSL / library / …).

Depending on the computational patterns used, things might usually
continue to work (e.g. if you're chaining functions and always pass a
state value along first like `{f,g,…}( self, ... )`) or it could break
completely for booleans (e.g. when using varargs as a stack – think
embedded forth – where pushing false becomes indistinguishable from
reporting failure in-band, and so you'd have to use a different
reporting pattern and put in more work to wrap the stdlib).

## 6.1 (basic functions)

### warn

[IMPLEMENTATION]

> By convention, a one-piece message starting with '@' is intended to
> be a control message [….] the standard warning function in Lua […]
> ignores unknown control messages.
It might be better to warn about unknown "@…" instead of completely
ignoring them.

So instead of

 > warn "@foo"
--> (nothing)

you'd get

 > warn "@foo"
--> Lua warning: invalid warning (starts with '@'): @foo

(But people can also wrap warn to always become warn( "", ... ), if
they're aware of this and don't need control messages…)

### xpcall

[OMISSION]

It'd be a good idea to repeat the note from `lua_pcall` that the message
handler is not run for all errors.

Not all Lua users know C / read the C API part of the manual.  (I have
seen invalid code that relied on xpcall always calling the message
handler to do cleanup.)

This could be combined with a nudge towards the new deterministic
cleanup feature, e.g.:

> This function is similar to [`pcall`], except that it sets a new
> message handler `msgh`.  (Note that the message handler is not called
> for some error types, use [to-be-closed variables] if you want to
> release resources in case of errors.)
Or, simpler

> This function is similar to [`pcall`], except that it sets a new
> message handler `msgh`.  (See [`lua_pcall`] for caveats.)

## 6.3 (modules)

### require / package.searchers

[IMPLEMENTATION]

> If [a searcher function] cannot find the module, it returns a string
> explaining why (or nil if it has nothing to say).

The error descriptions are concatenated without separators.  The default
searchers explicitly prefix their messages with "\n\t", but not all
custom loaders (e.g. Luarocks' loader) do this, which results in
weirdly-formatted error messages.

 > require "X"
stdin:1: module 'X' not found:No LuaRocks module found for X
        no field package.preload['X']
        no file './X.lua'
        […]

It'd be better if `findloader` in loadlib.c (which is already using a
`luaL_Buffer` for concatenation) would take care of adding the
separator, so that the searchers can just return the error message
instead of (separator + message).

## 6.4 (string manipulation)

### string.format

[IMPLEMENTATION]

It's great to see that '%q' now supports all primitive values, which
makes it even more useful for serialization.

(1) It might be a good idea to also have it accept other values for '%q'
if they have __tostring.  (You can't `("%q"):rep( #t, "," )` without
looking at the values.  Even if you did a tostring() pass first on just
those values that need it, they'd end up quoted as strings, which would
be wrong.  So you actually have build the format "manually" and use '%q'
or '%s' depending on the type, which takes A LOT more code than just the
:rep() call… OTOH, if '%q' accepts __tostring, dumping an array could be
as easy as `("%q"):rep( #t, "," ):format( table.unpack( t ) )`.)

(2) It might be better if newlines are escaped as "\n" instead of "\
", which would make '%q' more suitable for formats other than Lua (e.g.
line-based data files or other programming languages).  (One can
manually :gsub this afterwards, but that's an extra step… The only
advantage for '\<lf>' over '\n' I can see might be a slight increase in
readablility _within_ a single string, but if you're dumping many
strings, readability suffers as the large-scale structure (e.g. nested
tables / lines / indentation) is obscured by embedded newlines.)

### 6.4.2 (format strings for pack and unpack)

[WORDING]

> A format string is a sequence of conversion options. The conversion
> options are as follows […] ' ': (empty space) ignored
"empty space" is wrong(?), maybe 'space character'?

[EOF]

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Roberto Ierusalimschy
> >The string library uses metamethods that try to coerce strings to numbers
> >in all arithmetic operations. If the conversion fails, the library calls
> >the metamethod of the other operand (if present) or it raises an error.
> Actually it only tries the metamethod of the right side, to prevent a
> "ping-pong" loop with some other metamethod that applies the same logic.

That explanation is not quite right. It tries only the right side
because Lua already tried the left side. If the left operand had a
metamethod, Lua would have called it instead of this [right operand]
one that is running.


> It might be useful to be explicit about this so that other programmers
> reading the manual understand the potential problem and also write their
> metamethods in such a loop-preventing way.

In general, other metamethods do not need (and do not have) this
behavior. (At least mine don't.) Strings only act this way to be
compatible with previous versions, where they were builtin and
therefore did not follow a "metamethod logic".

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by nobody
> >A list of variables can contain at most one to-be-closed variable.
>
> This is somewhat surprising and makes it harder to work with functions
> that return multiple close-worthy resources. [...]

In our view, a function that returns multiple close-worthy resources
is a bad idea. While creating those multiple close-worthy resources,
if one of them fails, the function would have to close the previous
ones without the help of to-be-closed variables. It seems a recipe
for sloppy programming.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (beta-rc2) now available

Roberto Ierusalimschy
In reply to this post by nobody
> It's not clear whether the generational collections are also done
> incrementally or all at once.  (This matters for interactive programs
> like games.)  That this isn't mentioned at all (no parameters for step
> size etc. apart from minor/major thresholds)

Major collections in generational mode are stop-the-world collections.
We will add a note about that.


> makes me think that I'll have to deal with random long pauses from
> major collections.

You can mostly avoid them with a large major multiplier. (You can
force a full collection manually, from time to time, when it is
ok to stop the world.) Or you can use the incremental mode, which
can be more appropriate for that kind of application.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

situations that need manual metamethod retry (Re: [ANN] Lua 5.4.0 (beta-rc2) now available)

nobody
In reply to this post by Roberto Ierusalimschy
On 22/10/2019 15.25, Roberto Ierusalimschy wrote:
>>> The string library uses metamethods that try to coerce strings to numbers
>>> in all arithmetic operations. If the conversion fails, the library calls
>>> the metamethod of the other operand (if present) or it raises an error.
>> Actually it only tries the metamethod of the right side, to prevent a
>> "ping-pong" loop with some other metamethod that applies the same logic.
> In general, other metamethods do not need (and do not have) this
> behavior. (At least mine don't.)
OK, agree.

Just for completeness:  It *is* necessary to manually retry the right
side's metamethod when working with multiple levels of partially
compatible types a < b < c, e.g. the string situation of number < string
< (whatever a user may define), or a different (maybe simpler?) example:
numbers < complex numbers < vectors. If vectors treat non-vectors as
constants, number-anything works as usual or Lua uses the right side's
metamethod, vector-complex / vector-number work as expected, but
complex-vector needs to manually fall back to the right side (just like
Lua does for strings).

-- nobody