[ANN] Lua 5.4.3 (rc1) now available

[Luiz Henrique de Figueiredo]

Lua 5.4.3 (rc1) is now available for testing at
        http://www.lua.org/work/lua-5.4.3-rc1.tar.gz

The checksums are
        MD5 b8b5b2d309cdfa6c217dfb0ec64fd2c6  -
        SHA1 8ba37aa3fa89724bbcf500af10b861f09dacfdd4  -

Lua 5.4.3 fixes all bugs listed in
        http://www.lua.org/bugs.html#5.4.2

Lua 5.4.3 also contains several internal improvements and
includes a revised reference manual:
        http://www.lua.org/work/doc/

The complete diffs from Lua 5.4.2 are available at
        http://www.lua.org/work/diffs-lua-5.4.2-lua-5.4.3.html
        http://www.lua.org/work/diffu-lua-5.4.2-lua-5.4.3.html

We thank everyone for their feedback on Lua 5.4 till now.

All feedback welcome. Thanks.
--lhf

[Gé Weijers]

Good morning, Luiz,

Thanks for the update, I'll be running some tests this week.

One small issue I ran into:

luaL_archcheck and luaL_argexpected now use l_likely.

Because l_likely does not put its arguments in parentheses the following change removed the parentheses from the argument 'cond' after macro expansion, which gave me some warnings in my code. 

It did not break anything, but someone might possibly be bitten by this if they use the ternary operator (?:) without parentheses as the argument 'cond'. Probably not great code, but you never know.

Current definition:

 #define luaL_argcheck(L, cond,arg,extramsg)    \
       ((void)(l_likely(cond) || luaL_argerror(L, (arg), (extramsg))))

Perhaps l_likely should be defined as "#define l_likely(x) (x)", or luaL_argcheck as:

#define luaL_argcheck(L, cond, arg, extramsg) \

    ((void)(l_likely((cond)) || luaL_argerror(L, (arg), (extramsg))))

This would bring back the parentheses after macro expansion.

Gé 

--

[Roberto Ierusalimschy]

> Because l_likely does not put its arguments in parentheses the following
> change removed the parentheses from the argument 'cond' after macro
> expansion, which gave me some warnings in my code.
>
> [...]
>
> Perhaps l_likely should be defined as "#define l_likely(x) (x)", or
> luaL_argcheck as:
>
> [...]

We must correct the definition of l_likely to "#define l_likely(x) (x)",
as this problem may arise in other places.

Thanks for the feedback,

-- Roberto

[Gé Weijers]

On Wed, Mar 3, 2021 at 10:41 AM Roberto Ierusalimschy <[hidden email]> wrote:

We must correct the definition of l_likely to "#define l_likely(x) (x)",
as this problem may arise in other places.

One question: why are the definitions for l_likely and l_unlikely that use the GNU builtin "__builtin_expect" only available by default in the Lua core and libraries, and not in C extension modules?

Thanks,

Gé

--

Open the LUAI_ASSERT macro, and then run this code, it will get an assertion failure.

On Thu, Mar 4, 2021 at 4:47 AM Gé Weijers <[hidden email]> wrote:

On Wed, Mar 3, 2021 at 10:41 AM Roberto Ierusalimschy <[hidden email]> wrote:

We must correct the definition of l_likely to "#define l_likely(x) (x)",
as this problem may arise in other places.

One question: why are the definitions for l_likely and l_unlikely that use the GNU builtin "__builtin_expect" only available by default in the Lua core and libraries, and not in C extension modules?

Thanks,

Gé

--

Gé

--

[Gé Weijers]

On Wed, Mar 3, 2021 at 5:50 PM actboy168 <[hidden email]> wrote:

Open the LUAI_ASSERT macro, and then run this code, it will get an assertion failure.

I'm not seeing this on MacOS (neither x86_64 nor ARM v8), but I do see this on Debian Linux 10.8 (for x86_64)

--

[Gé Weijers]

On Wed, Mar 3, 2021 at 8:17 PM Gé Weijers <[hidden email]> wrote:

On Wed, Mar 3, 2021 at 5:50 PM actboy168 <[hidden email]> wrote:

Open the LUAI_ASSERT macro, and then run this code, it will get an assertion failure.

I'm not seeing this on MacOS (neither x86_64 nor ARM v8), but I do see this on Debian Linux 10.8 (for x86_64)

The issue is on line 66 of ldebug.c. The bug is in the assertion, the value of 'i' in this test is -1, which makes the second test of the assertion use the index -1 for the array f->abslineinfo. I don't believe there's anything wrong with the code beyond the assertion.

I changed the assertion to:

    lua_assert(i == -1 || (i >= 0 && i < f->sizeabslineinfo && f->abslineinfo[i].pc <= pc));.

and all seems to be fine now.

Gé

[Vinicius Jarina]

I guess the https://www.lua.org/work/lua-5.4.3-tests.tar.gz is missing tracegc.lua ?

On Wed, Mar 3, 2021 at 5:30 AM Luiz Henrique de Figueiredo <[hidden email]> wrote:

Lua 5.4.3 (rc1) is now available for testing at
        http://www.lua.org/work/lua-5.4.3-rc1.tar.gz

The checksums are
        MD5     b8b5b2d309cdfa6c217dfb0ec64fd2c6  -
        SHA1    8ba37aa3fa89724bbcf500af10b861f09dacfdd4  -

Lua 5.4.3 fixes all bugs listed in
        http://www.lua.org/bugs.html#5.4.2

Lua 5.4.3 also contains several internal improvements and
includes a revised reference manual:
        http://www.lua.org/work/doc/

The complete diffs from Lua 5.4.2 are available at
        http://www.lua.org/work/diffs-lua-5.4.2-lua-5.4.3.html
        http://www.lua.org/work/diffu-lua-5.4.2-lua-5.4.3.html

We thank everyone for their feedback on Lua 5.4 till now.

All feedback welcome. Thanks.
--lhf

[Vinicius Jarina]

MSVC 16 2019 Warning:

lfunc.c:170

level->tbclist.delta = level - L->tbclist;

lua\src\lfunc.c(170,44): warning C4244: '=': conversion from '__int64' to 'unsigned short', possible loss of data

On Wed, Mar 3, 2021 at 5:30 AM Luiz Henrique de Figueiredo <[hidden email]> wrote:

Lua 5.4.3 (rc1) is now available for testing at
        http://www.lua.org/work/lua-5.4.3-rc1.tar.gz

The checksums are
        MD5     b8b5b2d309cdfa6c217dfb0ec64fd2c6  -
        SHA1    8ba37aa3fa89724bbcf500af10b861f09dacfdd4  -

Lua 5.4.3 fixes all bugs listed in
        http://www.lua.org/bugs.html#5.4.2

Lua 5.4.3 also contains several internal improvements and
includes a revised reference manual:
        http://www.lua.org/work/doc/

The complete diffs from Lua 5.4.2 are available at
        http://www.lua.org/work/diffs-lua-5.4.2-lua-5.4.3.html
        http://www.lua.org/work/diffu-lua-5.4.2-lua-5.4.3.html

We thank everyone for their feedback on Lua 5.4 till now.

All feedback welcome. Thanks.
--lhf

[Roberto Ierusalimschy]

> > On Wed, Mar 3, 2021 at 5:50 PM actboy168 <[hidden email]> wrote:
> >
> >> Open the LUAI_ASSERT macro, and then run this code, it will get an
> >> assertion failure.
> [...]
>
> The issue is on line 66 of ldebug.c. [...]

Thanks both for the feedback.

-- Roberto

[Andrew Gierth]

>>>>> "Luiz" == Luiz Henrique de Figueiredo <[hidden email]> writes:

 Luiz> Lua 5.4.3 (rc1) is now available for testing at
 Luiz> http://www.lua.org/work/lua-5.4.3-rc1.tar.gz

This changes the API incompatibly in a minor revision.

How on earth did anyone possibly think this was a good idea?

(If you need to change the API, and I think on the whole the change is
an improvement, then this needs to be 5.5.0, NOT 5.4.3.)

--
Andrew.

[Roberto Ierusalimschy]

> I guess the https://www.lua.org/work/lua-5.4.3-tests.tar.gz is missing
> tracegc.lua ?

Thanks for the feedback.

-- Roberto

[Roberto Ierusalimschy]

> MSVC 16 2019 Warning:
>
> lfunc.c:170
> level->tbclist.delta = level - L->tbclist;
>
> lua\src\lfunc.c(170,44): warning C4244: '=': conversion from '__int64' to
> 'unsigned short', possible loss of data

Thanks for the feedback.

-- Roberto

[Roberto Ierusalimschy]

> >>>>> "Luiz" == Luiz Henrique de Figueiredo <[hidden email]> writes:
>
>  Luiz> Lua 5.4.3 (rc1) is now available for testing at
>  Luiz> http://www.lua.org/work/lua-5.4.3-rc1.tar.gz
>
> This changes the API incompatibly in a minor revision.

Whould you mind explaining?

-- Roberto

[Andrew Gierth]

>>>>> "Roberto" == Roberto Ierusalimschy <[hidden email]> writes:

 Luiz> Lua 5.4.3 (rc1) is now available for testing at
 Luiz> http://www.lua.org/work/lua-5.4.3-rc1.tar.gz

 >> This changes the API incompatibly in a minor revision.

 Roberto> Whould you mind explaining?

Code written for 5.4.2 expects lua_pop / lua_settop to close variables,
but as soon as you load it with a 5.4.3 binary or .so, that stops
happening and the code no longer behaves correctly (if you build lua
with asserts you can get an assert failure, I haven't explored what can
happen without asserts but I bet it's possible for it to crash).

--
Andrew.

[Dibyendu Majumdar]

On Wed, 3 Mar 2021 at 13:30, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
>
> Lua 5.4.3 (rc1) is now available for testing at
>         http://www.lua.org/work/lua-5.4.3-rc1.tar.gz
>

Hi,

I have a question related to this commit:

https://github.com/lua/lua/commit/4e47f81188d37e29027158b76271d02a781242e2

If the threading of stack variables occurs this way what happens if
someone swaps stack variables as I believe it is possible to do using
the C api?

Regards

[Roberto Ierusalimschy]

> One question: why are the definitions for l_likely and l_unlikely that use
> the GNU builtin "__builtin_expect" only available by default in the Lua
> core and libraries, and not in C extension modules?

No strong reason. We usually do not export names not prefixed with
'lua_' (or some variant). Of course we could rename these macros.

-- Roberto

[Roberto Ierusalimschy]

> >>>>> "Roberto" == Roberto Ierusalimschy <[hidden email]> writes:
>
>  Luiz> Lua 5.4.3 (rc1) is now available for testing at
>  Luiz> http://www.lua.org/work/lua-5.4.3-rc1.tar.gz
>
>  >> This changes the API incompatibly in a minor revision.
>
>  Roberto> Whould you mind explaining?
>
> Code written for 5.4.2 expects lua_pop / lua_settop to close variables,
> but as soon as you load it with a 5.4.3 binary or .so, that stops
> happening and the code no longer behaves correctly (if you build lua
> with asserts you can get an assert failure, I haven't explored what can
> happen without asserts but I bet it's possible for it to crash).

Probably we should have put LUA_COMPAT_5_4_0 on by default.

I don't see how that error could crash Lua. It probably would raise
an error that the object being closed does not have a __closing method.
If that can really crash Lua, the check should probably be always
on, as this seems a quite common error for one to make. Maybe the
function could always raise a normal error when popping an active
to-be-closed variable?

Our thinking in this change is that probably very few people are already
using to-be-closed variables in the C API, so this change would affect
very few people. To wait for 5.5 for this change (in particular for
adding lua_closeslot) would make the incompatibility more troubling.

-- Roberto

[Roberto Ierusalimschy]

> I have a question related to this commit:
>
> https://github.com/lua/lua/commit/4e47f81188d37e29027158b76271d02a781242e2
>
> If the threading of stack variables occurs this way what happens if
> someone swaps stack variables as I believe it is possible to do using
> the C api?

What do you mean by "swaps stack variables"?

-- Roberto

[Andrew Gierth]

>>>>> "Roberto" == Roberto Ierusalimschy <[hidden email]> writes:

 >> Code written for 5.4.2 expects lua_pop / lua_settop to close
 >> variables, but as soon as you load it with a 5.4.3 binary or .so,
 >> that stops happening and the code no longer behaves correctly (if
 >> you build lua with asserts you can get an assert failure, I haven't
 >> explored what can happen without asserts but I bet it's possible for
 >> it to crash).

 Roberto> Probably we should have put LUA_COMPAT_5_4_0 on by default.

If you want to make changes that warrant a LUA_COMPAT_* compile option,
then you should be making a new major version. As it stands, there's no
way for source code to know whether it should be trying to use the 5.4.2
or 5.4.3 protocols for close vars, and _even if it could know at compile
time_ it would break at runtime since there's no way for a module to
know whether it's linked against 5.4.2 or .3.

 Roberto> I don't see how that error could crash Lua.

What about the fact that L->tbclist is now pointing off the top of the
stack? Are you guaranteed that (even after, say, a garbage collection)
that the invariants that luaF_close relies on are still maintained?

 Roberto> Our thinking in this change is that probably very few people
 Roberto> are already using to-be-closed variables in the C API,

Hi. I'm one of those "very few people".

 Roberto> so this change would affect very few people. To wait for 5.5
 Roberto> for this change (in particular for adding lua_closeslot) would
 Roberto> make the incompatibility more troubling.

Then you need to break the API even further to make sure there aren't
silent failures; my suggestion would be for 5.4.3 to have a new #define
for lua_toclose that changes the external function name, so that code
compiled for 5.4.2 that uses lua_toclose will fail to load under 5.4.3
due to the missing function. (Code written for 5.4.3 that uses
lua_closeslot will obviously fail to load under 5.4.2 the same way.)

Then those "few" of us (neither you nor I know how many of us there
really are) can just say "this code requires 5.4.3 minimum".

(I've mentioned in the past that you should add a way for modules to
know at runtime the full version of the interpreter, not just the major
version. This already came up with 5.3.6 (whether workarounds for the
lua_pushcfunction bug are required) and 5.4.2 (whether the C stack size
needs to be set), and now we see it again.)

--
Andrew.