[ANN] Lua 5.4.0 (alpha) now available

classic Classic list List threaded Threaded
94 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Egor Skriptunoff-2

There are two different operations: x+1 and 1+x
They could give different results (if x has "__add" metamethod).
But they are indistinguishable in luac output:

local x
x = x + 1
x = 1 + x

        1       [1]     VARARGPREP      0
        2       [1]     LOADNIL         0 0     ; 1 out
        3       [2]     ADDI            0 0 1
        4       [3]     ADDI            0 0 1
        5       [3]     RETURN          1 1 1   ; 0 out
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Egor Skriptunoff-2
In reply to this post by Luiz Henrique de Figueiredo
Why the same operation (marking VM register "to-be-closed") is written in different words in comments in lopcodes.h
OP_TBC,     /* A     mark variable A "to be closed"      */
OP_TFORPREP,/* A Bx  create upvalue for R(A + 3); pc+=Bx */

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Luiz Henrique de Figueiredo
In reply to this post by Egor Skriptunoff-2
You're quite right. The listing should take the value of isk into consideration.

On Tue, Jul 2, 2019 at 5:53 PM Egor Skriptunoff
<[hidden email]> wrote:

>
>
> There are two different operations: x+1 and 1+x
> They could give different results (if x has "__add" metamethod).
> But they are indistinguishable in luac output:
>
> local x
> x = x + 1
> x = 1 + x
>
>         1       [1]     VARARGPREP      0
>         2       [1]     LOADNIL         0 0     ; 1 out
>         3       [2]     ADDI            0 0 1
>         4       [3]     ADDI            0 0 1
>         5       [3]     RETURN          1 1 1   ; 0 out

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Egor Skriptunoff-2
In reply to this post by Egor Skriptunoff-2
More nitpicking:

1)
Temporary variable "(for limit)" doesn't contain the limit for integer for-loops.
And it might be impossible to determine the proper name at compile-time :-(
What about name "(for limit/count)" ?


2)
Since there is no subtraction in this instruction anymore, the following comment should be rewritten somehow:
OP_FORPREP,/* A Bx R(A)-=R(A+2); pc+=Bx */

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Hisham
In reply to this post by Dirk Laurie-2
On Thu, 27 Jun 2019 at 10:13, Dirk Laurie <[hidden email]> wrote:
> Op Do. 27 Jun. 2019 om 13:08 het Dibyendu Majumdar
> <[hidden email]> geskryf:
> I've been a Lua user for almost ten years, and I've succeeded in
> understanding and sometimes using every language addition in that
> time. Not so with <toclose>.
>
> My perceptions about it are:
>
> 1. It solves a problem encountered by a very small fraction of Lua users.

Have you ever a directory iterator? (e.g. for file in lfs.dir() ) If
so, you ran into this problem (maybe without realizing it).

To be quite honest, I've considered suggesting an ultra-minimalistic
approach for this whole issue: only provide to-be-closed behavior to
the `for` loop, and allow `for` to be used as a sort of `with`.

The thing about a so-called "minimalistic" langauge is that it has few
concepts, but those are often heavily overloaded.

The most obvious example is that a Lua table is an array, a hashtable,
an object etc.

Some languages have separate statements for `for` and `foreach`. Lua
overloads `for` with "numeric for" and "generic for".

So here's an idea for a third one, let's call it "resource for":

stat ::=
   ...
   for Name ‘=’ exp ‘,’ exp [‘,’ exp] do block end |
   for namelist in explist do block end |
   for namelist ‘=’ exp do block end |
   ...

(as you can see, it is syntactically a mix of the other two: you can
get multiple names on the LHS but a single expression on the RHS: if
it uses `=` but no `,`, it's a resource-for). It works in pseudo-code
like this:

do
   local x, y, ... = exp
   if x == nil and y == nil then break end -- only auto-breaks if
_all_ values are nil, so we can use libraries which use nil-err
errors.
   block
   if x then
      local xmt = getmetatable(x)
      if xmt and xmt.__close then
         xmt.__close(x)
      end
   end
end

Example usage:

for fd, err = io.open("file", "r") do
   if err then
      print("failed!", err)
      break
   end
   print(fd:read("*a"))
end

You could read this as a "for that only runs once because it has no
iteration second argument".

Generic-for would also be extended to support __close, of course.

HOWEVER, if we were to take a page from what's now mainstream syntax
in 21st century languages like Swift and Rust, and instead of
overloading `for` we overload `if`. The above example would turn into
this instead, which is super nice:

if local fd, err = io.open("file", "r") then
   print(fd:read("*a"))
else
   print("failed!", err)
end

Since this is new syntax, one could simply establish that the __close
check runs at the end of every if-local block.

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Sergey Kovalev
пн, 15 июл. 2019 г. в 23:01, Hisham <[hidden email]>:
>
> Have you ever a directory iterator? (e.g. for file in lfs.dir() ) If
> so, you ran into this problem (maybe without realizing it).
lua 5.4 solve this using toclose and 4-th parameter for "for"
print(io.lines"readme.txt")
function: 0x23af370    nil    nil    file (0x23af130)

> So here's an idea for a third one, let's call it "resource for":
>
> stat ::=
>    ...
>    for Name ‘=’ exp ‘,’ exp [‘,’ exp] do block end |
>    for namelist in explist do block end |
>    for namelist ‘=’ exp do block end |
>    ...
>
> (as you can see, it is syntactically a mix of the other two: you can
> get multiple names on the LHS but a single expression on the RHS: if
> it uses `=` but no `,`, it's a resource-for). It works in pseudo-code
> like this:
>
> do
>    local x, y, ... = exp
>    if x == nil and y == nil then break end -- only auto-breaks if
> _all_ values are nil, so we can use libraries which use nil-err
> errors.
>    block
>    if x then
>       local xmt = getmetatable(x)
>       if xmt and xmt.__close then
>          xmt.__close(x)
>       end
>    end
> end
>
> Example usage:
>
> for fd, err = io.open("file", "r") do
>    if err then
>       print("failed!", err)
>       break
>    end
>    print(fd:read("*a"))
> end

I think solution should cover typical usage scenarios. For example
open source and destination files for processing and rise error if it
impossible. "Happy ways" is shorter and simpler to write. Also it
should be able to handle errors by user if he want it.
src=auto(io.close){ io.open(src_filename,"rb") }
dst=auto(io.close){ io.open(dst_filename,"wb") }
-- do somethig with src and dst


> You could read this as a "for that only runs once because it has no
> iteration second argument".
This could be done even now in lua 5.4

-- open.lua
function io_open(name,mode)
  local first=true
  local f,err=io.open(name,mode)
  local close=function() if f then f:close() end f=nil end
  return function()
    if first then first=false return f or false,err end
  end,nil,nil,setmetatable({},{__close=close})
end

for f,err in io_open "readme.txt" do
  if not f then warn(err) break end
  print(f:read())
end

> if local fd, err = io.open("file", "r") then
>    print(fd:read("*a"))
> else
>    print("failed!", err)
> end
This is too heavy nested in case of 2 files or if you open an array of
files you have to use recursion. So this is not like it should be.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Gé Weijers


On Mon, Jul 15, 2019 at 2:07 PM Sergey Kovalev <[hidden email]> wrote:
 
> if local fd, err = io.open("file", "r") then
>    print(fd:read("*a"))
> else
>    print("failed!", err)
> end
This is too heavy nested in case of 2 files or if you open an array of
files you have to use recursion. So this is not like it should be.


You could make an array that calls __close on its contents whenever its own __close metamethod is called. This would not require recursion.

 
--
--

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Sergey Kovalev
вт, 16 июл. 2019 г. в 00:13, Gé Weijers <[hidden email]>:
>
>
> You could make an array that calls __close on its contents whenever its own __close metamethod is called. This would not require recursion.
>
Show me code example.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Gé Weijers


On Mon, Jul 15, 2019 at 2:16 PM Sergey Kovalev <[hidden email]> wrote:
вт, 16 июл. 2019 г. в 00:13, Gé Weijers <[hidden email]>:
>
>
> You could make an array that calls __close on its contents whenever its own __close metamethod is called. This would not require recursion.
>
Show me code example.


Something like his (untested):

local function close_values(t)
  for _, val in pairs(t) do
    local <toclose> tmp = val
  end
end

local <toclose> array = setmetatable({}, {__close = close_values})

When 'array' goes out of scope close_values will be called, which will call __close on all the values in the array.



--
--

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Sergey Kovalev
вт, 16 июл. 2019 г. в 00:23, Gé Weijers <[hidden email]>:

>
>
> Something like his (untested):
>
> local function close_values(t)
>   for _, val in pairs(t) do
>     local <toclose> tmp = val
>   end
> end
>
> local <toclose> array = setmetatable({}, {__close = close_values})
>
> When 'array' goes out of scope close_values will be called, which will call __close on all the values in the array.
>
1. I mean using syntax
if local f,err=io.open ...
and for f,err=io.open ...

2. Even this case.How you suggest to fill this array in case of possible errors?

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Gé Weijers


On Mon, Jul 15, 2019 at 2:31 PM Sergey Kovalev <[hidden email]> wrote:
вт, 16 июл. 2019 г. в 00:23, Gé Weijers <[hidden email]>:
>
>
> Something like his (untested):
>
> local function close_values(t)
>   for _, val in pairs(t) do
>     local <toclose> tmp = val
>   end
> end
>
> local <toclose> array = setmetatable({}, {__close = close_values})
>
> When 'array' goes out of scope close_values will be called, which will call __close on all the values in the array.
>
 
1. I mean using syntax
if local f,err=io.open ...
and for f,err=io.open ...

I probably would not use this construct, it's way too limited, I basically agree with you on that. I just wanted to make the point that you can construct an array that keeps you from leaking resources, in stead of using recursion. It'll still be clumsy.



--
--

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Hisham
In reply to this post by Sergey Kovalev
On Mon, 15 Jul 2019 at 18:07, Sergey Kovalev <[hidden email]> wrote:
>
> пн, 15 июл. 2019 г. в 23:01, Hisham <[hidden email]>:
> >
> > Have you ever a directory iterator? (e.g. for file in lfs.dir() ) If
> > so, you ran into this problem (maybe without realizing it).
> lua 5.4 solve this using toclose and 4-th parameter for "for"
> print(io.lines"readme.txt")
> function: 0x23af370    nil    nil    file (0x23af130)

I know, that was my point: to explain to Dirk why this feature is
making into Lua 5.4 iterators, and why it is something necessary and
not bloat.

> > So here's an idea for a third one, let's call it "resource for":
> >
> > stat ::=
> >    ...
> >    for Name ‘=’ exp ‘,’ exp [‘,’ exp] do block end |
> >    for namelist in explist do block end |
> >    for namelist ‘=’ exp do block end |
> >    ...
> >
> > (as you can see, it is syntactically a mix of the other two: you can
> > get multiple names on the LHS but a single expression on the RHS: if
> > it uses `=` but no `,`, it's a resource-for). It works in pseudo-code
> > like this:
> >
> > do
> >    local x, y, ... = exp
> >    if x == nil and y == nil then break end -- only auto-breaks if
> > _all_ values are nil, so we can use libraries which use nil-err
> > errors.
> >    block
> >    if x then
> >       local xmt = getmetatable(x)
> >       if xmt and xmt.__close then
> >          xmt.__close(x)
> >       end
> >    end
> > end
> >
> > Example usage:
> >
> > for fd, err = io.open("file", "r") do
> >    if err then
> >       print("failed!", err)
> >       break
> >    end
> >    print(fd:read("*a"))
> > end
>
> I think solution should cover typical usage scenarios. For example
> open source and destination files for processing and rise error if it
> impossible. "Happy ways" is shorter and simpler to write. Also it
> should be able to handle errors by user if he want it.
> src=auto(io.close){ io.open(src_filename,"rb") }
> dst=auto(io.close){ io.open(dst_filename,"wb") }
> -- do somethig with src and dst
>
>
> > You could read this as a "for that only runs once because it has no
> > iteration second argument".
> This could be done even now in lua 5.4

Yes, I know. In fact, it could be generalized: the <toclose> modifier
can be expressed purely in terms of the "closing value" support of the
generic-for in Lua 5.4. Consider this function, scoped():

----------------------------------------
function scoped(...)
   local cv = ...
   local args = table.pack(...)
   return function(_, v)
      if v == cv then
         cv = nil

         -- hack: force false instead of nil so that
         -- for runs in case of an error
         if args.n > 1 and args[1] == nil then
            args[1] = false
         end

         return table.unpack(args, 1, args.n)
      end
   end, nil, cv, cv
end
----------------------------------------

----------------------------------------
-- example use:
----------------------------------------
local fref

for f, err in scoped(io.open("scoped.lua")) do
   if err then
      print(err)
      break
   end
   fref = f
   print(f:read("*a"))
end

print(io.type(fref)) -- prints "closed file", hooray!
----------------------------------------

The above implementation, while it does demonstrate that <toclose> can
be expressed in terms of generic-for, is too heavyweight for
real-world use (with the extra closure and table packing and
unpacking). That's why I suggested adding it as a core language
feature instead of <toclose>.

Let's say I want to make the equivalent of the above program (have a
resource whose lifetime is scoped to a block) and use <toclose>. This
is honest-to-goodness the nicest way I managed to produce a version of
the above program using <toclose>, I'm not sure I like it better than
the one above:

----------------------------------------
local fref

do
   local f, err = io.open("scoped.lua")
   local <toclose> f = f    -- *****
   if err then
      print(err)
   else
      fref = f
      print(f:read("*a"))
   end
end

print(io.type(fref)) -- prints "closed file", hooray!
----------------------------------------

^ ***** to reiterate a previous post, it's really bad we need a
separate line for this. I might as well just write f:close() at the
end of the block. (It also throws off linters, which complain at the
shadowing.) If nothing else changes about to-be-closed variables
before Lua 5.4 final, I'd like at least this to be addressed.

I agree with you that one downside of this proposal is that it
requires a nesting level for each closable variable within scope (but
remember you can't make an array of closable variables, even with
<toclose>; you can make a closable array, though, as Gé suggested). On
the other hand, providing a visual cue about the lifetime of resources
is also an advantage.

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Sergey Kovalev
чт, 18 июл. 2019 г. в 02:15, Hisham <[hidden email]>:

>
>
> I know, that was my point: to explain to Dirk why this feature is
> making into Lua 5.4 iterators, and why it is something necessary and
> not bloat.
>
> Yes, I know. In fact, it could be generalized: the <toclose> modifier
> can be expressed purely in terms of the "closing value" support of the
> generic-for in Lua 5.4. Consider this function, scoped():
>
> ----------------------------------------
> function scoped(...)
>    local cv = ...
>    local args = table.pack(...)
>    return function(_, v)
>       if v == cv then
>          cv = nil
>
>          -- hack: force false instead of nil so that
>          -- for runs in case of an error
>          if args.n > 1 and args[1] == nil then
>             args[1] = false
>          end
>
>          return table.unpack(args, 1, args.n)
>       end
>    end, nil, cv, cv
> end
> ----------------------------------------
>
> ----------------------------------------
> -- example use:
> ----------------------------------------
> local fref
>
> for f, err in scoped(io.open("scoped.lua")) do
>    if err then
>       print(err)
>       break
>    end
>    fref = f
>    print(f:read("*a"))
> end
>
> print(io.type(fref)) -- prints "closed file", hooray!
> ----------------------------------------
>
> The above implementation, while it does demonstrate that <toclose> can
> be expressed in terms of generic-for, is too heavyweight for
> real-world use (with the extra closure and table packing and
> unpacking). That's why I suggested adding it as a core language
> feature instead of <toclose>.
>
> Let's say I want to make the equivalent of the above program (have a
> resource whose lifetime is scoped to a block) and use <toclose>. This
> is honest-to-goodness the nicest way I managed to produce a version of
> the above program using <toclose>, I'm not sure I like it better than
> the one above:
>
> ----------------------------------------
> local fref
>
> do
>    local f, err = io.open("scoped.lua")
>    local <toclose> f = f    -- *****
>    if err then
>       print(err)
>    else
>       fref = f
>       print(f:read("*a"))
>    end
> end
>
> print(io.type(fref)) -- prints "closed file", hooray!
> ----------------------------------------
>
> ^ ***** to reiterate a previous post, it's really bad we need a
> separate line for this. I might as well just write f:close() at the
> end of the block. (It also throws off linters, which complain at the
> shadowing.) If nothing else changes about to-be-closed variables
> before Lua 5.4 final, I'd like at least this to be addressed.
>
> I agree with you that one downside of this proposal is that it
> requires a nesting level for each closable variable within scope (but
> remember you can't make an array of closable variables, even with
> <toclose>; you can make a closable array, though, as Gé suggested). On
> the other hand, providing a visual cue about the lifetime of resources
> is also an advantage.
>
> -- Hisham
>
Idea is good but syntax is inconsistent. It is too far from that it
should. <toclose> in for look nice but <toclose> in local is not.
Do you consider following scope prototype:

function io.scope()
    local m=setmetatable({ files={} },{index=io})
    function m.open(name,mode)
        local f,err=io.open(name,mode)
        if f then table.insert(m.files,f) end
        return f,err
    end
    function m.popen(name,mode)
        local f,err=io.popen(name,mode)
        if f then table.insert(m.files,f) end
        return f,err
    end
    local close_all=function()
        for i=#m.files,1,-1 do
            local f=m.files[i]
            m.files[i]=nil
            if io.type(f)=='file' then io.close(f) end
        end
    end
    m.close_all=close_all
    return function(ctx,prev)
        if prev==nil then
            return m
        else
            m.close_all() -- lua 5.3: will not work with break inside for
        end
    end,nil,nil,setmetatable({},{__close=close_all}) -- lua 5.4
end

-- usage:
for io in io.scope() do
    local s,serr=io.open("src","rb")
    local d,derr=io.open("dst","wb")
    if s and d then d:write(s:read()) end
end

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Roberto Ierusalimschy
In reply to this post by Hisham
> [...]
>
> ^ ***** to reiterate a previous post, it's really bad we need a
> separate line for this. I might as well just write f:close() at the
> end of the block. (It also throws off linters, which complain at the
> shadowing.) If nothing else changes about to-be-closed variables
> before Lua 5.4 final, I'd like at least this to be addressed.

This was addressed almost a month ago:

  http://lua-users.org/lists/lua-l/2019-06/msg00556.html

(See also commit 4d46289 in github.)

-- Roberto

12345