[ANN] Lua 5.4.0 (alpha) now available

classic Classic list List threaded Threaded
94 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Sergey Kovalev
In case of memory error best solution is abort execution ( terminate process )

пт, 28 июн. 2019 г. в 17:48, Roberto Ierusalimschy <[hidden email]>:

>
> > Why we need extra functionality if it could be done with existing one?
> >
> > [...]
> >     local f=auto(io.close){ io.open "foobar" } -- will be closed
>
> It won't, if the table creation raises a memory error.
>
> -- Roberto
>

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

pocomane
In reply to this post by Sergey Kovalev


Il ven 28 giu 2019, 15:06 Sergey Kovalev <[hidden email]> ha scritto:
Why we need extra functionality if it could be done with existing one?


...

What about coroutines? I.e. "body" resumes a coroutine that calls "auto" (not protected by an inner "scope").
Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Sergey Kovalev
Can you give an example?

> What about coroutines? I.e. "body" resumes a coroutine that calls "auto" (not protected by an inner "scope").

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

pocomane
On Fri, Jun 28, 2019 at 6:12 PM Sergey Kovalev <[hidden email]> wrote:
>
> Can you give an example?
>
> > What about coroutines? I.e. "body" resumes a coroutine that calls "auto" (not protected by an inner "scope").

```
local function my_handler(f)
  print("closing",f)
end

local a_coroutine = coroutine.create(function()
  local auto = coroutine.yield()
  local f1 = auto(my_handler){ "file 1" }
  coroutine.yield()
  local f2 = auto(my_handler){ "file 2" }
  coroutine.yield()
  print("using", f1, "and", f2)
end)
coroutine.resume(a_coroutine)

scope(function(auto)
  coroutine.resume(a_coroutine, auto)
  coroutine.resume(a_coroutine)
end)
coroutine.resume(a_coroutine)
```

Yes, I know, I should use the scope/auto INSIDE the coroutine. Or I
have to put the last resume INSIDE scope. But the point is that with
this implementation I can also mess around like in the example. With a
real "Scoped annotation" like <toclose> I cannot.

In the past I have used something like your scope/auto, but exposing
auto outside the scope [1]. I admit that passing auto to the body
somehow protect you from some mistake, but I think it is not enough.

pocomane

[1] It was something like:

```
local list

local function defer( func )
  if not list then error("defer autside a scope", 2) end
  list[1+#list] = func
end

local function scope( block )
  local old_list = list
  list = {}

  local ok, err = pcall(block)

  for i = 1, #list do
    local ok, err = pcall(list[i])
    if not ok then warn(err) end
  end

  if not ok then error(err, 1) end

  list = old_list
end

----------------------------
-- usage example:

scope(function()

  local f1 = "file 1"
  defer(function() print("closing", f1) end)

  local f2 = "file 2"
  defer(function() print("closing", f2) end)

  print("using",f1,"and",f2)

end)
```

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Sergey Kovalev
> Yes, I know, I should use the scope/auto INSIDE the coroutine. Or I
> have to put the last resume INSIDE scope. But the point is that with
> this implementation I can also mess around like in the example. With a
> real "Scoped annotation" like <toclose> I cannot.

I see no problem even in this case:

function scope(body)
  local list,res={}
  local function auto(close,msg)
    return function(t)
      if type(t)~='table' then error("need table: { expression }",2) end
      if t[1] then table.insert(list,{ arg=t[1], fn=close or io.close })
      else
        if msg=='weak' then return table.unpack(t) end
        error(msg or t[2] or "no resource",2)
      end
      return table.unpack(t)
    end
  end
  local ok,err=pcall(function() res={body(auto)} end)
  for i=#list,1,-1 do list[i].fn(list[i].arg) end
  if not ok then
    if type(err)~='string' then error(err,2)
    else error("scope error\nlua: "..err,2) end
  end
  return table.unpack(res)
end

--------------------------------------------------------------------------------

local function my_open(...) print("open",...) return ... end
local function my_close(...) print("close",...) end

local a_coroutine = coroutine.create(function()
  local auto = coroutine.yield() -- WTF? This beak normal logic. Just
use scope(function(auto) ... end) here
  local f1 = auto(my_close){ my_open "file 1" }
  coroutine.yield()
  local f2 = auto(my_close){ my_open "file 2" }
  coroutine.yield() -- here all files f1 and f2 are closed so you will
have error in case of real files
  print("using", f1, "and", f2)
end)

coroutine.resume(a_coroutine)

print"-- scope.begin"
scope(function(auto)
  coroutine.resume(a_coroutine, auto)
  coroutine.resume(a_coroutine)
end)
print"-- scope.end"
coroutine.resume(a_coroutine)

The output:
-- scope.begin
open    file 1
open    file 2
close    file 2
close    file 1
-- scope.end
using    file 1    and    file 2

You use files after they have been closed. So you will have an error
in case of real files. Don't do like this.
Typical usage should look like this:

local a_coroutine = coroutine.create(function()
  print"-- scope2.begin"
  scope(function(auto)
    local f1 = auto(my_close){ my_open "file 1" }
    coroutine.yield()
    local f2 = auto(my_close){ my_open "file 2" }
    coroutine.yield()
    print("using", f1, "and", f2)
  end)
  print"-- scope2.end"
end)
print"-- scope1.begin"
scope(function(auto)
  local f0=auto(my_close){ my_open "file 0" }
  coroutine.resume(a_coroutine)
coroutine.resume(a_coroutine)
end)
print"-- scope1.end"
coroutine.resume(a_coroutine)

It outputs:
-- scope1.begin
open    file 0
-- scope2.begin
open    file 1
open    file 2
close    file 0
-- scope1.end
using    file 1    and    file 2
close    file 2
close    file 1
-- scope2.end

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Egor Skriptunoff-2
In reply to this post by Luiz Henrique de Figueiredo
On Thu, Jun 13, 2019 at 4:36 PM Luiz Henrique de Figueiredo wrote:
We also welcome feedback on the listings output by luac -l -l, because
luac has been rewritten to account for the new VM instructions.



Shifts are displayed in non-user-friendly way.
What do 126 and 128 mean?

local x
local a, b = x<<1, x>>1

        1       [1]     VARARGPREP      0
        2       [1]     LOADNIL         0 0     ; 1 out
        3       [2]     SHRI            1 0 126
        4       [2]     SHRI            2 0 128
        5       [2]     RETURN          3 1 1   ; 0 out

 
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Coda Highland
On Fri, Jun 28, 2019 at 4:14 PM Egor Skriptunoff <[hidden email]> wrote:
On Thu, Jun 13, 2019 at 4:36 PM Luiz Henrique de Figueiredo wrote:
We also welcome feedback on the listings output by luac -l -l, because
luac has been rewritten to account for the new VM instructions.



Shifts are displayed in non-user-friendly way.
What do 126 and 128 mean?

local x
local a, b = x<<1, x>>1

        1       [1]     VARARGPREP      0
        2       [1]     LOADNIL         0 0     ; 1 out
        3       [2]     SHRI            1 0 126
        4       [2]     SHRI            2 0 128
        5       [2]     RETURN          3 1 1   ; 0 out


I imagine they're expressed in excess-127, much like the exponents of IEEE floating-point numbers.

/s/ Adam 
Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Dibyendu Majumdar
In reply to this post by Sean Conner
On Thu, 27 Jun 2019 at 22:55, Sean Conner <[hidden email]> wrote:

>
> It was thus said that the Great Dirk Laurie once stated:
> >
> > I've been a Lua user for almost ten years, and I've succeeded in
> > understanding and sometimes using every language addition in that
> > time. Not so with <toclose>.
> >
> > My perceptions about it are:
> >
> > 1. It solves a problem encountered by a very small fraction of Lua users.
> > 2. Although it looks like a property of a variable, i.e. a name, it is
> > a property of the associated value.
> > 3. All variables declared <toclose> must have a metatable with a
> > __close metamethod, otherwise nothing different happens anyway.
> > 4. Therefore they must be either tables or full userdata.
> > 5. The 5.4 alpha syntax for it is the least ugly among various
> > possibilities, but still ugly.
> > 6. It runs counter to Lua's design goals of being minimal, neat and compact.
> >
> > In other words, <toclose> is feature bloat.
>
>   Reading this, I just had an idea.
>           scope f,err = io.open("foobar")


I think that handling multiple return values elegantly requires
specific names to be declared as to-be-closed.

Given the issues with finding a good syntax for it - maybe we can live
with the proposed <toclose> syntax.

Alternatively, and this is an idea proposed by Daurnimator, and one I
think perhaps will be a better solution is to introduce the 'defer'
statement like the Go language.
Interestingly the Go language also supports multiple return values and
uses idioms similar to Lua where an error value is returned as
additional parameter.

The proposed 'defer' statement looks like this:

defer ... end

Where it is simply a synonym for:

local f = function()  ... end

with the guarantee that f will be invoked at scope exit.

This has several nice properties.

1) It is more generic
2) The syntax is nice
3) It is explicit and visible
4) One can perform any kind of cleanup - not just closing resources
5) Variables may be captured as upvalues in the function too


Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Sergey Kovalev
> The proposed 'defer' statement looks like this:
>
> defer ... end
>
> with the guarantee that f will be invoked at scope exit.
>
Can be done following way:

scope(function(auto) local defer=function(fn) auto(fn){true} end

    defer(function() print "defer1" end)
    defer(function() print "defer2" end)

end)

> 5) Variables may be captured as upvalues in the function too
Main problem of upvalues is: there is no effective way to isolate
function body from upvalues in lua.

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

pocomane
In reply to this post by Sergey Kovalev
On Fri, Jun 28, 2019 at 10:49 PM Sergey Kovalev <[hidden email]> wrote:
> I see no problem even in this case:
>
...
>
> You use files after they have been closed. So you will have an error
> in case of real files. Don't do like this.
>

I see how to use correctly your code. What I want to stress is that it
let you to define, by mistake, a "Scope" that spawn multiple
coroutines and functions, and that can also be interrupted in the
middle of a coroutine.

This for me is an issue. However... thinking deeper... maybe the
<toclose> have a similar issue too... :(

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

pocomane
In reply to this post by Sergey Kovalev
On Sat, Jun 29, 2019 at 1:12 AM Sergey Kovalev <[hidden email]> wrote:
>
> > The proposed 'defer' statement looks like this:
> >
> > defer ... end
> >
> > with the guarantee that f will be invoked at scope exit.
> >

I think that <toclose> is more powerful since it let you to define an
object in a scope, but close it in another one. E.g. (I already wrote
something like this in another post):

local function my_complex_object()
  local result = {} -- NOT <toclose>
  -- ...
  return setmetatable( result, {__close = my_finalizer } )
end

do
  local <toclose> obj = my_complex_object()
  -- ... use obj as you wish, it will be close at end of THIS scope
end

Of course you can simulate this with defer

local obj = my_complex_object()
defer getmetatable(obj).__close() end

but the api/protocol is up to you, while with <toclose> there is a
single mechanism that will be shared among all the lua libraries.
(I.e. with defer, someone would use a finalize method, others a
metamethod, others an additional return value, and so on).

However, I found a Go-like defer more elegant too (and probably more
"luaish"). So probably this is a issue of tradeoff... for now I am
slightly more incline to <toclose>.

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Sergey Kovalev
> I think that <toclose> is more powerful since it let you to define an
> object in a scope, but close it in another one.

do -- scope1
  local <toclose> obj1 = my_complex_object(1)
  local obj2 = my_complex_object(2)
  local <toclose> obj3
  local <toclose> obj4
  local <toclose> obj5
  do -- scope2
    obj3 = my_complex_object(3)
    obj4 = my_complex_object(4)
    obj5 = my_complex_object(5)
    obj4 = nil
    obj5 = my_complex_object(55)
 end -- end of scope 2
end -- end of scope 1

And what should be the difference in the fates of obj1, obj2, obj3,
obj4 and obj5 ?

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Philipp Janda
In reply to this post by pocomane
Am 29.06.19 um 07:29 schröbte pocomane:
> Of course you can simulate this with defer
>
> local obj = my_complex_object()
> defer getmetatable(obj).__close() end

Since function definitions allocate memory and thus can fail, this idiom
isn't safe. You'd have to use the more awkward

local obj
defer
   if obj then
     getmetatable(obj).__close()
   end
end
obj = my_complex_object()


And then you realize that Lua may allocate memory during function
*calls* (e.g. growing the stack). I don't think this is covered in the
current <toclose> approach either.

Philipp



Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Coda Highland
In reply to this post by Sergey Kovalev


On Sat, Jun 29, 2019 at 1:09 AM Sergey Kovalev <[hidden email]> wrote:
> I think that <toclose> is more powerful since it let you to define an
> object in a scope, but close it in another one.

do -- scope1
  local <toclose> obj1 = my_complex_object(1)
  local obj2 = my_complex_object(2)
  local <toclose> obj3
  local <toclose> obj4
  local <toclose> obj5
  do -- scope2
    obj3 = my_complex_object(3)
    obj4 = my_complex_object(4)
    obj5 = my_complex_object(5)
    obj4 = nil
    obj5 = my_complex_object(55)
 end -- end of scope 2
end -- end of scope 1

And what should be the difference in the fates of obj1, obj2, obj3,
obj4 and obj5 ?


This isn't permitted. Every line in scope2 will throw an error for assigning to a constant variable.

However, what you CAN do is call obj1.close() somewhere. Assuming that your code is smart enough to not muck with it when it's already closed, then the second close when the <toclose> variable goes out of scope is a no-op.

/s/ Adam 
Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Roberto Ierusalimschy
In reply to this post by Philipp Janda
> And then you realize that Lua may allocate memory during function *calls*
> (e.g. growing the stack). I don't think this is covered in the current
> <toclose> approach either.

I think it is.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Sergey Kovalev
In reply to this post by Philipp Janda
> Since function definitions allocate memory and thus can fail, this idiom
> isn't safe. You'd have to use the more awkward
I see no practical reason too be correct in all possible and
impossible situation.
There is no language and no os that correctly work in case of no resource.
The only solution is to pre allocate resource for code that will kill
innocence to release resources to other.
In case of limited resources you will do only limited amount of work.
You should has tools to measure resources used by program to predict
requirements. Or at least measure typical requirements for typical
tasks.

> And then you realize that Lua may allocate memory during function
> *calls* (e.g. growing the stack). I don't think this is covered in the
> current <toclose> approach either.
Lua has stack reserve for emergency, hasn't it?

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Philipp Janda
In reply to this post by Roberto Ierusalimschy
Am 29.06.19 um 19:55 schröbte Roberto Ierusalimschy:
>> And then you realize that Lua may allocate memory during function *calls*
>> (e.g. growing the stack). I don't think this is covered in the current
>> <toclose> approach either.
>
> I think it is.

How many stack slots do I have available when __close is called?

>
> -- Roberto

Philipp




Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Roberto Ierusalimschy
> How many stack slots do I have available when __close is called?

In any realistic use, the code needs to call something to "acquire a
resource" in an expression:

  local <toclose> x = <acquire a resource>

So, if that call worked, there is at least one stack slot available
to release that resource.

Of course, we can come up with some weird (at least in my view)
code like that, to prove me wrong:

  -- 'f' is a resource acquired in some different scope
  local <toclose> x = f

We will try to improve on that.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.4.0 (alpha) now available

Luiz Henrique de Figueiredo
In reply to this post by Egor Skriptunoff-2
>> We also welcome feedback on the listings output by luac -l -l, because
>> luac has been rewritten to account for the new VM instructions.
>
> Shifts are displayed in non-user-friendly way.

Thanks for spotting this. Here is a fix (use sc instead of c):

   case OP_SHRI:
        printf("%d %d %d",a,b,sc);
        break;
   case OP_SHLI:
        printf("%d %d %d",a,b,sc);
        break;

Reply | Threaded
Open this post in threaded view
|

Re: Yet another proposal for <toclose> (was Re: [ANN] Lua 5.4.0 (alpha) now available)

Coda Highland
In reply to this post by Roberto Ierusalimschy


On Mon, Jul 1, 2019 at 7:30 AM Roberto Ierusalimschy <[hidden email]> wrote:
> How many stack slots do I have available when __close is called?

In any realistic use, the code needs to call something to "acquire a
resource" in an expression:

  local <toclose> x = <acquire a resource>

So, if that call worked, there is at least one stack slot available
to release that resource.

Of course, we can come up with some weird (at least in my view)
code like that, to prove me wrong:

  -- 'f' is a resource acquired in some different scope
  local <toclose> x = f

We will try to improve on that.

-- Roberto


It's not that strange if f is a function argument: it's not an unusual pattern at all to do something like run(newContext()).

/s/ Adam
12345