How to convert to-be-closed variable to normal one in runtime?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

How to convert to-be-closed variable to normal one in runtime?

Egor Skriptunoff-2
Hi!

I have a problem while trying to use a to-be-closed variable as usual upvalue.
An upvalue's lifespan might go beyond the current block, and the lifespan is unknown at compile time.
This is a code of simple Lua program to explain the problem:


local say = print

local function log_start_time()
   local logfile <close> = assert(io.open("log.txt", "a"))

   local function write_to_log(msg)
      logfile:write(msg.."\n")
   end

   write_to_log("Started at "..os.date())
   -- Usually, only start time is written to the log
   -- Unless user specified "--log-everything" option
   if arg[1] == "--log-everything" then
      -- All messages to say() must go also to the log
      function say(msg)
         print(msg)
         write_to_log(msg)
      end
      -- Now I want the file to remain opened after exiting from the current function
      -- I need to de-<close>-ify the variable "logfile"
      -- How can I do that?
   end
end

log_start_time()
say("Hello")
say("Result = "..some_long_time_calculations())
say("Bye")

Reply | Threaded
Open this post in threaded view
|

Re: How to convert to-be-closed variable to normal one in runtime?

Dibyendu Majumdar
On Tue, 21 Jul 2020 at 12:19, Egor Skriptunoff
<[hidden email]> wrote:

> I have a problem while trying to use a to-be-closed variable as usual upvalue.

One problem with the to-be-closed variable approach is that it exposes
the variable ... so all kinds of issues arise from that.
The 'defer' approach on the other hand is to hide all the upvalue - so
it avoids a whole class of issues / questions / misuses.

Regards
Reply | Threaded
Open this post in threaded view
|

Re: How to convert to-be-closed variable to normal one in runtime?

Andrew Gierth
In reply to this post by Egor Skriptunoff-2
>>>>> "Egor" == Egor Skriptunoff <[hidden email]> writes:

 Egor> Hi!

 Egor> I have a problem while trying to use a to-be-closed variable as
 Egor> usual upvalue. An upvalue's lifespan might go beyond the current
 Egor> block, and the lifespan is unknown at compile time.

The approach I've been looking into is to make the equivalent of a C++
auto_ptr, i.e. an object whose sole responsibility is to proxy the
__close of another object.

I have a C implementation, because I also wanted to be able to do some
stuff from C modules with lua_toclose in a reasonably portable (between
5.3 and 5.4) way and almost all the code was the same, but a simple Lua
implementation could be done too (this is largely untested, but uses the
same API as the C version I have):

-- auto_close.lua

local emptytab = {}

local function get_close_method(o)
  if type(o) == 'function' then
    return o
  end
  -- relies on o's metatable not being hidden by __metatable
  -- use debug.getmetatable or the C version if this is an issue
  return (getmetatable(o or emptytab) or emptytab).__close or function()end
end
local function do_close(o,...)
  local refobj = o.refobj
  o.refobj = nil
  return get_close_method(refobj)(refobj,...)
end
local function do_get(o)
  return o.refobj
end
local function do_release(o)
  local refobj = o.refobj
  o.refobj = nil
  return refobj
end
   
local meta = {
  __close = do_close,
  __call = do_get,   -- just a convenience shorthand for :get()
  __index = {
    get = do_get,
    release = do_release,
    close = do_close
  }
}

return function(o,...)
  return setmetatable({ refobj = o }, meta), o, ...
end

--

Intended usage:

local auto_close = require 'auto_close'

-- note that this returns both the new close object _and_ all of
-- the original results

local fc <close>, f, err, errno = auto_close(io.open(whatever))

-- do stuff either with f, or fc() or fc:get()  to access the file

-- then use  fc:release()  to release the underlying file from
-- the auto_close object

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: How to convert to-be-closed variable to normal one in runtime?

Dmitry Meyer
In reply to this post by Egor Skriptunoff-2
It's an ad-hoc solution, but how about:

    local function log_start_time()
       local logfile = assert(io.open("log.txt", "a"))
       local _autoclose_logfile <close> = arg[1] ~= "--log-everything"
and logfile
    ...

It would be more flexible if the <close> attribute doesn't imply the
<const> (it would be possible to just rebind logfile to nil). But I
understand why it was designed that way.
Reply | Threaded
Open this post in threaded view
|

Re: How to convert to-be-closed variable to normal one in runtime?

pocomane
In reply to this post by Egor Skriptunoff-2
On Tue, Jul 21, 2020 at 1:19 PM Egor Skriptunoff wrote:
> I have a problem while trying to use a to-be-closed variable as usual upvalue.
> An upvalue's lifespan might go beyond the current block, and the lifespan is unknown at compile time.
...
>       -- Now I want the file to remain opened after exiting from the current function
>       -- I need to de-<close>-ify the variable "logfile"
>       -- How can I do that?

Normally, you can declosify just overriding the __close() metamethod,
but here it is impossible without affecting the other files. You can
still use the proxy trick suggested by Andrew, or invert the logic:

```
local say = print

local function log_start_time()
   local logfile = assert(io.open("log.txt", "a"))

   local function write_to_log(msg)
      logfile:write(msg.."\n")
   end

   write_to_log("Started at "..os.date())
   if arg[1] ~= "--log-everything" then
     local please <close> = logfile
   else
      function say(msg)
         print(msg)
         write_to_log(msg)
      end
   end
end
```

About the sub-scope trick: in this case it would be simpler to just
call file:close() or getmetatable().__close(), however in some cases
you just can do it (`getmetatable().__metatable = "Not your buisness"`
with an ad-hoc __close() metamethod).
Reply | Threaded
Open this post in threaded view
|

Re: How to convert to-be-closed variable to normal one in runtime?

Andrew Gierth
>>>>> "Mimmo" == Mimmo Mane <[hidden email]> writes:

 Mimmo> Normally, you can declosify just overriding the __close()
 Mimmo> metamethod, but here it is impossible without affecting the
 Mimmo> other files. You can still use the proxy trick suggested by
 Mimmo> Andrew

Incidentally, I forgot to mention the main reason for the proxy, which
is to be able to _return_ the value.

e.g.

function f1(fn)
  local fc <close>, f, err, errno = auto_close(io.open(fn,"r"))
  --[[ check for errors ]]
  --[[ do stuff with f, that might throw an error ]]
  -- if successful, return the file to the caller
  return fc:release()
end

function f2()
  local fc <close>, f = auto_close(f1("filename"))
  --[[ do stuff ]]
end

This does leave a small window for error in the auto_close call itself,
but I believe with the C implementation and with no intervention from
hooks, the only likely failure is an "out of memory" on allocating the
new proxy object.

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: How to convert to-be-closed variable to normal one in runtime?

Egor Skriptunoff-2
In reply to this post by Dibyendu Majumdar
On Tue, Jul 21, 2020 at 3:14 PM Dibyendu Majumdar wrote:
One problem with the to-be-closed variable approach is that it exposes
the variable ... so all kinds of issues arise from that.


Yes, hidden to-be-closed variables (in generic-for loops) are very good.
Only visible ones could be misused  :-)

IMO, the "defer" statement is more comfortable than the metamethod.
For example, the following line would be a simple solution:
defer if arg[1] ~= "--log-everything" then logfile:close() end end
But there are reasons why "defer" is not used in Lua.

The problem can be reworded as follows:
Previously in Lua, a dependency was a king.
But a new behavior was introduced, which deliberately ignores dependencies.
A closeable value is being closed despite some Lua objects still referring to the variable.

So, the <close> keyword should be used with caution.
Isn't it the reason for such conspicuous <syntax> ? :-)

Reply | Threaded
Open this post in threaded view
|

Re: How to convert to-be-closed variable to normal one in runtime?

Egor Skriptunoff-2
In reply to this post by Andrew Gierth
On Tue, Jul 21, 2020 at 5:52 PM Andrew Gierth wrote:

function f1(fn)
  local fc <close>, f, err, errno = auto_close(io.open(fn,"r"))
  --[[ check for errors ]]
  --[[ do stuff with f, that might throw an error ]]
  -- if successful, return the file to the caller
  return fc:release()
end

function f2()
  local fc <close>, f = auto_close(f1("filename"))
  --[[ do stuff ]]
end

This does leave a small window for error in the auto_close call itself


Unfortunately, the window is a bit larger.

function f1(fn)
  local obj <close> = create_temporary_obj()
  local fc <close>, f, err, errno = auto_close(io.open(fn,"r"))
  --[[ check for errors ]]
  --[[ do stuff with f, that might throw an error ]]
  -- if successful, return the file to the caller
  return fc:release()
end

If an error is raised inside the "__close" metamethod of "obj", the file "f" remains non-closed.
When "obj" is being closed, the "fc:release()" is already complete and "f" is currently not protected.