Proposal: Finally statement of function

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal: Finally statement of function

Nodir Temirkhodjaev
Hi.
There is pcall() ('try') without 'finally' to properly free resources in the called functions.
What do you think about adding the optionally finally block statement to functions?
This finally block must be executed once whenever returned from function or error throwed.

funcbody ::= '(' [parlist1] ')' block1 [finally block2] end

Examples:

---- Parsing error: finally block cannot return

function a()
  ...
finally
  return
end

---- Close file

function g()
  error("Exception")
end

function f(filename)
  local file = io.open(filename)
  g()
finally
  if file then file:close() end
end

---- Finalization chain: prints a b c

function c() print'c' end

function b() return c() finally print'b' end

function a() return b() finally print'a' end

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

D Burgess-4
Luasocket implements finalizers by using xpcall() and it does simplify
and shorten the code.

I do like your suggested syntax.
I wonder what it would take to implement it.

Nodir Temirhodzhaev wrote:
funcbody ::= '(' [parlist1] ')' block1 [finally block2] end

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Doug Rogers-4
In reply to this post by Nodir Temirkhodjaev
Nodir Temirhodzhaev wrote:

> This finally block must be executed once whenever returned from function
> or error throwed.
> funcbody ::= '(' [parlist1] ')' block1 [finally block2] end

The use case (cleaning up after errors) is like exceptions in Ada, but
what you propose is more general, but without passing arguments to the
handler. I think it would be even more general, and would provide the
ability to clean up inner scoped locals, if 'finally' were applied to
code blocks. So:

  block ::= chunk | finally { stat[;] }

It can't replace chunk since chunk includes a return or break statement.

I believe this would preclude tail calls in functions with finalization.

Doug

-- 
Innovative Concepts, Inc. www.innocon.com 703-893-2007 x220

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Rici Lake-2

On 9-Feb-07, at 12:46 PM, Doug Rogers wrote:

Nodir Temirhodzhaev wrote:

This finally block must be executed once whenever returned from function
or error throwed.
funcbody ::= '(' [parlist1] ')' block1 [finally block2] end

The use case (cleaning up after errors) is like exceptions in Ada, but
what you propose is more general, but without passing arguments to the
handler. I think it would be even more general, and would provide the
ability to clean up inner scoped locals, if 'finally' were applied to
code blocks. So:

  block ::= chunk | finally { stat[;] }

It can't replace chunk since chunk includes a return or break statement.

I believe this would preclude tail calls in functions with finalization.

Not necessarily. I sketched out an implementation of finally clauses
in the midst of a longer discussion about non-local exits some
time ago. Thanks to the lua-l archives time machine, here it is:
<http://lua-users.org/lists/lua-l/2005-08/msg00357.html>

The question is, how to make this work with tailcalls. In OP_RETURN
and OP_TAILCALL, open upvalues are closed down to the beginning
of the call-frame. With the implementation sketched out in the
preceding message, that would trigger the finalizer, but
unfortunately it will finalize too soon in the case of a tailcall
(i.e. it will trigger before the tailcall.)

However, there is a stack slot associated with the current
function, effectively stack slot 0. If we associate the finalizer
with slot 0 instead of slot 1, and only close upvalues down to
slot 1 on a tailcall, the finalizer will not be triggered until
the call-frame is popped, which is the correct moment.

Of course, it may be the case that more than one function installs
a finalizer on the same call-frame, but that will work fine as long
as the order of upvalues on the open upvalue chain is kept stable.
Currently, it's impossible to have more than one upvalue on the
chain with the same level, but I don't see any reason why this
invariant can't be loosened a bit.

Since the call-frame itself is cleared by a tailcall, the finalizer
cannot refer to any locals in the function for which it is a finalizer
without turning those locals into upvalues. However, compiling
finalizers into functions. That fails to take advantage of
the potential efficiencies of not turning locals into upvalues.
An optimization of that form would require a little more work, and
it would, in my opinion, be necessary to demonstrate that it was
actually necessary before attempting an implementation.


Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Doug Rogers-4
Rici Lake wrote:
> On 9-Feb-07, at 12:46 PM, Doug Rogers wrote:
>>   block ::= chunk | finally { stat[;] }
>> I believe this would preclude tail calls in functions with finalization.

> Not necessarily. I sketched out an implementation of finally clauses
> in the midst of a longer discussion about non-local exits some
> time ago. Thanks to the lua-l archives time machine, here it is:
> <http://lua-users.org/lists/lua-l/2005-08/msg00357.html>
> ...
> However, there is a stack slot associated with the current
> function, effectively stack slot 0. If we associate the finalizer
> with slot 0 instead of slot 1, and only close upvalues down to
> slot 1 on a tailcall, the finalizer will not be triggered until
> the call-frame is popped, which is the correct moment.

Clever! It will be laborious to verify its operation under all cases and
conditions. But we live to labor, do we not?

Doug

-- 
Innovative Concepts, Inc. www.innocon.com 703-893-2007 x220

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Rici Lake-2

On 9-Feb-07, at 3:07 PM, Doug Rogers wrote:

Clever! It will be laborious to verify its operation under all cases and
conditions. But we live to labor, do we not?

No doubt. :)

I should have mentioned the one issue with that approach. Normally, we want finalizers to be timely (otherwise, we could have just used the garbage collector); finalizers implemented with the upvalue mechanism will certainly be timely if execution does not do a coroutine.yield() -- since the finalizers are not implemented with pcall, that's actually possible.

Now, we might have different views about what should happen if a thread yields; in essence, we certainly want finalizers to run if the thread is never going to be resumed, but it's hard to know that in advance. So that leaves the finalizer running when/if the thread is garbage collected.

Since that's somewhat unpredictable, many people prefer dynamic windows to finalizers; the idea being that when an invariant is maintained *while the thread is active.* If the thread yields, the invariant can be broken (for example, a lock can be released) provided that it is restored before the thread is resumed. That may be too complicated for a language like Lua (indeed, finalizers themselves may be too complicated); certainly, experience with such mechanisms in Lisp and Scheme suggests that they work fine when used correctly, but they are very easy to get wrong.


Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Glenn Maynard
On Fri, Feb 09, 2007 at 03:29:57PM -0500, Rici Lake wrote:
>>There is pcall() ('try') without 'finally' to properly free resources in the
>>called functions.  What do you think about adding the optionally finally block
>>statement to functions?  This finally block must be executed once whenever
>>returned from function or error throwed.

> Now, we might have different views about what should happen if a thread 
> yields; in essence, we certainly want finalizers to run if the thread 
> is never going to be resumed, but it's hard to know that in advance. So 
> that leaves the finalizer running when/if the thread is garbage 
> collected.

I havn't understood all of the details of this idea, since I don't know
Lua's internals well enough, but what if an error is thrown?  I think
that's most of the point: to be able to clean up promptly if an error
happens.

It seems like it couldn't run the finalizer then call the error
function; errfunc needs to be called at the actual error, for
backtracing.  In principle, it could call the error handler, then
call the finalizer before returning to the nearest pcall.

A similar problem exists in C.  In one case, I want a CFunction
that does this:

        int func(lua_State *L)
        {
                object *obj = new object;
                lua_call(L, 0, 0);
                delete obj;
                return 0;
        }    

where the given function is called with "obj" existing, and "obj" is
guaranteed to be deleted before it returns.  (This sets some state,
and unsets it when deleted.)

The trouble is deleting p on error.  I can't just use lua_pcall, because
I do want errors propagated to the caller transparently.  If I pcall,
delete, then throw the error again, the error handler will get an unwound
stack.

I can't stash p in a userdata, because p can't be deleted lazily, even
on error.

It seems like changing the current errfunc would work, in principle:

        object *obj = new object;

        lua_geterrorhandler(L); // push errfunc or panic func

	lua_pushvalue(L, -1); // A
	lua_pushlightuserdata(L, obj);
	lua_pushcclosure(L, function_to_delete_obj, 2);
        lua_seterrorhandler(L);

        lua_call(L, f);

        /* if we get here, no error occurred */
        lua_seterrorhandler(L); // restore A
        delete obj;

        int function_to_delete_obj(L)
        {
	   obj = lua_touserdata(L, lua_upvalueindex(L, 1));
           delete obj;
	   lua_pushvalue(L, lua_upvalueindex(L, 2));
	   lua_call(L, lua_gettop(L)-1, LUA_MULTRET); // call the original error handler
	   return lua_gettop(L);
        }

but I have no idea about the implementation.

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Rici Lake-2

On 9-Feb-07, at 10:39 PM, Glenn Maynard wrote:

I havn't understood all of the details of this idea, since I don't know
Lua's internals well enough, but what if an error is thrown?  I think
that's most of the point: to be able to clean up promptly if an error
happens.

It seems like it couldn't run the finalizer then call the error
function; errfunc needs to be called at the actual error, for
backtracing.  In principle, it could call the error handler, then
call the finalizer before returning to the nearest pcall.


That's exactly what it does (or would do).

This is how Lua works (currently) when an error is thrown:

1) the error function is called immediately when an
error is encountered, so it sees the current stack.

2) Then upvalues are closed back to the last pcall() (or to the
beginning of the stack.) That's necessary because the stack
is about to be popped back to that pcall().

3) Finally, the stack is popped back to the pcall, and control
is returned to the pcall.

That's precisely why I think that the upvalue closure
mechanism can be used as well to implement finalizers.
The proposed finalizer mechanism runs the finalizer during
the upvalue closure sequence, which means it will run
at step 2, above, ensuring that the finalizer executes
before the pcall is returned to.


Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Glenn Maynard
On Fri, Feb 09, 2007 at 10:49:48PM -0500, Rici Lake wrote:
> That's exactly what it does (or would do).
> 
> This is how Lua works (currently) when an error is thrown:
> 
> 1) the error function is called immediately when an
> error is encountered, so it sees the current stack.
> 
> 2) Then upvalues are closed back to the last pcall() (or to the
> beginning of the stack.) That's necessary because the stack
> is about to be popped back to that pcall().
> 
> 3) Finally, the stack is popped back to the pcall, and control
> is returned to the pcall.
> 
> That's precisely why I think that the upvalue closure
> mechanism can be used as well to implement finalizers.
> The proposed finalizer mechanism runs the finalizer during
> the upvalue closure sequence, which means it will run
> at step 2, above, ensuring that the finalizer executes
> before the pcall is returned to.

In this case, could it be possible to have a C API to add a cleanup
function to the running CFunction (rather than messing with the error
handler) through the same mechanism (perhaps requiring reserving
a closure slot in advance, though ideally not)?

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Finally statement of function

Rici Lake-2

On 9-Feb-07, at 11:12 PM, Glenn Maynard wrote:

On Fri, Feb 09, 2007 at 10:49:48PM -0500, Rici Lake wrote:
That's exactly what it does (or would do).

This is how Lua works (currently) when an error is thrown:

1) the error function is called immediately when an
error is encountered, so it sees the current stack.

2) Then upvalues are closed back to the last pcall() (or to the
beginning of the stack.) That's necessary because the stack
is about to be popped back to that pcall().

3) Finally, the stack is popped back to the pcall, and control
is returned to the pcall.

That's precisely why I think that the upvalue closure
mechanism can be used as well to implement finalizers.
The proposed finalizer mechanism runs the finalizer during
the upvalue closure sequence, which means it will run
at step 2, above, ensuring that the finalizer executes
before the pcall is returned to.

In this case, could it be possible to have a C API to add a cleanup
function to the running CFunction (rather than messing with the error
handler) through the same mechanism (perhaps requiring reserving
a closure slot in advance, though ideally not)?

I think so. As with regular Lua functions, you should be able to use
the call-frame's "Slot 0"; the upvalue finalizer is not actually stored
on the stack; it just refers to the (absolute) stack level.

The cleanup function would have to be a regular lua_CFunction and it
wouldn't have access to the Lua stack when it ran. Also, it would always
run (not just on errors). But that might solve some problems, anyway.

Bear in mind that I haven't actually tried to write this patch,
although the original message from 2005 does have quite a lot
of code in it; if the idea works, it should be pretty simple
to implement.