Delayed evaluation of expressions

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Delayed evaluation of expressions

Lorenzo Donati-2
Hi All!

This is an attempt to both satisfy a curiosity of mine and
start a discussion on a topic that seems recurring lately and (IIRC)
pops up from time to time.

More specifically, I happened to notice in various threads that there
could be a need for a way in Lua to delay the evaluation of expressions.

Lately, in particular, it has been mentioned in the context of error
management to avoid building a complex error message when the error
actually doesn't occur.

What I wonder now is whether the possibility of delaying the evaluation
of an expression would be useful also in other contexts.

If it would, then maybe it is worth considering what should be done to
add to Lua such a feature (or a more general one that could comprise it)
and what could be the consequences (pros and cons) and if the feature
could be general enough to be in "Lua spirit".

Thanks for any contribution.

Cheers.
-- Lorenzo

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

steve donovan
On Fri, Jul 1, 2011 at 7:16 AM, Lorenzo Donati
<[hidden email]> wrote:
> What I wonder now is whether the possibility of delaying the evaluation of
> an expression would be useful also in other contexts.

Well, we already have such a mechanism - function() return
delayed_fun(something) end. Some have felt that this feels over-wordy,
so that ends up in the 'short function form' discussion meme[1].

Irrespective of actual notation, a closure is relatively cheap -
laziness can't be had for free.

Anything else requires major re-engineering, I suspect; how to
harmonize it with the rest of the language is probably the hardest
part.

steve d.

[1] things like |x| x^2 (GSL-shell and metalua) and \x(x^2) (LuaMacro)

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Lorenzo Donati-2
On 01/07/2011 8.10, steve donovan wrote:

> On Fri, Jul 1, 2011 at 7:16 AM, Lorenzo Donati
> <[hidden email]>  wrote:
>> What I wonder now is whether the possibility of delaying the evaluation of
>> an expression would be useful also in other contexts.
>
> Well, we already have such a mechanism - function() return
> delayed_fun(something) end. Some have felt that this feels over-wordy,
> so that ends up in the 'short function form' discussion meme[1].
>
> Irrespective of actual notation, a closure is relatively cheap -
> laziness can't be had for free.
>

Yes, that's the usual approach. I don't care too much about it being too
wordy, although I realize that in some contexts it may be an issue
(short anon funcs, etc.).

I don't know very much the internals of Lua, so I cannot myself judge
about it being cheap enough. Reading the objections that occasionally
come up to that idiom, I got the feeling that people don't think it is
so cheap (relatively speaking).

If it is (and so the subject of this thread would be moot indeed), then
it may be worth, at least to avoid this recurrent complaint, discussing
to some extent its inexpensiveness, so that one could be pointed at a
specific thread whenever in the future the complaint is raised again (if
this extensive discussion has already taken place, then sorry for the
noise and please be so kind to give me a pointer).

> Anything else requires major re-engineering, I suspect; how to
> harmonize it with the rest of the language is probably the hardest
> part.
>
> steve d.
>
> [1] things like |x| x^2 (GSL-shell and metalua) and \x(x^2) (LuaMacro)
>
>
>
-- Lorenzo

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Xavier Wang
In reply to this post by Lorenzo Donati-2


在 2011-7-1 下午1:17,"Lorenzo Donati" <[hidden email]>写道:
>
> Hi All!
>
> This is an attempt to both satisfy a curiosity of mine and
> start a discussion on a topic that seems recurring lately and (IIRC) pops up from time to time.
>
> More specifically, I happened to notice in various threads that there could be a need for a way in Lua to delay the evaluation of expressions.
>
> Lately, in particular, it has been mentioned in the context of error management to avoid building a complex error message when the error actually doesn't occur.
>
> What I wonder now is whether the possibility of delaying the evaluation of an expression would be useful also in other contexts.
>
> If it would, then maybe it is worth considering what should be done to add to Lua such a feature (or a more general one that could comprise it) and what could be the consequences (pros and cons) and if the feature could be general enough to be in "Lua spirit".
>
> Thanks for any contribution.
>
> Cheers.
> -- Lorenzo
>
I have posted a proposal about a new data type of "macro", that macro is same with function except its argument only calculate when they occur in the body of macro, like this:

macro assert(cond, ...)
    if not cond then error (...) end
    return ...
end

The cond only calculate in if, as ... only calculate in error or return.

Is that OK ?

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

steve donovan
In reply to this post by Lorenzo Donati-2
On Fri, Jul 1, 2011 at 8:28 AM, Lorenzo Donati
<[hidden email]> wrote:
> it being cheap enough. Reading the objections that occasionally come up to
> that idiom, I got the feeling that people don't think it is so cheap
> (relatively speaking).

If you have a lot of assertions, then it's true that every such
assertion will generate a new closure. It ain't pretty - whether it is
a problem for a particular application is something that only
profiling can tell.

Here's one little extension which could make life easier in a natural
way, borrowed from Perl:

do_something() or fail "something failed!"

That is, to allow expressions as statements[1].  Because of boolean
operator short-circuiting, anything after 'or' will only be evaluated
when something bad happens.

steve d.
[1] http://lua-users.org/wiki/ExpressionsAsStatements

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Lorenzo Donati-2
On 01/07/2011 8.42, steve donovan wrote:

> On Fri, Jul 1, 2011 at 8:28 AM, Lorenzo Donati
> <[hidden email]>  wrote:
>> it being cheap enough. Reading the objections that occasionally come up to
>> that idiom, I got the feeling that people don't think it is so cheap
>> (relatively speaking).
>
> If you have a lot of assertions, then it's true that every such
> assertion will generate a new closure. It ain't pretty - whether it is
> a problem for a particular application is something that only
> profiling can tell.
>
> Here's one little extension which could make life easier in a natural
> way, borrowed from Perl:
>
> do_something() or fail "something failed!"
>
> That is, to allow expressions as statements[1].  Because of boolean
> operator short-circuiting, anything after 'or' will only be evaluated
> when something bad happens.
>

Yes, I knew that, thanks, and I use some of those idioms from time to time.

On that page it is said that expressions aren't allowed to be statements
to avoid ambiguities. Are there elegant way to avoid those ambiguities?
Now that 5.2 is about to come out, maybe it may be worth addressing this
issue. Or would it be too big a language change?

It would be very elegant to be able to write something like:

do_something() or error "aaargggghhh!"

I once heard something about a "let" statement. Would that solve the
problem in a general and elegant way? Maybe something on the lines:


let
   do_something() or error "aaargggghhh!"
end

Am I missing something?


> steve d.
> [1] http://lua-users.org/wiki/ExpressionsAsStatements
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

steve donovan
In reply to this post by steve donovan
On Fri, Jul 1, 2011 at 8:42 AM, steve donovan <[hidden email]> wrote:
> [1] http://lua-users.org/wiki/ExpressionsAsStatements

I must share one gem from that page, from the ever-prolific Rici Lake:

repeat until nsfm:fileExistsAtPath(testfile) or Shriek "File doesn't exist"

The 'repeat until' is a mouthful, but giving it a new name is a good
job for a preprocessor.

Actually, there is one special kind of expression which is allowed as
a statement, and that is a function call. Allowing _any_ expression to
be a statement would probably result in grammatical ambiguities.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Lorenzo Donati-2
On 01/07/2011 9.08, steve donovan wrote:

> On Fri, Jul 1, 2011 at 8:42 AM, steve donovan<[hidden email]>  wrote:
>> [1] http://lua-users.org/wiki/ExpressionsAsStatements
>
> I must share one gem from that page, from the ever-prolific Rici Lake:
>
> repeat until nsfm:fileExistsAtPath(testfile) or Shriek "File doesn't exist"
>
> The 'repeat until' is a mouthful, but giving it a new name is a good
> job for a preprocessor.
>

I got curious and wrote:

-------------------
local function condition_ok()
end
local function E()
end

repeat until condition_ok() or error "AAARGHH!"

local _ = condition_ok() or error "AAARGHH!"

E( condition_ok() or error "AAARGHH!" )

print "hooray!"
-------------------

Lua 5.1.4 gives as bytecode (comments added):



main <lua_scratchpad2.lua:0,0> (31 instructions, 124 bytes at 003D3D40)
0+ params, 6 slots, 0 upvalues, 3 locals, 4 constants, 2 functions
        1 [3] CLOSURE   0 0 ; 003D3E68
        2 [5] CLOSURE   1 1 ; 003D3F90

-- "repeat until" version

        3 [7] MOVE     2 0
        4 [7] CALL     2 1 2
        5 [7] TEST     2 0 1
        6 [7] JMP       5 ; to 12
        7 [7] GETGLOBAL 2 -1 ; error
        8 [7] LOADK     3 -2 ; "AAARGHH!"
        9 [7] CALL     2 2 2
        10 [7] TEST     2 0 0
        11 [7] JMP       -9 ; to 3

-- "local _" version

        12 [9] MOVE     2 0
        13 [9] CALL     2 1 2
        14 [9] TEST     2 0 1
        15 [9] JMP       3 ; to 19
        16 [9] GETGLOBAL 2 -1 ; error
        17 [9] LOADK     3 -2 ; "AAARGHH!"
        18 [9] CALL     2 2 2

-- "E(...)" version

        19 [11] MOVE     3 1
        20 [11] MOVE     4 0
        21 [11] CALL     4 1 2
        22 [11] TEST     4 0 1
        23 [11] JMP       3 ; to 27
        24 [11] GETGLOBAL 4 -1 ; error
        25 [11] LOADK     5 -2 ; "AAARGHH!"
        26 [11] CALL     4 2 2
        27 [11] CALL     3 2 1


        28 [13] GETGLOBAL 3 -3 ; print
        29 [13] LOADK     4 -4 ; "hooray!"
        30 [13] CALL     3 2 1
        31 [13] RETURN   0 1

function <lua_scratchpad2.lua:2,3> (1 instruction, 4 bytes at 003D3E68)
0 params, 2 slots, 0 upvalues, 0 locals, 0 constants, 0 functions
        1 [3] RETURN   0 1

function <lua_scratchpad2.lua:4,5> (1 instruction, 4 bytes at 003D3F90)
0 params, 2 slots, 0 upvalues, 0 locals, 0 constants, 0 functions
        1 [5] RETURN   0 1



So it seems that the most efficient is "local _ = ...": less bytecode
and no extra function calls.



> Actually, there is one special kind of expression which is allowed as
> a statement, and that is a function call. Allowing _any_ expression to
> be a statement would probably result in grammatical ambiguities.
>
> steve d.
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Rena
In reply to this post by steve donovan
On Fri, Jul 1, 2011 at 01:08, steve donovan <[hidden email]> wrote:

> On Fri, Jul 1, 2011 at 8:42 AM, steve donovan <[hidden email]> wrote:
>> [1] http://lua-users.org/wiki/ExpressionsAsStatements
>
> I must share one gem from that page, from the ever-prolific Rici Lake:
>
> repeat until nsfm:fileExistsAtPath(testfile) or Shriek "File doesn't exist"
>
> The 'repeat until' is a mouthful, but giving it a new name is a good
> job for a preprocessor.
>
> [snip]
>
> steve d.
>
>

At that point we're not far from:
> try assert(doSomething()) catch e handle_error(e) end

compared to:
> repeat until doSomething() or handle_error('oh no')

or:
> local r, e = pcall(doSomething); r = r or handle_error(e)

try/catch is one thing I've often missed in Lua... using pcall instead
tends to be syntactically awkward.

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Jerome Vuarand
In reply to this post by steve donovan
2011/7/1 steve donovan <[hidden email]>:
> On Fri, Jul 1, 2011 at 7:16 AM, Lorenzo Donati
> <[hidden email]> wrote:
>> What I wonder now is whether the possibility of delaying the evaluation of
>> an expression would be useful also in other contexts.
>
> Well, we already have such a mechanism - function() return
> delayed_fun(something) end. Some have felt that this feels over-wordy,
> so that ends up in the 'short function form' discussion meme[1].

So far everyone seem to answer these kind of requests (delayed
evaluation of function parameters) with solution based on one form of
"explicit closure creation". In other languages, there are
alternatives which I'd call "implicit closure creation". For example
in Scala, there is a "by-name" parameter passing mechanism. In
essence, the function itself declares somehow that the expression
passed should not be evaluated at the caller site, but rather wrapped
in a closure and passed to the callee.

Not only is the caller site easier to write, also in the callee code
you just evaluate the parameter, you don't have to explicitly call it.

I understand that this "by-name" mechanism requires the compiler to
know the callee prototype at the caller site, and is therefore well
suited for statically linked languages like Scala but not that much
for Lua. But maybe there is an "implicit closure creation" solution
that would fit Lua.

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

steve donovan
On Fri, Jul 1, 2011 at 12:36 PM, Jerome Vuarand
<[hidden email]> wrote:
> I understand that this "by-name" mechanism requires the compiler to
> know the callee prototype at the caller site,

Also, the programmer needs to know the prototype ;)  I'm thinking here
of C++ reference parameters, which are less obvious than the old C
pass-a-pointer method. C# actually has a keyword 'ref' to make such
parameters stand out more.

So, assuming a list object that can be filtered, we could ask for a
list of all positive elements like so:

lpos = ls:filter(lazy _ > 0)

where lazy <expr> is short for function(_) return <expr> end

Having an explicit keyword/macro/whatever 'lazy' feels better than an
implicit quoting for arguments to this kind of function.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Jerome Vuarand
2011/7/1 steve donovan <[hidden email]>:

> On Fri, Jul 1, 2011 at 12:36 PM, Jerome Vuarand
> <[hidden email]> wrote:
>> I understand that this "by-name" mechanism requires the compiler to
>> know the callee prototype at the caller site,
>
> Also, the programmer needs to know the prototype ;)  I'm thinking here
> of C++ reference parameters, which are less obvious than the old C
> pass-a-pointer method. C# actually has a keyword 'ref' to make such
> parameters stand out more.
>
> So, assuming a list object that can be filtered, we could ask for a
> list of all positive elements like so:
>
> lpos = ls:filter(lazy _ > 0)
>
> where lazy <expr> is short for function(_) return <expr> end
>
> Having an explicit keyword/macro/whatever 'lazy' feels better than an
> implicit quoting for arguments to this kind of function.

The problem with a single keyword is that you have no end delimiter,
and therefore you cannot return multiple values, but maybe that's nice
enough.

I think a goal for all that lazy stuff, would be to allow programmers
use Lua in more functional style, while keeping its simple syntax. For
example it would be nice to be able to write:

  local x = 0
  mywhile x < 10 do
    x = x + 1
    print(x)
  end

Let's dream a bit. With 1. your lazy keyword proposal, 2. a "do end"
short syntax for "function() end", and 3. the ability to avoid
parenthesis around "do end" lambda constructors like it is already the
case for table and string constructors (but that would introduce
ambiguities in current syntax), we could get:

  local x = 0
  mywhile (lazy x < 10) do
    x = x + 1
    print(x)
  end

which is pretty close to the goal, with mywhile defined as:

  local function mywhile_helper(condition, action)
    if condition() then
      action()
      -- return for tail call optimization
      return mywhile_helper(condition, action)
    end
  end

  function mywhile(condition)
    return function(action)
      return mywhile_helper(condition, action)
    end
  end

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

steve donovan
On Fri, Jul 1, 2011 at 7:37 PM, Jerome Vuarand <[hidden email]> wrote:
> The problem with a single keyword is that you have no end delimiter,
> and therefore you cannot return multiple values, but maybe that's nice
> enough.

It would be a deliberate restriction.

> ..., 2. a "do end"
> short syntax for "function() end", and 3. the ability to avoid
> parenthesis around "do end"

I like this one, because it's a fairly common pattern, and continues
the Lua tradition of sugar-coated function calls - with strings, table
constructors, and now a special do..end block.

Compiler will have trouble with this code, of course;

local f = iden
do
  ...
end

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

Matthew Frazier
On 07/02/2011 01:29 PM, steve donovan wrote:

> I like this one, because it's a fairly common pattern, and continues
> the Lua tradition of sugar-coated function calls - with strings, table
> constructors, and now a special do..end block.
>
> Compiler will have trouble with this code, of course;
>
> local f = iden
> do
>    ...
> end
>
> steve d.
>

So you use begin as the keyword instead of do. Like I suggested a few
months ago.
--
Regards, Matthew "LeafStorm" Frazier
http://leafstorm.us/

Reply | Threaded
Open this post in threaded view
|

Re: Delayed evaluation of expressions

steve donovan
On Sun, Jul 3, 2011 at 3:31 AM, Matthew Frazier <[hidden email]> wrote:
> So you use begin as the keyword instead of do. Like I suggested a few months
> ago.

It definitely makes things clearer and less ambiguous. I know people
are reluctant to allow new keywords, although the compiler catches
these things quickly enough. (This does feel like one of the many
'standard' lua-l discussions - a useful thing for a bored person to do
would be to make a catalogue of these recurring themes)

steve d.