[CFD] Local functions for Lua 4.1

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[CFD] Local functions for Lua 4.1

Edgar Toernig
Hi,

with the implementation of full featured lexical scoping in Lua 4.1
I would suggest a change in the semantics of the function statement:

  A function statement with a single identifier as the functions
  name makes the name implicitly local.  That is:

    function name(...

  has the semantic of

    local name
    name = function(...

  The semantics with complex names (x.name etc) stays the same.



The reason I suggest this change is that the current behavior is not
what you would normally expect.  Namely:

  function foo()
    ...
    function bar()
       ...
       bar()
       ...
    end
    ...
    bar()
    ...
  end

While this works as expected, bar becomes a global function and this
is not what you would expect.  It may result in hard to find bugs when
i.e. foo or bar call a function that itself creates a helper function
named bar.  When trying to work around this problem, people may try
this:

  function foo()
    ...
    local bar = function()
       ...
       bar()
       ...
     end
     ...
     bar()
     ...
  end

But this will _not_ work: The local bar will not be visible while the
initialization expression is evaluated so the first (recursive) call
to bar will use the global variable bar while the later call will take
the local bar.  The real solution to this problem is:

  function foo()
    ...
    local bar
    function bar()
      ...
      bar()
      ...
    end
    ...
    bar()
    ...
  end

And IMHO this is what should be the default.  I can't think of very
much cases where you really want a nested function statement to create
a global function.



Now, what about function statements in the top level chunk (main) and
how to define global functions?  One may be tempted to restrict my
proposed change to nested functions.  But I would not do so because
of three reasons: first, by making top level functions local too means
that each file by default will have it's own namespace for functions.
You have to perform explicit actions to export a function.  Second,
access to locals is faster than access to globals.  And at last,
restricting the local behavior to nested functions make the rules
non-uniform, more complicated, and may give other problems.

So, how do you create global functions?  Two methods; either use the
normal assignment syntax:

  foo = function()
    ...
  end

or, what I would prefer, introduce a global variable named 'global'
that holds the global table (global=globals(), exported by baselib)
and then write:

  function global.foo()
    ...
  end

That way you immediately see which functions are local and which are
global.



So, what do you think about this proposal?  Discussion opened :-)

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

RE: [CFD] Local functions for Lua 4.1

Peter Prade
What is so special about function variables?
I guess the same applies also for all other kind of variables:

function foo()
   bar = {}
   -- do somthing with bar
end
foo()
-- the variable bar stays active in the globals
-- while normally no one cares, it is a possible bug, and at least a waste
of resources.

> And IMHO this is what should be the default.  I can't think of very
> much cases where you really want a nested function statement to create
> a global function.
I guess with the new 4.1 locals, the next logical step would be to make all
variables local by default.

> So, what do you think about this proposal?  Discussion opened :-)
Let's take one step after another, not all at once.
This should be a proposal for Lua 4.2 ;-)

Cheers,
Peter


Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Terence Martin-3
Please, this discussion has been opened, closed, kicked, slashed, hashed,
reopened, rediscussed and eventually jammed into soft peat for 3 months and
recycled as firestarters. Do we really need to start another holey debate
about how lua (surprise!) isn't like every other languge, or that 50% of us
want the language to stay as it is and the other 50% of us want it to work
like all the other languges we're "comfortable" with. Along with all the
reasons for so, such as, it makes things easier, less prone to bugs, don't
want to work around it on your own, it's unclean, it seems somehow
unnatural.

Proposals were made for this a long time ago (check the archives). I'm
pretty sure some conclusion was reached, but we have yet to see anything
from it.

> -- the variable bar stays active in the globals
> -- while normally no one cares, it is a possible bug, and at least a
waste
> of resources.
> 
> > And IMHO this is what should be the default.  I can't think of very
> > much cases where you really want a nested function statement to create
> > a global function.
> I guess with the new 4.1 locals, the next logical step would be to make
all
> variables local by default.
> 
> > So, what do you think about this proposal?  Discussion opened :-)
> Let's take one step after another, not all at once.
> This should be a proposal for Lua 4.2 ;-)

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
In reply to this post by Peter Prade
Peter Prade wrote:
> 
> What is so special about function variables?

Nothing.  But the function _statement_ is especially designed to
ease programming and make the code more readable.  The suggestion
is more in line with the for statement that makes it iteration
variable implicitly local.

> I guess with the new 4.1 locals, the next logical step would be to make all
> variables local by default.
>[...] 
> Let's take one step after another, not all at once.

Locals by default is another thing (that introduces a lot of problems;
ask the Python guys) and should be kept separate.

> This should be a proposal for Lua 4.2 ;-)

Hmm... why?  I'm pretty sure, the last snapshot of 4.1 will be different
to the final 4.1.  And so much things have changed to 4.0 that this minor
point shouldn't hurt.  I see it more as a useful addition to the scope
changes.

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
In reply to this post by Terence Martin-3
Terence Martin wrote:
> 
> Proposals were made for this a long time ago (check the archives). I'm
> pretty sure some conclusion was reached, but we have yet to see anything
> from it.

Please read my posting and the archives more closely.  I'm not proposing
for local variables as default!  It's only about the function statement!
And then, with new language features (lexical scope changes) some of the
arguments may need another look.

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Terence Martin-3
I was referring more to "I guess with the new 4.1 locals, the next logical
step would be to make all variables local by default." that Peter added to
your proposal. The notion that everything should be local by default and
made global explicitely, instead of being the opposite as it is now, has
been thrashed around quite a bit. 

IMO, what you proposed is different, in that it's actually a problem, as
opposed to people being upset that the lua designers aren't going out of
their way to add every feature of every language in the universe to lua to
make sure everyone is satisfied with how it works.

Perhaps the two are interrelated enough that going back over that with new
features in mind would be a good idea. 

On Wed, 21 Nov 2001 22:03:45 +0100
Edgar Toernig <[hidden email]> wrote:

> Terence Martin wrote:
> > 
> > Proposals were made for this a long time ago (check the archives). I'm
> > pretty sure some conclusion was reached, but we have yet to see
anything
> > from it.
> 
> Please read my posting and the archives more closely.  I'm not proposing
> for local variables as default!  It's only about the function statement!
> And then, with new language features (lexical scope changes) some of the
> arguments may need another look.
> 
> Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

RLak
In reply to this post by Edgar Toernig
OK, my S/0.02 worth.

It took me a few minutes to see the problem with local bar = function ...
and you're quite right, that's a bugtrap.

And I agree that making the function local is *probably* what the author
would have wanted.

Nonetheless, I don't like the idea of having one default for function and a
different default for variables. Furthermore, one notes that it is
apparently possible to put

local a = 7

at the top level of a file (or chunk) making it local for the duration of
the chunk.

It therefore seems reasonable to me to add the syntax:

local function f(args) body

which would mean exactly the same as

local f
f = function(args) body

I would also argue for the syntactic inclusion of

global v = 7

and

global function g(args) body

in parallel, in case (a) someone wants to make a point or (b) some day the
default scope changes.

Rici.

PS: I'm not actually crazy about changing the default scope to the minimum
scope. I'd rather have the default changed to "chunk local". In this case,
both the local and global modifiers would be required, and it would open
the opportunity to allow the option of strict declarations.
-----------------------------------------------------------
Hi,

with the implementation of full featured lexical scoping in Lua 4.1
I would suggest a change in the semantics of the function statement:

  A function statement with a single identifier as the functions
  name makes the name implicitly local.  That is:

    function name(...

  has the semantic of

    local name
    name = function(...

  The semantics with complex names (x.name etc) stays the same.



The reason I suggest this change is that the current behavior is not
what you would normally expect.  Namely:

  function foo()
    ...
    function bar()
       ...
       bar()
       ...
    end
    ...
    bar()
    ...
  end

While this works as expected, bar becomes a global function and this
is not what you would expect.  It may result in hard to find bugs when
i.e. foo or bar call a function that itself creates a helper function
named bar.  When trying to work around this problem, people may try
this:

  function foo()
    ...
    local bar = function()
       ...
       bar()
       ...
     end
     ...
     bar()
     ...
  end

But this will _not_ work: The local bar will not be visible while the
initialization expression is evaluated so the first (recursive) call
to bar will use the global variable bar while the later call will take
the local bar.  The real solution to this problem is:

  function foo()
    ...
    local bar
    function bar()
      ...
      bar()
      ...
    end
    ...
    bar()
    ...
  end

And IMHO this is what should be the default.  I can't think of very
much cases where you really want a nested function statement to create
a global function.



Now, what about function statements in the top level chunk (main) and
how to define global functions?  One may be tempted to restrict my
proposed change to nested functions.  But I would not do so because
of three reasons: first, by making top level functions local too means
that each file by default will have it's own namespace for functions.
You have to perform explicit actions to export a function.  Second,
access to locals is faster than access to globals.  And at last,
restricting the local behavior to nested functions make the rules
non-uniform, more complicated, and may give other problems.

So, how do you create global functions?  Two methods; either use the
normal assignment syntax:

  foo = function()
    ...
  end

or, what I would prefer, introduce a global variable named 'global'
that holds the global table (global=globals(), exported by baselib)
and then write:

  function global.foo()
    ...
  end

That way you immediately see which functions are local and which are
global.



So, what do you think about this proposal?  Discussion opened :-)

Ciao, ET.


Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
[hidden email] wrote:
> 
> Nonetheless, I don't like the idea of having one default for function and a
> different default for variables.

There's no difference.  The local-statement introduces a new local as does
the for-statement or the new definition of the function-statement.  It only
tries to give a more sensible semantic to the "syntactic sugar" of the
function-statement (that it introduces a new local).

> local function f(args) body

I thought about that too.  But I prefer the local to be implicit.  You don't
want to write "for local i=1,100 do..." either ;-)

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

James Hearn
In reply to this post by RLak
Modifying the function statement to allow the use of a 'local' or 'global'
prefix makes sense to me in the case ET gives. It brings the connotation of
function declaration (local function foo() end) more in line with variable
declaration (local foo = 10), while still avoiding the can of worms which
would be making all variables local by default.

Once possible negative effect is that it (local function foo() end) would be
semantically different from (local foo = function() end) in the context of
the described function, which may be nonobvious. On the other hand, the only
option with the function statement currently is (function foo() end) which
leaves no choice to have the function be created locally, unlike vanilla
variable declarations which allow the use of a scoping specifier.

The only pitfall I can see is that now one would have to differentiate
between (local function foo() foo() end) and (local  foo = function () foo()
end), which do different things. The nested foo() in the first example calls
itself recursively, while the nested foo in the second example calls some
global foo. The first would instead be semantically identical to (local foo
function foo() foo() end).

I'm not a language designer, but this seems to make sense to me. It
introduces a new subtle difference between the different ways of creating a
function, which could cause bugs; however, some would consider the current
behavior a bug. It's a matter of picking one's poison.

I'm sure I've missed something here. Feel free to enlighten me :)

--James Hearn


Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

John Belmonte-2
In reply to this post by Edgar Toernig
> So, what do you think about this proposal?  Discussion opened :-)
>
> Ciao, ET.

A slippery slope, this proper lexical scoping...

It's a sound proposal, but what about backwards compatibility?  How much
code is out there doing "function x()" at the top scope and expecting to
generate a global?

Someday it would nice to have a mechanism for gradually introducing
incompatibilities like Python's "__future__" module
(http://python.sourceforge.net/peps/pep-0236.html).

-John



Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
John Belmonte wrote:
> 
> A slippery slope, this proper lexical scoping...

Yeah.  But I think one only has to get used to it.

> It's a sound proposal, but what about backwards compatibility?

There are already so much changes in the 4.1 snapshots that _I_
would call the final version 5.0.

> How much code is out there doing "function x()" at the top scope
> and expecting to generate a global?

I'm not really sure that there are so many that expect a global.
Most of them just expect that they may access the function from
any scope and that will still be the case most of the time.

The exception are files intended to be dofile'd that provide globals.
I think people writing these "modules" would even prefer a simpler
seperation between different files.

And then there's always the choice to stay with Lua 4.0 if large
part of your application is already written in 4.0 ;-)

> Someday it would nice to have a mechanism for gradually introducing
> incompatibilities like Python's "__future__" module

I'm not sure how this is supposed to work with syntactical or semantic
changes.

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
Edgar Toernig wrote:
> 
> John Belmonte wrote:
> >
> > Someday it would nice to have a mechanism for gradually introducing
> > incompatibilities like Python's "__future__" module
> 
> I'm not sure how this is supposed to work with syntactical or semantic
> changes.

Uhh... I see.  The compiler catches the "from __future__ import" statement
and implements all possible variants.  To me this sounds like somebody
wants to be only a little bit pregnant ;-)

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Eduardo Ochs-2
In reply to this post by Edgar Toernig
> So, what do you think about this proposal?  Discussion opened :-)
>
> Ciao, ET.

I think that the current behaviour is simpler --

  function foo() body end

is just syntactic sugar for

  foo = function() body end

and nothing else. It could be nice to allow things like

  local function foo() body end

but then, to make things consistent,

  local foo, function bar() body end

would have to be translated to

  local foo, bar = function() body end

and that would assign the value `function() body end' to foo, instead
of to bar, and some people would be confused...

In my opinion the best solution is to discipline people to write
things like

  local foo = function() body end

when they are intending to do strange things with the functions that
they create (I can't think of any interesting variation now, sorry). I
like the fact the Lua has *NO* function declarations, only function
values, that are assigned to variables; and a function value is a
pointer to a sequence of bytes stored somewhere, that contains the
bytecode for the function, and that sequence of bytes can even be
garbage-collected when nothing else references it -- just like strings
or tables.

Lua is the most elegant language that I've stumbled on in many years,
and it is by far the simplest to learn and to use; I would like to
keep it that way.

BTW, here is what I think that we should be dreaming of -- in the long
term, of course -- instead of other rules of scope:

  1) Nice drawings of the Lua datatypes so that more people could
     readily grasp the details of what Lua "really" does. My canonical
     examples of what are good drawings of this type always come from
     the Emacs Lisp manual: for example,
       <http://angg.twu.net/info2www.cgi?(elisp)Lists+as+Boxes> and
       <http://angg.twu.net/info2www.cgi?(elisp)Cons+Cell+Type>.

  2) Another Lua parser, written in Yacc or in Lua, and possibly much
     slower than the default one, that people could change more easily.

  3) Small changes to the standard parser to let people replace parts
     of it -- like the code that says how it converts, say, a
     "function foo() body end" construct to bytecode -- on the fly.

I think that (3) is much in the Lua spirit -- the default Lua parser
would become a meta-mechanism for implementing similar languages with
slightly different rules of scope... and it starts in a state that is
simple and quite convenient for writing small programs...

  Buenas salenas,
    Eduardo Ochs   (who never understood C++, Python or Java),
    http://angg.twu.net/
    http://www.mat.puc-rio.br/~edrx/
    [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Reuben Thomas-4
>   2) Another Lua parser, written in Yacc or in Lua, and possibly much
>      slower than the default one, that people could change more easily.

luayacc and lualex in the style of Ocaml's camlyacc and camllex would be
wonderful (the good thing about Ocaml's lex and yacc is that they are
really just preprocessors for the compiler, so there is no sectioning of
code, they just let you use syntactically simple constructs for embedding
bits of lexers and parsers in your program. It's a very nice way to
program if you're parsing text a lot.)

> I think that (3) is much in the Lua spirit -- the default Lua parser
> would become a meta-mechanism for implementing similar languages with
> slightly different rules of scope... and it starts in a state that is
> simple and quite convenient for writing small programs...

This too is very much the way to go. Anything which lowers the bar to
experimenting with the language is a good thing; instead of having to
write patches in C, you could test the idea in Lua; for efficiency good
ideas might be better implemented in C, and very rarely you might want to
add a new feature to the core implementation.

For those who use Lua for scripting, however, a Lua written entirely in
Lua (apart from the VM) would be really useful. It's a pity that it would
necessarily perform rather badly, but for many applications it would be
much more flexible than the current implementation. If only there were
ways of messing with types to reduce the amount of dynamic checking that
you had to do, you could make Lua efficient too, but that is really
dreaming... (I am thinking of things like being able to assert the type of
a variable or argument, so that the program becomes at the same time
faster and unsafe).


Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

RLak
In reply to this post by Edgar Toernig
Edgar Toernig escribió:

>>
>> Nonetheless, I don't like the idea of having one default for function
and a
>> different default for variables.

> There's no difference.  The local-statement introduces a new local as
does
> the for-statement or the new definition of the function-statement.  It
only
> tries to give a more sensible semantic to the "syntactic sugar" of the
> function-statement (that it introduces a new local).

I don't think that's quite accurate, unless you intend that global
functions *always* need to be flagged as global using what I consider to be
an ugly syntax. (function global.foo ...) (but that's just one person's
opinion.) Otherwise, every function would be chunk-local, which is probably
not intuitive either.

>> local function f(args) body

> I thought about that too.  But I prefer the local to be implicit.  You
don't
> want to write "for local i=1,100 do..." either ;-)

Actually, I wouldn't mind if there was an alternative. I would actually be
like to use local in while loops sometimes, but I can't think of a good
syntax since assignment is not an operator (and I usually approve of that
choice).

As I mentioned in some earlier posting, I don't actually like the 4.1alpha
version of

for i = 1, 100 do ... end

which is more or less equivalent to

begin local i = 1
  while i <= 100 do ... i = i+1 end
end

with the consequence that if I want to create a closure with the index
variable I need to say

for i = 1, 100 do local i = i ... end

In this particular case, I'd be happier being able to say

for local i = 1, 100 do ... end

which would be easier to read (and type).

On the other hand, I'd be just as happy if the semantics of a for-loop were
changed with respect to the scope of a local.

....

Mientras tanto, Eduardo Ochs escribió:

> It could be nice to allow things like

>  local function foo() body end

> but then, to make things consistent,

>  local foo, function bar() body end

> would have to be translated to

>  local foo, bar = function() body end

> and that would assign the value `function() body end' to foo, instead
> of to bar, and some people would be confused...

Well, some people are going to be confused no matter what :-) I get
confused a lot, myself.

However, the definition I proposed of local function f(args) body end was
not:

local f = function (args) body end

but rather

local f
f = function(args) body end

for the reasons explained by Edgar.

I failed to mention in my original proposal that I would actually like the
syntax to be something like:

local function f(args1) body1 end , function g(args2) body2 end ...

to be equivalent to:

local f, g
f = function(args1) body1 end
g = function(args2) body2 end

which would handle mutual recursion properly (unlike Edgar's proposal,
which wouldn't). However, I accept that this is both inconsistent (with,
for example, local a,b = b,a)
and subtle (the comma could easily be ignored in the absence of good
margins and syntax colouring.)

This is the letrec of Scheme-like languages, but the word "letrec" has
always struck me as opaque. It could be that a better syntax would be:

local functions f(args1) body1 end, g(args2) body2 end

or even

localfunctions f(args1) body1 end, g(args2) body2 end

but that's creeping keyword-ism.

This is not a formal proposal, you understand. Just random thoughts to
hopefully contribute to the debate.

Rici


Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Nick Trout-2
In reply to this post by Edgar Toernig
> The reason I suggest this change is that the current behavior is not
> what you would normally expect.  Namely:
>
>   function foo()
>     ...
>     function bar()
>        ...
>        bar()
>        ...
>     end
>     ...
>     bar()
>     ...
>   end


Could you not use the new "global" keyword to stop this?  (I think this has
been added)

eg.

 function foo()
   global
   function bar()
       bar()
    end
    bar()
 end

This will cause an error as bar() is not defined at global level, so you must
do

 function foo()
   global
   local bar
   function bar()
       bar()
    end
    bar()
 end

or

 function foo()
   global
   local bar = function bar()
       bar()
    end
    bar()
 end

This would stop your problem but whether you want to add: local function() end
is a different matter.

Nick



Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Nick Trout-2
In reply to this post by Terence Martin-3
Sorry, my last post wont solve the problem, but it will point out that there
is a problem!

Terence Martin:
> Please, this discussion has been opened, closed, kicked, slashed, hashed,
> reopened, rediscussed and eventually jammed into soft peat for 3 months and
> recycled as firestarters. Do we really need to start another holey debate
> about how lua (surprise!) isn't like every other languge

Do you mean "holy" debate? :-)  I'm not smirking at you, I think its a
Freudian misspelling! The argument did rage for a while and was difficult to
follow but I think people learnt a lot from it. I think are a few little
nagging details of the scoping issue which could probably do with being
addressed and this is one of them.

Riki:
>> I would also argue for the syntactic inclusion of
global v = 7
and
global function g(args) body

You may be able to do this with the global keyword already (untested) eg.
function foo()
  global v
  v = 7
end

just like:
function foo()
  local v
  v = 7
end

I think the global keyword only turns on testing of globals and doesnt allow
you to create them from with another scope (although Python does allow you to
do this).

ET:
>> There are already so much changes in the 4.1 snapshots that _I_
would call the final version 5.0.

I think I agree. 3.2 to 4.0 introduced major API changes. 4.0 to 4.1
introduces major syntax changes. I think calling it version 5.0 better
reflects the amount of discussion and effort that has gone into the version
change.

Regards,
Nick







Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
In reply to this post by RLak
[hidden email] wrote:
> 
> local function f(args1) body1 end , function g(args2) body2 end
>
> local f, g
> f = function(args1) body1 end
> g = function(args2) body2 end
> 
> which would handle mutual recursion properly (unlike Edgar's proposal,
> which wouldn't).

The first pure technical point against my proposal and in fact such a
strong one that I redraw it unless I find a proper solution for this
problem.

>[...]
> and subtle (the comma could easily be ignored in the absence of good
> margins and syntax colouring.)

I 100% agree.

Thanks, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
I wrote:
> 
> ... that I redraw it unless I find a proper solution

Upps typo, sorry.  Make the "unless" an "until".

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: [CFD] Local functions for Lua 4.1

Edgar Toernig
I wrote:
> 
> > ... that I redraw it unless I find a proper solution
> 
> Upps typo, sorry.  Make the "unless" an "until".

Not my day today :-(  Looking for a brown paperbag...

"... that I withdraw it until I find a proper solution"

Sorry again, ET.

12