Locals by default yet again, sorry :)

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Locals by default yet again, sorry :)

Abhijit Nandy
Hi,

So we have quite a large Lua + Luabind code base and new comers from other languages often tend to forget the word local when declaring local variables in a block/scope. Of course the Lua static analyzer flags such code and we fix it, but sometimes such code still slips through basically because this particular warning is not an error at the moment.

What I am wondering is, apart from using the static analyzer to catch it, is there any code patch that can make variable declarations local by default?
Or is it too core to the language that changing this could cause issues elsewhere? I am willing to sacrifice performance as we are on PC and its worth doing this for us to prevent erroneous code if it can be done without requiring significant change to syntax.

I had a look at http://lua-users.org/wiki/LocalByDefault but there does not yet seem to be any solutions for Lua 5.3 for doing this globally in the language.

Thanks,
Abhijit
Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

Philippe Verdy
Do you mean that any code that has not explicitly declared a variable as "external" or "local" (or defined as a function parameter) ***and visible in the same source file*** must imply that "local" must apply by default ?

The problem may the definition of "file-level scope" : what can happen with code generators or code templates: these templates do not work if they can't use outer variables that are declared by other templates in the appropriate scope.

But for simple autonomous .lua files, this rule would be enough. Code template should not use the ".lua" extension as they are intended to be used within a preprocessor that will assemby and repack them into a runnable lua source "file". Such preprocessor may be for code generator, or it could be a "streaming" preprocessor (taking input from a continuous flow, e.g. a Lua I/O console, whose code is executed progressively as it is entered, while keeping the existing "local" variables and their values between multiple inputs of the same stream: the "file" level in that case is the whole stream, and not an isolated input; this is a special case where Lua code gets parsed, compiled and executed even if it is not complete...).


Le mar. 30 juil. 2019 à 08:45, Abhijit Nandy <[hidden email]> a écrit :
Hi,

So we have quite a large Lua + Luabind code base and new comers from other languages often tend to forget the word local when declaring local variables in a block/scope. Of course the Lua static analyzer flags such code and we fix it, but sometimes such code still slips through basically because this particular warning is not an error at the moment.

What I am wondering is, apart from using the static analyzer to catch it, is there any code patch that can make variable declarations local by default?
Or is it too core to the language that changing this could cause issues elsewhere? I am willing to sacrifice performance as we are on PC and its worth doing this for us to prevent erroneous code if it can be done without requiring significant change to syntax.

I had a look at http://lua-users.org/wiki/LocalByDefault but there does not yet seem to be any solutions for Lua 5.3 for doing this globally in the language.

Thanks,
Abhijit
Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

Sergey Kovalev
In reply to this post by Abhijit Nandy
вт, 30 июл. 2019 г. в 09:45, Abhijit Nandy <[hidden email]>:

>
> Hi,
>
> So we have quite a large Lua + Luabind code base and new comers from other languages often tend to forget the word local when declaring local variables in a block/scope. Of course the Lua static analyzer flags such code and we fix it, but sometimes such code still slips through basically because this particular warning is not an error at the moment.
>
> What I am wondering is, apart from using the static analyzer to catch it, is there any code patch that can make variable declarations local by default?
> Or is it too core to the language that changing this could cause issues elsewhere? I am willing to sacrifice performance as we are on PC and its worth doing this for us to prevent erroneous code if it can be done without requiring significant change to syntax.
>
> I had a look at http://lua-users.org/wiki/LocalByDefault but there does not yet seem to be any solutions for Lua 5.3 for doing this globally in the language.
>
I experimenting with "noupvalues" may be it could be useful
Here is bash script to build it
[build-lua-5.4-noupvalues.sh]
#/bin/sh
wget -c https://www.lua.org/work/lua-5.4.0-alpha.tar.gz
wget https://raw.githubusercontent.com/kov-serg/lua-aux/master/noupvalues54.patch
tar xf lua-5.4.0-alpha.tar.gz
cd lua-5.4.0-alpha
patch -p 1 < ../noupvalues54.patch
make linux-readline
cp src/{lua,luac} ..
cd ..

You can use it for local by default.
synatax https://www.lua.org/manual/5.3/manual.html#9
...
funcbody ::= ‘(’ [parlist] ‘)’ [noupvalues] block end
...


x=1 local y,z=2,3
function fn() noupvalues _ENV={ print=print }
  print(x,y,z) -- nil,nil,nil
  x,y,z=10,20,30
  print(x,y,z) -- 10,20,30 all unknown variables will be in local _ENV
  return _ENV
end
fn()
print(x,y,z) --1,2,3

or

function strict(s) s=s or {}
  return setmetatable({},{
    __index=function(t,n)
      if s[n]==nil then error("no "..n.." defined") end
      return s[n]
    end,
    __newindex=function(t,n,v)
      if s[n]==nil then error("no "..n.." defined") end
      s[n]=v
    end,
  })
end

local z=30
function fn(_ENV) noupvalues
  local x,y=10,20
  print(x,y,z) -- error: no z defined
end

fn(strict{ print=print })

Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

Sergey Kovalev
Instead
> function fn() noupvalues _ENV={ print=print }
Should be
function fn() noupvalues local _ENV={ print=print }

Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

pocomane
In reply to this post by Abhijit Nandy
On Tue, Jul 30, 2019 at 8:45 AM Abhijit Nandy <[hidden email]> wrote:
> What I am wondering is, apart from using the static analyzer to catch it, is there any code patch that can make variable declarations local by default?

How do you access globals then?
Have you to use `__ENV.name` everywhere ?
So, could to completely forbid the globals work for you?

If this is the case, I just wrote a patch some day ago. It is for the
last commit on github [1] but it is very simple, probably you can
adapt it to the official release. I attached the last version.

The patch lets you to mark a do/end block with one of the following:

- <autoglobal>: it works like the common do/end block
- <withupvalue>: a free name inside this block raises an error instead
of being translated to `__ENV.name`
- <localonly>: an error is raised when accessing an upvalue too (so
local variables only are allowed, it is similar to the noupvalues by
Sergey)

Note:

- A non-marked do/end block inherits the behaviour from the parent block.
- The main chunk starts in the <autoglobal> mode (as usual).

For your use case, I think you could do one of the following:

1) in the patch, you can change the default for the main chunk to
<withupvalue>, or
2) in your lua code, you can write a wrapper for the lua function
"load" that puts any lua code in a

do <withupvalue>
  -- paste here the original script
end

> Or is it too core to the language that changing this could cause issues elsewhere? I am willing to sacrifice performance as we are on PC and its worth doing this for us to prevent erroneous code if it can be done without requiring significant change to syntax.

The patch rises error at compile time, so the performance should not
be affected.

Please note that the patch is somehow "Compatible" with the regular
lua: if you write a script that works as expected with the patch, you
can remove any mark from the do/end blocks, and the result is valid
for the regular lua too [2].

pocomane

[1] https://github.com/lua/lua/commit/9e6807c3c9d5036e999f636f936df07b72284442
[2] Of course, if no do/end block is marked it is alright valid for a
regular lua. And this is true also if you changed the default mode, or
if you wrapped the "load" function.

do_attrib_proof_of_concept.diff (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

Pedro Tammela-2
In reply to this post by Abhijit Nandy
Hi,

So we have quite a large Lua + Luabind code base and new comers from other
languages often tend to forget the word *local *when declaring local
variables in a block/scope. Of course the Lua static analyzer flags such
code and we fix it, but sometimes such code still slips through basically
because this particular warning is not an error at the moment.

What I am wondering is, apart from using the static analyzer to catch it,
is there any code patch that can make variable declarations local by
default?
Or is it too core to the language that changing this could cause issues
elsewhere? I am willing to sacrifice performance as we are on PC and its
worth doing this for us to prevent erroneous code if it can be done without
requiring significant change to syntax.

I had a look at http://lua-users.org/wiki/LocalByDefault but there does not
yet seem to be any solutions for Lua 5.3 for doing this globally in the
language.

Thanks,
Abhijit

Perhaps https://github.com/lua-stdlib/strict could be of interest.
Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

Sergey Kovalev
In reply to this post by pocomane
> - <localonly>: an error is raised when accessing an upvalue too (so
> local variables only are allowed, it is similar to the noupvalues by
> Sergey)
I try "do <localonly>".It is not similar to noupvalues. It is like
pure function more.
I think patch with pure fuction was too vicious in many cases.
So noupvalues more friendly. It simply hides all upvalues except _ENV.
But now I cann't say what kind is better.

Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

pocomane
On Tue, Jul 30, 2019 at 4:37 PM Sergey Kovalev <[hidden email]> wrote:
> I try "do <localonly>".It is not similar to noupvalues. It is like
> pure function more.
> I think patch with pure fuction was too vicious in many cases.
> So noupvalues more friendly. It simply hides all upvalues except _ENV.
> But now I cann't say what kind is better.

Yes, it is not fully equivalent, but you can workaround the limitation
with something like:

local env = _ENV -- or local _ENV = _ENV if you prefer
do <localonly>
  env.print"ok"
end

However I think <withupvalue> is closer to what Abhijit was looking
for (if he can live without direct access to globals).

Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

Abhijit Nandy
In reply to this post by Pedro Tammela-2
Thanks Pedro. I'll have a look. Yes we are using a similar tool called luacheck to detect unintended globals:  https://github.com/mpeterv/luacheck

Abhijit

On Tue, Jul 30, 2019 at 7:17 PM Pedro Tammela <[hidden email]> wrote:
Hi,

So we have quite a large Lua + Luabind code base and new comers from other
languages often tend to forget the word *local *when declaring local
variables in a block/scope. Of course the Lua static analyzer flags such
code and we fix it, but sometimes such code still slips through basically
because this particular warning is not an error at the moment.

What I am wondering is, apart from using the static analyzer to catch it,
is there any code patch that can make variable declarations local by
default?
Or is it too core to the language that changing this could cause issues
elsewhere? I am willing to sacrifice performance as we are on PC and its
worth doing this for us to prevent erroneous code if it can be done without
requiring significant change to syntax.

I had a look at http://lua-users.org/wiki/LocalByDefault but there does not
yet seem to be any solutions for Lua 5.3 for doing this globally in the
language.

Thanks,
Abhijit

Perhaps https://github.com/lua-stdlib/strict could be of interest.
Reply | Threaded
Open this post in threaded view
|

Re: Locals by default yet again, sorry :)

Abhijit Nandy
In reply to this post by pocomane
Thanks everyone for the suggestions :)

It will take me a while to fully absorb them, I ll get back with any queries after trying them all out.

Abhijit

On Tue, Jul 30, 2019 at 8:18 PM pocomane <[hidden email]> wrote:
On Tue, Jul 30, 2019 at 4:37 PM Sergey Kovalev <[hidden email]> wrote:
> I try "do <localonly>".It is not similar to noupvalues. It is like
> pure function more.
> I think patch with pure fuction was too vicious in many cases.
> So noupvalues more friendly. It simply hides all upvalues except _ENV.
> But now I cann't say what kind is better.

Yes, it is not fully equivalent, but you can workaround the limitation
with something like:

local env = _ENV -- or local _ENV = _ENV if you prefer
do <localonly>
  env.print"ok"
end

However I think <withupvalue> is closer to what Abhijit was looking
for (if he can live without direct access to globals).