Lua strictlessness

classic Classic list List threaded Threaded
35 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Dirk Laurie-2
2013/8/27 Sven Olsen <[hidden email]>:

> Enforcing naming discipline on locals is a mostly unrelated topic -- and
> most examples of local shadowing will happen inside nested blocks, a
> situation where they're not, actually, likely to be a source of memory
> inefficiencies.

Enforcing discipline of whatever kind is the category  of often-suggested
Lua "enhancements" that I hate most.

The only good discipline is self-discipline.

Anybody who does enough programming acquires certain habits willy-nilly,
and it's not a bad idea to acquire some good ones consciously.

For example, currently I put my entire file in one huge do-block. The
locals I declare at the top of that block feel like globals, but actually
are upvalues. In general, I declare locals only directly after 'do', 'then' and
'function', except when there is a cogent reason to transgress: then
I comment that exception.

I'm not saying you should do exactly these too, merely that they represent
a conscious attempt at forming some good habits.

Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Thijs Schreijer
In reply to this post by Sven Olsen
Considering that the discussion focusses on checking at compile time, why not move one step beyond and simply resolve it by static analysis. The compiler benefit would only be for dynamically loading code in your application like user scripts, plugins etc. and in those cases, with untrusted code, an extra local is the least of your worries.

ZeroBrane Studio code analysis does a good job at finding shadowing variable definitions (as well as unintended globals).

Other than that it is just discipline to make sure that large data structures are released early by assigning nil explicitly, or by using more do blocks so they automatically go out of scope.

Thijs


Sven Olsen <[hidden email]>schreef:

My point, however, was that the reuse of stack slots by the compiler involves more than just a quick check to see if the new local name matches any existing one at the same scope.

Yes, I think we're actually in agreement here.  While it's probably doable, a "strict locals" patch won't help programmers avoid most of the suboptimal code that involves memory needlessly trapped by local variables.  For example, consider:

function chunk()
  local a=big_table()
  do_something(a)
  do_something_else()
end

The problem here is that, if do_something_else() takes a while, and uses a lot of resources, a will still be hogging up memory while it runs -- Lua's only confident that it can release a after chunk() returns.

The fix is to wrap local a inside it's own block.  i.e.:

function chunk()
  do
    local a=big_table()
    do_something(a)
  end
  do_something_else()
end

(Now a can be freed before do_something_else() is called.)

So, while you could certainly hack together a "strict locals" patch, one that would do for locals about what strict.lua does for globals -- I'm not convinced it's a particularly good idea.  If the problem we're trying to fix is unnecessary memory use by locals, the solution is to use more do-blocks.  Enforcing naming discipline on locals is a mostly unrelated topic -- and most examples of local shadowing will happen inside nested blocks, a situation where they're not, actually, likely to be a source of memory inefficiencies.  

-Sven
Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

steve donovan
On Tue, Aug 27, 2013 at 10:55 AM, Thijs Schreijer
<[hidden email]> wrote:
> ZeroBrane Studio code analysis does a good job at finding shadowing variable
> definitions (as well as unintended globals).

That really is the most sensible way to go - do it as early as
possible using a suitable IDE.

I will update lglob to add this kind of check, for those who prefer to
use a plain editor.

Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Tim Hill

On Aug 27, 2013, at 2:41 AM, steve donovan <[hidden email]> wrote:

> On Tue, Aug 27, 2013 at 10:55 AM, Thijs Schreijer
> <[hidden email]> wrote:
>> ZeroBrane Studio code analysis does a good job at finding shadowing variable
>> definitions (as well as unintended globals).
>
> That really is the most sensible way to go - do it as early as
> possible using a suitable IDE.
>
> I will update lglob to add this kind of check, for those who prefer to
> use a plain editor.
>

While smart editors are good, it's also a good idea to be able to catch these kinds of things as warnings during a build imho.

--Tim


Reply | Threaded
Open this post in threaded view
|

RE: Lua strictlessness

Dan Tull
> > That really is the most sensible way to go - do it as early as
> > possible using a suitable IDE.
> While smart editors are good, it's also a good idea to be able to catch these kinds of
> things as warnings during a build imho.

Yes, the same linter engine that provides live feedback in our IDE is run headless as
part of a build for just that reason.

That said, while we have undeclared globals, unused locals,  and shadow locals, we
only consider undeclared globals to be worthy of a build break.

Those are almost always a bug, whereas unused locals are (relatively) harmless.

There's one case of shadow locals I'd seriously consider including:

  function foo:bar( self, a, b ) -- <-- linter complains about self being shadowed

  end

I have never seen a case where that wasn't a declaration typo, but other kinds of
shadowing are less unambiguously wrong. I'm very conservative about false
positives when doing analysis.

The other lint hit that we could (but don't) include in the build is this one:

  if not foo == "bar" then

  end

I call it "not equals misuse" lint and it's always a mistake, but rarely happens once
people are used to the language, so we haven't bothered including it, either.

Dan Tull
Lua Tools Dev
Adobe

Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Andrew Starks

On Tue, Aug 27, 2013 at 3:20 PM, Dan Tull <[hidden email]> wrote:
if not foo == "bar" then

  end

I call it "not equals misuse" lint and it's always a mistake, but rarely happens once
people are used to the language, so we haven't bothered including it, either.

FWIW: This one gets me often.

-Andrew
Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Mark Hamburg-3
In reply to this post by Dan Tull
Any warning for shadowing would almost certainly want to make an exception for '_' since it is frequently used for don't care variables including in constructs like:

local _, _, third = returnThree()

Mark


Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Tim Hill
In reply to this post by Dan Tull

On Aug 27, 2013, at 1:20 PM, Dan Tull <[hidden email]> wrote:

The other lint hit that we could (but don't) include in the build is this one:

 if not foo == "bar" then

 end

I call it "not equals misuse" lint and it's always a mistake, but rarely happens once 
people are used to the language, so we haven't bothered including it, either.

I've seen a few "false positives" on this one, but only because our build includes a Lua macro processor, and sometimes you end up with constructs like this only because the part after the "not" is hidden inside a (perfectly valid) macro.

--Tim

Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Tim Hill

On Aug 27, 2013, at 8:01 PM, Tim Hill <[hidden email]> wrote:


On Aug 27, 2013, at 1:20 PM, Dan Tull <[hidden email]> wrote:

The other lint hit that we could (but don't) include in the build is this one:

 if not foo == "bar" then

 end

I call it "not equals misuse" lint and it's always a mistake, but rarely happens once 
people are used to the language, so we haven't bothered including it, either.

I've seen a few "false positives" on this one, but only because our build includes a Lua macro processor, and sometimes you end up with constructs like this only because the part after the "not" is hidden inside a (perfectly valid) macro.

Sigh… I meant "hidden positives" not "false".

--Tim


Reply | Threaded
Open this post in threaded view
|

RE: Lua strictlessness

Dan Tull
In reply to this post by Tim Hill
> > if not foo == "bar" then
> >
> > end
> I've seen a few "false positives" on this one...
In my case, I'm analyzing the bytecode and only flag it as a hit if
the right hand side is a constant pool entry of a type other than
boolean, which I think is a pretty solid assurance that the if body
will never execute because the types will mismatch.

It means I may miss some "bad" cases, but false positives haven't
come up so far.

> because our build includes a Lua macro processor, and sometimes
> you end up with constructs like this only because the part after the
> "not" is hidden inside a (perfectly valid) macro.
Interesting. Yeah, we don't use macros. I guess if we did I'd probably
have to analyze the bytecode after replacement.

DT

Reply | Threaded
Open this post in threaded view
|

RE: Lua strictlessness

Dan Tull
In reply to this post by Tim Hill
> Sigh… I meant "hidden positives" not "false".
Oops. This is what I get for reading in order and responding before finishing.

Yes, there could definitely be hidden positives.

DT
Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Robert Virding
In reply to this post by Mark Hamburg-3
I think the first thing to decide is whether creating a new local variable is a bug or a feature. Currently it is a feature so at best the compiler could warn that this is happening as it can create strange errors. If it should be a bug then you need to decide whether to allow multiple declarations of the SAME local variable or not. It is definitely not a difficult thing for the compiler to check however you decide to do it, in fact having separate variables can actually be more difficult.

Coming with an argument that you don't need a check, all you need is enforcing discipline, is IMAO a very bad option as it hits the people who can least defend themselves, inexperienced users. They are the ones who are least likely to know about it at all (the manual doesn't shout out loud about it) and the least likely to understand what is going wrong. Checking it in tools is ok, sort of, but the inexperienced user is the one who is least likely to know about the tools.

So I think at least the compiler should at least warn by default, with an option to keep quiet. Preferably disallow it.

Robert

----- Original Message -----

> From: "Mark Hamburg" <[hidden email]>
> To: "Lua mailing list" <[hidden email]>
> Cc: "Lua mailing list" <[hidden email]>
> Sent: Tuesday, 27 August, 2013 11:10:15 PM
> Subject: Re: Lua strictlessness
>
> Any warning for shadowing would almost certainly want to make an exception
> for '_' since it is frequently used for don't care variables including in
> constructs like:
>
> local _, _, third = returnThree()
>
> Mark
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

steve donovan
On Thu, Aug 29, 2013 at 3:53 PM, Robert Virding
<[hidden email]> wrote:
people who can least defend themselves, inexperienced users. They are
the ones who are least likely to know about it at all (the manual
doesn't shout out loud about it) and the least likely to understand
what is going wrong. Checking it in tools is ok, sort of, but the
inexperienced user is the one who is least likely to know about the
tools.

A very good point.  It suggests that someone experiment with the idea
of 'training Lua' which is an unusually strict (but helpful) version
of the compiler.  Although there is nothing _that_ special about the
Lua executable, and we could provide versions that embed some static
bytecode checks written in Lua such as lglob.

Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Luiz Henrique de Figueiredo
In reply to this post by Robert Virding
> So I think at least the compiler should at least warn by default, with an option to keep quiet. Preferably disallow it.

How do you propose to handle warnings in a embedded language? The Lua
parser cannot emit warnings to stdout or stderr because these files may
not even exist or be visible. Moreover, it'd be impolite to trample on
the host's output. So, should the parser maintain a list of warnings for
the host to check? What can the host do about them?

Reply | Threaded
Open this post in threaded view
|

Re: Lua strictlessness

Robert Virding
"The same way you handle errors today" would be my reply. You can have errors today in your code which the parser/compiler must handle, even in embedded systems, this would be logical place to handle warnings as well.

Robert

----- Original Message -----

> From: "Luiz Henrique de Figueiredo" <[hidden email]>
> To: "Lua mailing list" <[hidden email]>
> Sent: Thursday, 29 August, 2013 4:16:01 PM
> Subject: Re: Lua strictlessness
>
> > So I think at least the compiler should at least warn by default, with an
> > option to keep quiet. Preferably disallow it.
>
> How do you propose to handle warnings in a embedded language? The Lua
> parser cannot emit warnings to stdout or stderr because these files may
> not even exist or be visible. Moreover, it'd be impolite to trample on
> the host's output. So, should the parser maintain a list of warnings for
> the host to check? What can the host do about them?
>
>

12