In praise of globals

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

In praise of globals

Dirk Laurie-2
2013/4/15 Brian Kelley <[hidden email]>:

> I look forward to a future Lua that abandons globals completely in favor
> of manifest upvalues!

That remark comes from another thread, and it's maybe not fair to pick
on it, but it is an extreme example of something that has been bothering
me for a long time.

    It's getting to be politically correct in the Lua community
    to regard globals as bad.

The attitude (the term 'prejudice' almost sneaked out, but I bit it back)
is illustrated most clearly by the pejorative term "polluting the global
namespace".

You can tell the programs written by the supporters of this ideology.
They start out something like this:

    local pairs, ipairs, print, tostring, getmetatable, setmetatable
        = pairs, ipairs, print, tostring, getmetatable, setmetatable

There are also plenty of forward declarations of local functions.

Now, I don't want to knock this style of programming altogether. I've used
it myself. But to preach that is the One Correct Way is going too far.
If one is using Lua via the interactive interpreter (which I do a lot)
it is intensely annoying to cut-and-paste code written this way. Those
locals last only for the chunk in which they are defined, and the
interactive interpreter bites off the smallest chunks possible. I keep
forgetting to make extra `do`..`end` blocks all the time.

In Lua, it's a mistake, a Pythonic mistake, to think of local variables
as the usual thing. Local variables are special. You must declare them
explicitly. Their names don't survive compilation. You can have at
most 200 of them in one scope. They should be used with care, and only
for the purpose they were designed for, which is to be short-lived.

Globals, on the other hand, don't have to be declared, have long-lived
names, and you can have as many as you like.

You look at a `luac` listing of your compiled bytecode, the names are
there for all to see.  They are just keys in a table.

Compare the following two programming assignments:

1. Write a function 'showglobal(str)' that prints out the value of the
   global variable whose name is the given parameter.

2. Write a function 'showlocal(str)' that prints out the value of the
   local variable whose name is the given parameter.

I've made myself two laws of for naming functions.

   1. Don't make it local if you plan exporting it, but define it inside
      its table to begin with.

   2. Make it global if it is used so often that it almost feels
      as if it is part of the language.

Look, if you are a global-hater, by all means don't use them. Assign
those from Lua's standard library that you can't do without to local
variables if you must. But don't insist that other people must do so too.

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Vadi
I have to agree with you. Abandoning globals in at least one use case I know, where complete programming newbies and non-programmers are using embedded Lua for scripting, would wreak havoc. I prefer the option to be there instead.
Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Josh Simmons
The reason people so often create local copies of global functions is
partially because it's significantly faster, and partially because
it's necessary if you're using `module`.

I'm not quite sure what the point of this thread is to be honest.

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

steve donovan
In reply to this post by Dirk Laurie-2
On Mon, Apr 15, 2013 at 8:38 AM, Dirk Laurie <[hidden email]> wrote:
The attitude (the term 'prejudice' almost sneaked out, but I bit it back)
is illustrated most clearly by the pejorative term "polluting the global
namespace".

It is a particularly painful prejudice in the C++ universe; there was a near-riot when one of the leading lights wrote this line:

using namespace std; // so sue me
 
   1. Don't make it local if you plan exporting it, but define it inside
      its table to begin with.

That's a simple enough rule, although of course the table itself might be local.  

   2. Make it global if it is used so often that it almost feels
      as if it is part of the language.

Here we have one of those 'beauty is in the eye of the beholder' things.  Since everyone's taste and priorities differ, then using such libraries ends up leaving _G cluttered.  By all means provide a way to inject globals (I use pl.utils.import for this) but one has to respect the Lua state of one's users.  In particular, what's convenient in interactive shells is a special but important case.

I do agree that pushing the 'correct way' is unnecessary pedantry, especially for newcomers and 'casual' scripters as Vadim says. Sometimes bicycles need training wheels.

There is a non-ideological use of local on the module level, which is for optimization. Then Knuth's remark always applies ;)

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Pierre Chapuis
In reply to this post by Dirk Laurie-2
> 2013/4/15 Brian Kelley <[hidden email]>:
>
>     It's getting to be politically correct in the Lua community
>     to regard globals as bad.
>
> The attitude (the term 'prejudice' almost sneaked out, but I bit it back)
> is illustrated most clearly by the pejorative term "polluting the global
> namespace".

I have said that sentence more than once, and I think it is one of
the most important things to care about if we want a thriving Lua
ecosystem.

> You can tell the programs written by the supporters of this ideology.
> They start out something like this:
>
>     local pairs, ipairs, print, tostring, getmetatable, setmetatable
>         = pairs, ipairs, print, tostring, getmetatable, setmetatable
>
> There are also plenty of forward declarations of local functions.

This idiom (redeclaring the standard library functions locally) does
not have much to do with the issue of global namespace pollution.

When we say "don't pollute the global namespace" we mean something
else, see below.

> Now, I don't want to knock this style of programming altogether. I've used
> it myself. But to preach that is the One Correct Way is going too far.
> If one is using Lua via the interactive interpreter (which I do a lot)
> it is intensely annoying to cut-and-paste code written this way. Those
> locals last only for the chunk in which they are defined, and the
> interactive interpreter bites off the smallest chunks possible. I keep
> forgetting to make extra `do`..`end` blocks all the time.

I agree, but this is a technical problem of the interpreter.

So what do we (or at least I) mean by "don't pollute the global namespace"?
We mean that you should not create globals in *library code*.

Why is that? Imagine that your program depends on library A which uses
global foo. Now *you* use global foo, and library A stops working.

Now you can say "don't do that, check which globals libraries are using".
OK. Now let's say you depend on library A and library B. Library B also
uses global foo. There is no way library B author could have known you
would use it with library A.

Even worse: library B depends on library C which uses global foo...

These issues are not imaginary, they are real, they happen when you make
large scale programs. I am not even talking about modules that abuse the
fact that some modules expose globals to monkey-patch them and do it
wrong: https://gist.github.com/catwell/3803669

I have not written require "socket.unix" to discover this bug, I have written
require "redis", because redis-lua required socket.unix (it has been patched
since then).

So if you use Redis you cannot use TCP sockets anymore. This is the kind of
thing that should not happen. I don't care what you do within your own
codebase, but libraries should not break other code when required.

--
Pierre Chapuis


Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Enrico Colombini
On 15/04/2013 10.34, Pierre Chapuis wrote:
> So what do we (or at least I) mean by "don't pollute the global namespace"?
> We mean that you should not create globals in*library code*.

I was about to write the same thing.

--
   Enrico

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Miles Bader-2
In reply to this post by Josh Simmons
Josh Simmons <[hidden email]> writes:
> The reason people so often create local copies of global functions is
> partially because it's significantly faster, and partially because
> it's necessary if you're using `module`.
>
> I'm not quite sure what the point of this thread is to be honest.

Seriously, the original claim ("It's getting to be politically correct
in the Lua community to regard globals as bad") seemed kind of dubious
in the first place (and the use of charged pejorative terms like
"politically correct" suggests a polemic rather an open-minded
argument).

I think a certain suspicion of globals is evident amongst _any_
programming community that's reached a certain level of sophistication
(for good reasons!), and the Lua community seems to be pretty average
in this sense.

-miles

--
Opposition, n. In politics the party that prevents the Goverment from running
amok by hamstringing it.

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

steve donovan
In reply to this post by Pierre Chapuis
On Mon, Apr 15, 2013 at 10:34 AM, Pierre Chapuis <[hidden email]> wrote:
These issues are not imaginary, they are real, they happen when you make
large scale programs. I am not even talking about modules that abuse the
fact that some modules expose globals to monkey-patch them and do it
wrong: https://gist.github.com/catwell/3803669

Ouch!  That is most definitely a fail.  To burden a person working on their own complex problem with that kind of module issue is not fair.

The first law of monkey-patching is "Don't". The second is .. "Not yet...".

I'd say that module developers work under special constraints, in order to allow their users to relax and do whatever monkey-patching _they_ desire to do!   A library writer does extra work, so that the users don't have to.

This indeed has nothing to do with 'local pairs,ipairs = pairs,ipairs'. We all know these guys are hanging around, wanting to be used. But this can indeed speed things up. However, optimization should never be a spinal reflex, and can never be reduced to dogma. Here I suspect that a local alias of pairs would hardly make any difference, whereas a local alias of tostring could be worth it. (Although I appreciate that this style allows dependencies to presented explicitly up-front, which is a good thing for your readers. It would be straightforward to integrate a static globals analysis tool with an IDE to make this boilerplate generation a keystroke away)


Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Philipp Janda
In reply to this post by Dirk Laurie-2
Am 15.04.2013 08:38 schröbte Dirk Laurie:
>
> [...]
>
> You can tell the programs written by the supporters of this ideology.
> They start out something like this:
>
>      local pairs, ipairs, print, tostring, getmetatable, setmetatable
>          = pairs, ipairs, print, tostring, getmetatable, setmetatable

IMHO, that has the following advantages (in the context of modules):
-   you document the dependencies of your module (even though Lua's
standard library is small, there are still cases where not all standard
functions are available)
-   if used like 'local setmetatable = assert( setmetatable )' you fail
early, not in the middle of something important, if a required function
is not available
-   no one can mess with your functions once you have them copied
(whether you consider this a good thing depends on how you think about
monkey patching)
-   luac5.1 -l also lists line numbers -- if "luac5.1 -l x.lua | grep
ETGLOBAL" gets you get a line number that is not at the beginning of the
file, you found a typo (or a missing dependency)
-   you say more accurately what you want, which is that those functions
be available *to you* (you typically don't care if they are also
available to everyone else)
-   you can also use it for sandboxing if for some reason you can't or
don't want to use separate environments -- just remove all unwanted
globals after loading the module

That's why I prefer the above technique. Obviously avoiding additional
globals in third-party modules is much more important, but others have
already written about that.

Philipp




Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Philipp Janda
In reply to this post by steve donovan
Am 15.04.2013 11:43 schröbte steve donovan:
>
> The first law of monkey-patching is "Don't". The second is .. "Not yet...".

Wasn't that about optimization? Oh, and btw., the second rule is "for
experts only" ...

Ah, found it[1].

   [1]: https://en.wikipedia.org/wiki/Program_optimization#Quotes



Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

steve donovan
In reply to this post by Philipp Janda
On Mon, Apr 15, 2013 at 11:51 AM, Philipp Janda <[hidden email]> wrote:
-   no one can mess with your functions once you have them copied (whether you consider this a good thing depends on how you think about monkey patching)

This particular point does depend on the order of require() calls, of course.

This is good defensive programming. These are good points, and the extra effort is (a) worthwhile and (b) mostly automatable.

As for assert(setmetatable), it's less painful to make this a 'compile-time' problem with an appropriate static analysis tool.  And I suspect that the 'luac' trick could be covered as well.


Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Matthew Wild
In reply to this post by steve donovan
On 15 April 2013 10:43, steve donovan <[hidden email]> wrote:

> On Mon, Apr 15, 2013 at 10:34 AM, Pierre Chapuis <[hidden email]>
> wrote:
>>
>> These issues are not imaginary, they are real, they happen when you make
>> large scale programs. I am not even talking about modules that abuse the
>> fact that some modules expose globals to monkey-patch them and do it
>> wrong: https://gist.github.com/catwell/3803669
>
>
> Ouch!  That is most definitely a fail.  To burden a person working on their
> own complex problem with that kind of module issue is not fair.
>
> The first law of monkey-patching is "Don't". The second is .. "Not yet...".
>
> I'd say that module developers work under special constraints, in order to
> allow their users to relax and do whatever monkey-patching _they_ desire to
> do!   A library writer does extra work, so that the users don't have to.

Heh, we've been considering for a while a helper library in Prosody,
available to plugin authors, specifically to aid with monkey-patching
:)

People can and will do it anyway. It often breaks things - for example
reloading plugins must clean up after themselves, which is not hard to
do (just restore the original function), but when you have multiple
layers of monkey-patching the same 'victim' function (I've only seen
it once!) things get weird...

So as evil as it is, if we were to make a helper library that
magically made everything work as expected, surely that's at least an
improvement? :)

Regards,
Matthew (who, chuckling while he wrote, probably made more than a
dozen people cry out at this mail)

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

steve donovan
On Mon, Apr 15, 2013 at 12:18 PM, Matthew Wild <[hidden email]> wrote:
So as evil as it is, if we were to make a helper library that
magically made everything work as expected, surely that's at least an
improvement? :)

Totally!  Evil is a very relative term, much abused in debates about programming language usage.  (I think it should be reserved for breaches of the Geneva Convention and so forth, but I digress)

With a simple patch/unpatch pair, can keep a stack and penalize people when they don't provide the matching unpatch... a certain amount of hand-slapping is needed for the Greater Good.

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Philipp Janda
In reply to this post by steve donovan
Am 15.04.2013 12:09 schröbte steve donovan:
> On Mon, Apr 15, 2013 at 11:51 AM, Philipp Janda <[hidden email]> wrote:
>
>> -   no one can mess with your functions once you have them copied (whether
>> you consider this a good thing depends on how you think about monkey
>> patching)
>>
>
> This particular point does depend on the order of require() calls, of
> course.

Of course. If the environment is already messed up before you are
require'd, you can't do anything about it[*], but typically modules are
loaded before the heavy lifting starts ...

   [*]: I suspect unit-testing all Lua library functions at module load
time goes a bit far ...

>
> As for assert(setmetatable), it's less painful to make this a
> 'compile-time' problem with an appropriate static analysis tool.  And I
> suspect that the 'luac' trick could be covered as well.
>

The static analysis tool can't handle all possible situations where
someone might want to use a module. Silly example: Consider writing a
plugin for "Runes of Magic" an MMORPG that can be scripted in Lua 5.1.
If I want to use a third-party module for that I can dofile it (require
is not available in RoM), and if the red icon flashes (assuming the
module asserts on all used Lua functions), this module is not compatible
with the game, and I can start looking for another one before writing
any code myself. From time to time the set of available functions
changes, when the developers have found another way how to write bots.
It's better to find that out when loading the game and not when you are
in the middle of a group of monsters.

I suspect there are lots of restricted Lua environments, where debug is
not available, or the io/os modules are missing (setfenv/getfenv are
likely candidates as well). One might configure a static analysis tool
for the most common cases, but covering all of them is difficult.
Also setfenv/_ENV might complicate the matter for a static tool.

Automating those checks might be possible in some circumstances. E.g., I
have written a Lua-based build tool (like so many before me ;-) ), which
uses lhf's lbci to check that for each GETGLOBAL there is a
corresponding executable in PATH before actually running the build
script. I still use an assert-like mechanism for other dependencies like
include files, libraries, etc., but so far it works like a charm -- I
can even throw multiple build scripts for different build environments
at it and it will figure out which one to execute (or at least tell me
what is missing and where it is expected).

Philipp



Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

steve donovan
On Mon, Apr 15, 2013 at 1:55 PM, Philipp Janda <[hidden email]> wrote:
set of available functions changes, when the developers have found another way how to write bots. It's better to find that out when loading the game and not when you are in the middle of a group of monsters.

Good advice for life, not just game programming!

I suspect there are lots of restricted Lua environments, where debug is not available, or the io/os modules are missing (setfenv/getfenv are likely candidates as well). One might configure a static analysis tool for the most common cases, but covering all of them is difficult.

I handle this in lglob by -wx which is given an 'exclusive' whitelist, containing known globals:

tostring=true
ipairs=true
....

For restricted environments, a blacklist might make more sense, however, since they are often best defined by what's removed.

Also setfenv/_ENV might complicate the matter for a static tool.

Oh it does. I decided simply not to support the more odd uses ;)


Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

David Heiko Kolf-2
In reply to this post by Pierre Chapuis
Pierre Chapuis wrote:
> So what do we (or at least I) mean by "don't pollute the global namespace"?
> We mean that you should not create globals in *library code*.
[...]
> I don't care what you do within your own
> codebase, but libraries should not break other code when required.

I strongly agree with this point, a library should never write globals
unless explicitly instructed to do so and it is good when all the
dependencies are listed at the top.

In regard to the quote from the original post where it was said that
globals should be completely replaced by upvalues, I can think of a
situation where upvalues can actually be harmful and globals (or sandbox
environments or any table lookups) are the more useful way:

One of my first exercise projects with Lua was starting the development
of a server for Multi-User Dungeons (text-based role-playing games).  A
MUD server usually runs for many months without even an application
restart so it is extremely important to be able to swap code parts in
the running application.

Once a function is stored somewhere as an upvalue it is very hard to
change its code, so all functions should be accessed through table
lookups and on reload the existing tables are updated.  The way to find
out whether a table already exists usually involves the global namespace
(or the sandbox in which the script is loaded).

    my_room = my_room or {}

    function my_room.open_the_door()
      ...
    end

I guess this situation extends to many applications where live coding is
desired.

Best regards,

David Kolf

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Steve Litt
In reply to this post by Dirk Laurie-2
On Mon, 15 Apr 2013 08:38:40 +0200
Dirk Laurie <[hidden email]> wrote:

> 2013/4/15 Brian Kelley <[hidden email]>:
>
> > I look forward to a future Lua that abandons globals completely in
> > favor of manifest upvalues!
>
> That remark comes from another thread, and it's maybe not fair to pick
> on it, but it is an extreme example of something that has been
> bothering me for a long time.
>
>     It's getting to be politically correct in the Lua community
>     to regard globals as bad.

Globals suck. At least, globals intended to be or contain data. An
abdication of encapsulation.

>
> The attitude (the term 'prejudice' almost sneaked out, but I bit it
> back) is illustrated most clearly by the pejorative term "polluting
> the global namespace".
>
> You can tell the programs written by the supporters of this ideology.
> They start out something like this:
>
>     local pairs, ipairs, print, tostring, getmetatable, setmetatable
>         = pairs, ipairs, print, tostring, getmetatable, setmetatable

Yeah, that's going a little far. Everyone knows what ipairs is -- no
need to localize it.

>
> There are also plenty of forward declarations of local functions.
>
> Now, I don't want to knock this style of programming altogether. I've
> used it myself. But to preach that is the One Correct Way is going
> too far.

Within limits, you're right. The initial idea of banning globals is
stupid. What is EXPECIALLY stupid is banning global functions that
don't hold any static data -- what's the point unless you're writing a
library and are worried about clobbering somebody else's function name?

I started the preceding paragraph with "within limits". My point was
that I don't want to see Lua (or any other language) turn into Perl,
where "many ways to do it" morphs into "do your own thing, screw the
maintenance programmer!" Lua (and almost every other language) is
soooo far from that Perlistic "ideal", that it's not worth a second
thought.

So I think you're right that globals shouldn't be banned. On the other
hand, you're not going to see me using many global variables holding
data any time soon. To my way of thinking, it's a lot more encapsulated
to pass locals as function arguments. And of course, with upvalues and
the like, Lua gives all sorts of alternatives to globals that lesser
languages don't bestow.

Thanks,

SteveT

Steve Litt                *  http://www.troubleshooters.com/
Troubleshooting Training  *  Human Performance

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Dirk Laurie-2
2013/4/15 Steve Litt <[hidden email]>:

> So I think you're right that globals shouldn't be banned. On the other
> hand, you're not going to see me using many global variables holding
> data any time soon.

I like to use the word pseudo-globals for local variables whose scope
is all of the program file, and I declare them at the top.

They usually do contain data :-) but basically I agree with you.

Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Thomas Jericke
In reply to this post by Dirk Laurie-2
I don't think I can entirely agree with this praise.

My problem is that globals make it too easy for a writer of a Lua script to pollute the Lua state by mistake, and I don't want to sandbox all my users scripts, because then some of the "good" mechanisms of globals wont work any more.

So one way would be the much discussed declaration of globals (which is the solution we use in our project, something similar to StrictLua).

And I think you can get rid of many problems with some syntactic sugar:
On 04/15/2013 08:38 AM, Dirk Laurie wrote:
You can tell the programs written by the supporters of this ideology.
They start out something like this:

    local pairs, ipairs, print, tostring, getmetatable, setmetatable
        = pairs, ipairs, print, tostring, getmetatable, setmetatable
One could add a new keyword: using, or import
     using pairs, ipairs, print, tostring, getmetatable, setmetatable
Would just do the same as you wrote, then

  using pairs, ipairs from table
would be the same as
  local pairs, ipairs = table.pairs, table.ipairs

actually it might even be possible to define a syntax without new keywords, maybe:
  local pairs, ipairs in _ENV
  local pairs, ipairs in require "table"
ect.

This is just an example. But in my view, something like that would be needed if we really want to use less globals, because I think the basic functions you need often should be easily available without writing a lot of - let's call it - unproductive code.

In Lua, it's a mistake, a Pythonic mistake, to think of local variables
as the usual thing. Local variables are special. You must declare them
explicitly. Their names don't survive compilation. You can have at
most 200 of them in one scope. They should be used with care, and only
for the purpose they were designed for, which is to be short-lived.
I don't think we would have upvalues in Lua if this was true. Actually I like upvalues the most as they live exactly as long as they need to. Local function and closures are my favorite "higher level" mechanisms in Lua :) I should write a "In praise of the closures" text :-)
Globals, on the other hand, don't have to be declared, have long-lived
names, and you can have as many as you like.

Well actually if you have a local table you can store as many off them as you want.

What would be great for me, is to ban globals as LHS values.
One would have to write _ENV.print = function donotprint() end instead of print = function donotprint()
but
  print "hello"
or
  local x = print
would still be allowed. I think that would be just fine because almost all mistakes with globals happen on the LHS. That means globals would be read only unless you access them via _ENV or _G.
-- 

Thomas
Reply | Threaded
Open this post in threaded view
|

Re: In praise of globals

Dimiter 'malkia' Stanev
In reply to this post by Steve Litt
>> You can tell the programs written by the supporters of this ideology.
>> They start out something like this:
>>
>>      local pairs, ipairs, print, tostring, getmetatable, setmetatable
>>          = pairs, ipairs, print, tostring, getmetatable, setmetatable
>
> Yeah, that's going a little far. Everyone knows what ipairs is -- no
> need to localize it.

I think the point here is to avoid being "monkey patched" - e.g. someone
might chagne what pairs, ipairs, ... do affecting your module.

Effectively what Dirk is trying to say - is that there is so much global
state, that you might have to learn with it.

To fully isolate from global state, the first thing you have to do is to
capture (e.g. "un-free") all global functions, vars, etc.

I only wish there was easier way to do that in lua/luajit, a pattern
along the line of "local something = something" to be just "import
something" or "use something" or something like it.


123