modules, require, magic

classic Classic list List threaded Threaded
160 messages Options
123456 ... 8
Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Sean Conner
It was thus said that the Great Josh Simmons once stated:

> On Tue, Oct 18, 2011 at 6:20 PM, Sean Conner <[hidden email]> wrote:
> >
> >  "But module() and require() could keep a private table with references to
> > each module and not dump them into the global space," you say.  Okay, but
> > you *still* need a global namespace, because DarkGod has *his* DNS module
> > (named dns) and I have *my* DNS module (which would also be named "dns"), so
> > which one is returned when you require("dns")?  (assuming that I have
> > modules I wrote using my DNS module, and some other modules I want to use
> > might be using DarkGod's DNS module)
> >
>
> Require already does this, and you need to separate them at the
> filesystem level irregardless of what scope they're imported into.
>
> local dns = require 'crazy.weird.dns'
> local dns2 = require 'somebody.elses.dns'
>
> None of this has anything to do with global scope or module.

  My Lua paths have several locations where modules can lurk:

package.path:

        /home/spc/.luarocks/share/lua/5.1/?.lua
        ./?.lua
        /usr/local/share/lua/5.1/?.lua
        /usr/local/lib/lua/5.1/?.lua
        /home/spc/work/ecid/tests/common/lua/?.lua

package.cpath:

        /home/spc/.luarocks/lib/lua/5.1/?.so
        ./?.so
        /usr/local/lib/lua/5.1/?.so
        /home/spc/work/ecid/tests/common/lib/?.so

(that last path in each is for work related Lua stuff)

  And just doing a

        require "dns"

will grab the first file matching dns.lua or dns.so, and woe to code that
expects CrazyWeird DNS module (living under /usr/local/lib/lua/5.1/dns.so)
and gets SomebyElse's DNS module (living under
/home/spc/.luarocks/lib/lua/5.1/dns.so).

  -spc (So yeah, the filesystem does this to a degree, but you can still
        get messed up ... )



Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Josh Simmons
On Tue, Oct 18, 2011 at 6:47 PM, Sean Conner <[hidden email]> wrote:

> It was thus said that the Great Josh Simmons once stated:
>> On Tue, Oct 18, 2011 at 6:20 PM, Sean Conner <[hidden email]> wrote:
>> >
>> >  "But module() and require() could keep a private table with references to
>> > each module and not dump them into the global space," you say.  Okay, but
>> > you *still* need a global namespace, because DarkGod has *his* DNS module
>> > (named dns) and I have *my* DNS module (which would also be named "dns"), so
>> > which one is returned when you require("dns")?  (assuming that I have
>> > modules I wrote using my DNS module, and some other modules I want to use
>> > might be using DarkGod's DNS module)
>> >
>>
>> Require already does this, and you need to separate them at the
>> filesystem level irregardless of what scope they're imported into.
>>
>> local dns = require 'crazy.weird.dns'
>> local dns2 = require 'somebody.elses.dns'
>>
>> None of this has anything to do with global scope or module.
>
>  My Lua paths have several locations where modules can lurk:
>
> package.path:
>
>        /home/spc/.luarocks/share/lua/5.1/?.lua
>        ./?.lua
>        /usr/local/share/lua/5.1/?.lua
>        /usr/local/lib/lua/5.1/?.lua
>        /home/spc/work/ecid/tests/common/lua/?.lua
>
> package.cpath:
>
>        /home/spc/.luarocks/lib/lua/5.1/?.so
>        ./?.so
>        /usr/local/lib/lua/5.1/?.so
>        /home/spc/work/ecid/tests/common/lib/?.so
>
> (that last path in each is for work related Lua stuff)
>
>  And just doing a
>
>        require "dns"
>
> will grab the first file matching dns.lua or dns.so, and woe to code that
> expects CrazyWeird DNS module (living under /usr/local/lib/lua/5.1/dns.so)
> and gets SomebyElse's DNS module (living under
> /home/spc/.luarocks/lib/lua/5.1/dns.so).
>
>  -spc (So yeah, the filesystem does this to a degree, but you can still
>        get messed up ... )
>
>
>
>

What I'm getting at is this is tangential to module issues, if your
paths are crappy you're in trouble no matter how you name your module
and if your paths are good it doesn't matter whether you use module or
not.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

steve donovan
In reply to this post by Josh Simmons
On Tue, Oct 18, 2011 at 9:33 AM, Josh Simmons <[hidden email]> wrote:
> Interestingly enough this is actually worse if you use module and they
> both name themselves 'dns' in which case the value of the global just
> depends on who got in last.

This is true, but it's more a social problem. Sean's solution seems a
little extreme but at least he takes namespacing seriously!  And we
can always just ignore the global ;)

Personally, I'd declare myself as agnostic on the module() question.
It's dangers have been oversold, but personally I use the no-magic
style in public projects because then the code can integrate with any
other style, including sandboxed code.

(Moving Penlight away from module() was not trivial - can't do this
without a lot of tests, and the last problems only disappeared when I
started to use a global-checking script)

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Sean Conner
It was thus said that the Great steve donovan once stated:

> On Tue, Oct 18, 2011 at 9:33 AM, Josh Simmons <[hidden email]> wrote:
> > Interestingly enough this is actually worse if you use module and they
> > both name themselves 'dns' in which case the value of the global just
> > depends on who got in last.
>
> This is true, but it's more a social problem. Sean's solution seems a
> little extreme but at least he takes namespacing seriously!  And we
> can always just ignore the global ;)
>
> Personally, I'd declare myself as agnostic on the module() question.
> It's dangers have been oversold, but personally I use the no-magic
> style in public projects because then the code can integrate with any
> other style, including sandboxed code.

  A question:  Two modules use Penlight, foo and bar.  If I include foo and
bar into a project, does that mean I have two instances of Penlight loaded?

  -spc




Reply | Threaded
Open this post in threaded view
|

Penlight and globals (was: modules, require, magic)

Pierre Chapuis
In reply to this post by steve donovan
On Tue, 18 Oct 2011 09:59:08 +0200, steve donovan wrote:

> (Moving Penlight away from module() was not trivial - can't do this
> without a lot of tests, and the last problems only disappeared when I
> started to use a global-checking script)

Since you say that, I wanted to ask: is there a way to make Penlight
compatible with its own "strict" module?

What I mean is that using the latest version of Penlight in LuaRocks
(0.9.5-1) I get:

     > require "pl.strict"
     > require "pl"
     /usr/share/lua/5.1/pl/utils.lua:230: variable 'pack' is not
declared
     stack traceback:
         [C]: in function 'error'
         /usr/share/lua/5.1/pl/strict.lua:41: in function
</usr/share/lua/5.1/pl/strict.lua:39>
         /usr/share/lua/5.1/pl/utils.lua:230: in main chunk
         [C]: in function 'require'
         /usr/share/lua/5.1/pl/init.lua:20: in main chunk
         [C]: in function 'require'
         stdin:1: in main chunk
         [C]: ?


Reply | Threaded
Open this post in threaded view
|

RE: modules, require, magic

Julien Duminil
In reply to this post by Sean Conner
>   A question:  Two modules use Penlight, foo and bar.  If I include foo
> and
> bar into a project, does that mean I have two instances of Penlight
> loaded?

require stores "modules" that have already been required in package.loaded[modname] so that if you load the same module twice, it loads it only once:
http://www.lua.org/manual/5.1/manual.html#pdf-require


Reply | Threaded
Open this post in threaded view
|

Re: Penlight and globals (was: modules, require, magic)

steve donovan
In reply to this post by Pierre Chapuis
On Tue, Oct 18, 2011 at 11:06 AM, Pierre Chapuis <[hidden email]> wrote:
> Since you say that, I wanted to ask: is there a way to make Penlight
> compatible with its own "strict" module?

Oops, fell off the to-do list. Clearly my jihad against globals is not
yet finished ;)

Will be sorted out for next release, (ETA 2-3 days)

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Krunal Rao
In reply to this post by Luiz Henrique de Figueiredo
Luiz Henrique de Figueiredo <lhf <at> tecgraf.puc-rio.br> writes:

> This way works fine for both 5.1 and 5.2
>
> There is no need for a module function and no need for syntax either.
> No one needs to know about _ENV for that only for complicated things
> like avoiding writing "M." (probably not a good idea as it serves as
> documentation of what is exported).
>
> All my C modules follow this (but do create a global as well). Changing
> them to 5.2 is really easy, even removing the creation of the global.
>
> I do not see any big deal here.

I completely agree with all of the above.

I have written not-trivial (well, for me at least :P) libraries following this
mechanism and never ever had an issue with it. I agree that modules should not
pollute the global "namespace".

What is done is perfectly clear to everybody (newbies and seasoned developers
alike) and requires 1 extra single line versus the module approach. If a global
entry is wanted it's also very easy to add it as well from user side.








Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

steve donovan
In reply to this post by Julien Duminil
On Tue, Oct 18, 2011 at 11:05 AM, Julien Duminil
<[hidden email]> wrote:
> require stores "modules" that have already been required in package.loaded[modname] so that if you load the same module twice, it loads it only once:

Yes, it's a brilliant feature that we take for granted.  (dofile()
will do exactly what you tell it to do)

$> lua -lpl.utils
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> = package.loaded['pl.utils']
table: 0036BCC0

(as an aside, if you change a module and want to reload inside a
session, then package.loaded['mod'] is what you need to zap to nil to
actually get a re-load.)

Penlight uses some serious voodoo to prevent excessive code loading.
require 'pl' appears to be a kitchen-sink include, but the modules
(such as lapp, tablex, etc) are only loaded on first use.  Of course,
module writers and generally careful people don't just say require
'pl' ;)

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Xavier Wang
In reply to this post by Sam Roberts
2011/10/18 Sam Roberts <[hidden email]>:

> On Mon, Oct 17, 2011 at 9:07 AM, Petite Abeille
> <[hidden email]> wrote:
>>
>> On Oct 17, 2011, at 4:28 PM, Javier Guerra Giraldez wrote:
>>
>>> module() is gone,
>>
>> It's not gone. It's alive and kicking in 5.1.
>>
>> 5.2 is not released. In 5.2, it might  be deprecated. Which means it's going to be there for years to come.
>>
>>> the replacement is far simpler,
>>
>> Bollox :)
>>
>>> I really don't get why to keep discussing about it.
>
> Because module and require worked fine, and provided a common way to
> implement modules.
>
> Now we are trying to support two ways, check out this code at about line 60:
>
> https://github.com/dcurrie/lunit/blob/master/lunit.lua
>
May be this will be better:

if _VERSION == 'Lua 5.2' then
    lunit = {}
     _ENV = lunit
    package.loaded[...] = lunit
end


then you needn't to return lunit table.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

joao lobato
In reply to this post by Paul E. Merrell, J.D.
On 10/18/11, marbux <[hidden email]> wrote:

> On Mon, Oct 17, 2011 at 3:59 PM, joao lobato <[hidden email]>
> wrote:
>> On 10/17/11, Josh Simmons <[hidden email]> wrote:
>
>> Actually, what I don't understand is dofile; I always find it wierd
>> that it can't be passed arguments...
>
> Might you enlighten me with a use case that can't be worked around via
> dofile's ability to pass global variables?
>
> (Not trying to be sarcastic; I'm close to being a novice scripter and
> sincerely do not see a use case.)
>
>

My point was simply that dofile can be replaced by something like

function my_dofile(name,...)
  return assert(loadfile(name))(...)
end

and still end up with more functionality; in a way I was alluding that
we never had a dostring (it's pretty much the same idiom but with the
deprecated loadstring).

Globals tend to be dangerous, you have no guarantees regarding what
part of the script can end up mucking your state; shared memory is
always a source of endless problems.

Of course the point loses much of its weight for small scripts with a
single developer.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

steve donovan
On Tue, Oct 18, 2011 at 11:53 AM, joao lobato <[hidden email]> wrote:
> and still end up with more functionality; in a way I was alluding that
> we never had a dostring

Ah but we did - it got dropped ;)

> Globals tend to be dangerous, you have no guarantees regarding what
> part of the script can end up mucking your state;

Amen to that. One way of deciding this in a large system is to ask the
question "how much damage can a misspelling do?".  You end up having
to practically ban globals.

Fortunately, the Lua community is not so gung-ho about
'monkey-patching' which seems to be a frequent cause of problems among
the Rubyistas.

> Of course the point loses much of its weight for small scripts with a
> single developer.

Yes, so when we hear criticism from someone like Mark Hamburg you know
it comes from someone who handles hundreds of thousands of lines of
Lua.

Big Lua requires serious discipline!  And it's harder to do it
afterwards, as I found with the mere 10K lines of source for Penlight.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Mark Hamburg
In reply to this post by Sean Conner
On Oct 18, 2011, at 12:20 AM, Sean Conner wrote:

>  Assume for a moment there are several other modules that require this
> module.  Without a global namespace, each module will get their own copy of
> this module, which would mean wasted space as there are multiple copies of
> this data (and that paths table could be pretty big ... [1])

require will only load a module once. Subsequent uses of require will just return this same module.

The globals issue has to do with whether a change is made to the globals table. You can still refer to your module as org.conman.dns and require will do the right thing.

The danger with changing the globals table is that code might start referencing org.conman.dns without doing an appropriate require and would just happen to work because some other code that ran before it did the require. And then it later breaks because the code execution sequence changes somewhere and you can't figure out why code that used to work just fine no longer works.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Javier Guerra Giraldez
In reply to this post by Sean Conner
On Tue, Oct 18, 2011 at 3:12 AM, Sean Conner <[hidden email]> wrote:
>  A question:  Two modules use Penlight, foo and bar.  If I include foo and
> bar into a project, does that mean I have two instances of Penlight loaded?

no, as long as both use require()

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Fabio Mascarenhas
In reply to this post by David Manura
On Tue, Oct 18, 2011 at 5:35 AM, David Manura <[hidden email]> wrote:
>
> Now, if one wanted to "fix" the module function, I think a solution
> roughly along the following lines might be acceptable to all parties.
> Modules could be written like this:
>

<snip foo.lua and loadmodule.lua>

Why not run with this idea and make something like your loadmodule.lua
the standard loader for Lua modules? See below:

------- loader.lua -----
-- this could be just the base lib!
local _G = package.loaded._G

-- an API function can make this pretty fast
local function clone(t)
  local nt = {}
  for k, v in pairs(t) do
    nt[k] = v
  end
  return nt
end

local function loader(name, modpath)
  local env = setmetatable({}, { __index = _G })
  env.module = function (name)
    env._NAME = name
    return env
  end
  local fh = assert(io.open(modpath, 'rb'))
  local source = fh:read'*a'
  fh:close()
  local ret = assert(load(source, modpath, 'bt', env))(name)
  return ret or clone(env)
end

-- replace Lua loder
package.searchers[2] = function (name)
  local modpath, msg = package.searchpath(name, package.path)
  if modpath then
    return loader, modpath
  else
    return nil, msg
  end
end

------- end loader.lua -----

A simple module:

------- foo.lua -----
module "foo"

local msg = "Hello World"

function bar()
  print(msg)
end

------- end foo.lua -----

------- main.lua -----

local foo = require "foo"
foo.bar()
print(foo.print)
print(foo._NAME)

------- end main.lua -----

$ ./lua -e "dofile('loader.lua')" main.lua
Hello World
nil
foo
$

This loader loads a module in a sandbox environment (it could just
provide the Lua base library, making the module use require even for
io, os, etc.), and a shallow copy of this environment becomes the
module. This allows modules to export symbols through globals, as
current modules that use "module", but they cannot pollute the global
environment by accident, and the module table does not keep references
to Lua library functions.

If the module returns a table then the loader assumes that this table
is the module and does not clone the environment, this lets Lua 5.1
modules that do not use "module" keep working. Removing this will
break more modules, but brings more consistency among modules. An
alternate loader could even register the module in the global
environment, as module currently does (good for simple scripts, and
the REPL).

All the flexibility of the searchers/loaders/require mechanism (the
mechanisms) will be still there for anyone embedding Lua, this just
sets tighter policy for the ecosystem of standalone Lua.

--
Fabio Mascarenhas

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Doug Currie
In reply to this post by David Manura

On Oct 18, 2011, at 1:53 AM, David Manura wrote:

> On Mon, Oct 17, 2011 at 1:00 PM, Sam Roberts <[hidden email]> wrote:
>> Now we are trying to support two ways, check out this code at about line 60:
>>  https://github.com/dcurrie/lunit/blob/master/lunit.lua
>
> That's ugly, though not necessary either.  I would, like Luiz, avoid
> using both _ENV and 'module' for module definition.
>
>> The occaisonal suggestion that lua programmers widely disliked
>> module() is pretty galling, and I'm tempted to deface that wiki page
>> people keep pointing to like it represents some kind of consensus
>> opinion, when its only the opinion of a few.
>
> The page (which is my fault ;) ) could use a little updating/more
> balance indeed, starting, I believe, with Hisham's comments.
>
>> I like module(), in particular, I like the behaviour that it sets up
>> globally reachable names for modules, and I like that I can call it
>> like module(...,package.seeall), and while containing no internal
>> reference to its actual name, it will build itself into the global
>> namespace depending on where its deployed in the filesystem.
>
> I understand other pro-module users would not agree on those points.
> Petite Abeille and Hisham said (or suggested) it's ok to get rid of
> luaL_pushmodule, and Hisham argued the benefits of explicitly writing
> out the name to module.
>
> Personally, I've thought that if you do want to design the module
> system to automatically set up global names, this responsibility
> belongs more inside a function loading the module (like require) than
> a function defining the module (like the module function).
>
>> The require() function defines a namespace, a namespace that is global
>> to the lua state, inherited from the filesystem, and in which
>> conflicts must be avoided
>
> Mark Hamburg had argued that the main problem is not conflicts but
> rather hidden dependencies:
>  http://lua-users.org/lists/lua-l/2006-04/msg00547.html
> Improving visibility, I believe, follows in the same vein as some of
> Hisham's recent comments concerning passing an explicit module name
> and other things.
>


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Doug Currie
In reply to this post by David Manura

On Oct 18, 2011, at 1:53 AM, David Manura wrote:

> On Mon, Oct 17, 2011 at 1:00 PM, Sam Roberts <[hidden email]> wrote:
>> Now we are trying to support two ways, check out this code at about line 60:
>>  https://github.com/dcurrie/lunit/blob/master/lunit.lua
>
> That's ugly, though not necessary either.  I would, like Luiz, avoid
> using both _ENV and 'module' for module definition.

Of course it's not necessary for new modules. I avoid both _ENV and 'module' for any new modules I create. The challenge is trying to maintain maximum compatibility with uses of existing modules, such as lunit, in large code bases, with both Lua 5.1 and 5.2.

e


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Josh Simmons
On Wed, Oct 19, 2011 at 6:48 AM, Doug Currie <[hidden email]> wrote:

>
> On Oct 18, 2011, at 1:53 AM, David Manura wrote:
>
>> On Mon, Oct 17, 2011 at 1:00 PM, Sam Roberts <[hidden email]> wrote:
>>> Now we are trying to support two ways, check out this code at about line 60:
>>>  https://github.com/dcurrie/lunit/blob/master/lunit.lua
>>
>> That's ugly, though not necessary either.  I would, like Luiz, avoid
>> using both _ENV and 'module' for module definition.
>
> Of course it's not necessary for new modules. I avoid both _ENV and 'module' for any new modules I create. The challenge is trying to maintain maximum compatibility with uses of existing modules, such as lunit, in large code bases, with both Lua 5.1 and 5.2.
>
> e
>
>
>

It's really not that hard though, replace the module call with a table
creation, do a find and replace of non-local definitions prefixing
with the table name and then return the table at the end. You now
support everything. Calling code just needs to change from require 'a'
to local a = require 'a'.

What's more you don't have to do it all at once either, a module at a
time works fine.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Hisham Muhammad
In reply to this post by David Manura
I'll try to condense some replies into one, in order to avoid
spreading out this thread much further.

On Mon, Oct 17, 2011 at 7:11 PM, Luiz Henrique de Figueiredo
<[hidden email]> wrote:

>> My opinion is that modules are important enough so that there should
>> be a "blessed" way of writing modules, and preferrably a somewhat
>> obvious one. There has to be a policy.
>
> The recommended way of writing modules is the trivial way: a module should
> return a table with its data and functions. It'd be nice if it also did
> not pollute the global environment. In Lua, this is done trivially with:
>
> local M={}
> M.version="1.0.2"
> function M.f() ... end
> ...
> return M
>
> This way works fine for both 5.1 and 5.2

In fact, that works fine for 5.0 as well. But if it was not enough for
encouraging reusable code back then (and module() was created in hope
to "remedy this situation", according to the HOPL paper quoted below),
then why would it be for 5.2?

On Tue, Oct 18, 2011 at 3:53 AM, David Manura <[hidden email]> wrote:

> On Mon, Oct 17, 2011 at 1:00 PM, Sam Roberts <[hidden email]> wrote:
>> I like module(), in particular, I like the behaviour that it sets up
>> globally reachable names for modules, and I like that I can call it
>> like module(...,package.seeall), and while containing no internal
>> reference to its actual name, it will build itself into the global
>> namespace depending on where its deployed in the filesystem.
>
> I understand other pro-module users would not agree on those points.
> Petite Abeille and Hisham said (or suggested) it's ok to get rid of
> luaL_pushmodule, and Hisham argued the benefits of explicitly writing
> out the name to module.
>
> Personally, I've thought that if you do want to design the module
> system to automatically set up global names, this responsibility
> belongs more inside a function loading the module (like require) than
> a function defining the module (like the module function).

Correct. I think it's good to stop auto-declaring globals (if Sam needs
a globally reachable name for a module (for inspection purposes or
something similar, there's package.loaded["modname"]) and I don't like
the module(...) idiom.

>> The require() function defines a namespace, a namespace that is global
>> to the lua state, inherited from the filesystem, and in which
>> conflicts must be avoided
>
> Mark Hamburg had argued that the main problem is not conflicts but
> rather hidden dependencies:
>  http://lua-users.org/lists/lua-l/2006-04/msg00547.html
> Improving visibility, I believe, follows in the same vein as some of
> Hisham's recent comments concerning passing an explicit module name
> and other things.

Yes.

On Tue, Oct 18, 2011 at 4:37 AM, Mark Hamburg <[hidden email]> wrote:

> Let's run through what module does.
>
> 1. It handles constructing module tables including the fields _NAME, _M,
> and _PACKAGE. This is useful if one cares about these fields, but I note that
> they don't appear in the pre-defined modules like table and string.
>
> 2. It handles constructing submodules. I might question whether it makes
> sense to have socket.http available when socket is potentially just a stub
> table, but that's a more complicated matter.
>
> 3. It provides early registration for the module. This helps with mutually
> recursive requires, but it also means that require can return an incomplete
> module.
>
> 4. It adds the module to the global namespace. This, in my opinion, is a
> bad thing because it creates hidden dependencies -- i.e., code can use a
> module that it never required simply because some other code required
> it earlier and it became available in the global namespace.
>
> 5. It mucks with the environment to make it "easy" to export functions.
> But then to compensate for this, it offers package.seeall which results in
> a module that reveals a lot of globals in its table which have nothing to
> do with the module -- i.e., it pollutes the API for the module.
>
> I think it would be useful to hear which of these behaviors is deemed so
> valuable that people are fighting for it.

I'm indifferent to #1, I think #2 is nice for consistency but not
critical, never been bitten by any problems in #3, and agree that #4
is a bad thing.

I believe the _intention_ behind #5 is why module() was created: a
function so that you can declare a module and then what you code in
that file is the module. This is analogous to what we see in other
languages.

For that reason, I think #5 produces the clearest programming idiom.
Distinguishing private and public functions with "local function" and
"function" is very logical, if you think of "module" abstractly, as a
language feature.

The way it is implemented, with package.seeall, etc, is unfortunate.
Maybe that's what Mike Pall means when he says that module "has not
held up to its promise". But the promise was good: module() was
created with the goal of filling and existing need, and that's why
even with its flaws, it was a success.

To add to David Manura's statistics, here is a list of packages in the
LuaRocks repository that include Lua code (I unpacked all sources,
grepped and inspected the results by hand). The vast majority uses
module() (and are not Kepler modules, to get that out of the way).

abelhas: YES
alien: YES
alt-getopt: YES
bencode: YES
cgilua: YES
colors: YES
concurrentlua: YES
config: YES
copas: YES
cosmo: YES
cue: YES
dado: YES
diff: YES
dkjason: no
fbclient: YES
flu: no
htk: YES
json4lua: YES
lanes: YES
leg: YES
loop: YES
lpeg: no (YES, up to 0.9)
lrexlib-pcre: YES
lsqlite3: no
ltcltk: YES
lua-aplicado: no
lua-apr: no
luabitop: no
lua-coat: YES
lua-codegen: YES
luacov: YES
lua-discout: YES
luadoc: YES
lua-espeak: YES
lua-ev: YES
luaexpat: YES
luafft: no (used to)
lua-geoip: no
luagraph: YES
luahaml: YES
luahtml: YES
luaidl: YES
lua-imlub2: YES
luajson: YES
lua-llthreads: no
lualogging: YES
luamacro: no
luanotify: YES
lua-nucleo: no
luapod: YES
luaposix: YES
luapsql: YES
luasec: YES
luasoap: YES
luasocket: YES
luasofia: YES
luasolidstate: YES
lua-testlongstring: YES
lua-testmore: YES
luatexts: no
lua-tinycdb: YES
luaxml: YES
lua-xmlreader: no
lua-yajl: YES
lua-zmq: YES
lua-zmq-threads: YES
luchia: YES
luma: YES
lunary-optim: YES
lunatest: YES
lunit: YES
luse: YES
lxsh: no
markdown: no
math-evol: no
math-rungekutta: no
math-walshtransform: no
md5: YES
midi: no
midialsa: no
nixio: YES
oauth: YES
objectlua: YES
oil: YES
orbit: YES
penlight: YES
petrodoc: no
recaptcha: YES
redis-lua: YES
remdebug: YES
rings: YES
sha2: YES
shake: YES
simulua: YES
sociallua: YES
sputnik: YES
stdlib: YES
tamale: YES
tango-complete: YES
tango-copas: YES
telescope: YES
tethys: YES
tlua: YES
twitter: YES
validate-args: YES
vararg: YES
wsapi: YES
wsapi-fcgi: YES
xavante: YES
xssfilter: YES

In terms of the package.seeall issue, what I would consider
"intuitive" (for my intuition at least) would be to have only globals
declared by the module accessible through it, but have available in
its environment the same base library one sees at the beginning of a
Lua program.

> Oh. And on the documentation front, the "module( ... )" idiom obviously
> fails to document the module name and comments are another good
> way to write documentation.

Yes (and I dislike "module(...)"), but self-documented code is better.
Languages have high-level constructs for that reason, or else we'd
only use goto and instead of structured programming we'd just comment
our loops. :)

>From what I've seen of the non-module() users, some create modules
declaring a table called "Modname", some "M", some "_M" (wasn't there
some recommendation against using names with underscores followed by
capitals?). Some declaring those tables in the end, some on top, some
as a big table declaration spanning the whole file... some pollute the
global namespace, some don't. I've seen some polluting the global
namespace with names different from that of the module. Was it
require("luagl") that created a global table GL? I don't remember
precisely, but I've seen things like that. Without a module() function
that enforces some proper behaviors it's likely we'll start to see
things like that more and more. If we don't like the current behavior
of module(), let's fix it, but please don't throw out the baby with
the bath water.

Discussing this offlist with Fabio he also mentioned another benefit
of module()-style modules: that one can easily inspect the code
mechanically for global declarations and infer what's exported, while
module()-less-modules make this kind of static analysis a lot more
fragile.

Petite Abeille wrote:

> Paradoxical, the most articulated proponent of module is Roberto
> himself, before he had a change of heart and, sadly,  turned to the
> dark side:
>
> "Despite our “mechanisms, not policy” rule — which we have found
> valuable in guiding the evolution of Lua — we should have provided
> a precise set of policies for modules and packages earlier. The lack
> of a common policy for building modules and installing packages
> prevents different groups from sharing code and discourages the
> development of a community code base. Lua 5.1 provides a set of
> policies for modules and packages that we hope will remedy this
> situation." [1]
>
> "Usually, Lua does not set policies. Instead, Lua provides mechanisms
> that are powerful enough for groups of developers to implement the
> policies that best suit them. However, this approach does not work
> well for modules. One of the main goals of a module system is to allow
> different groups to share code. The lack of a common policy impedes
> this sharing." [2]
>
> The young and idealistic Roberto will be fondly  remembered by the many mourning little modules he is leaving
> behind.

PA's remarks are often a bit too dry for my taste, but this one made
me smile. :)

But apart from the humor, indeed, the first quote is right on the
mark. If "we should have provided a precise set of policies for
modules and packages earlier", then getting rid of module() altogether
is a step backwards.

As you can see from the modules list above, the community code base is
still in its infancy compared to repositories from other languages.
The Lua community is still trying to catch up with the lack of module
policies that held it back in the past. A loose recommendation to
create modules such-and-such way in Lua 5.0 days was clearly not
enough, hence the creation of module(), and once a more precise policy
promoted by a standard library function was offered, it was widely
adopted. If that function has policy problems in its implementation,
please fix the function instead of just getting rid of it and
returning things back to their previous state, which "prevents
different groups from sharing code and discourages the development of
a community code base".

To sum it all up: I agree with the concerns voiced against module() by
the anti-module camp. At least some of the pro's I've mentioned (or at
least the motivation to have a module declaration function) were
agreed with by the anti-module proponents. I'd like us to try to meet
halfway, and perhaps to show the Lua authors that the intentions they
had when they wrote those paragraphs above were indeed valid.

In terms of "fixing module()", I think Fabio's implementation is right
on the money:

http://article.gmane.org/gmane.comp.lang.lua.general/84283

It shows that a sane implementation of module() can be done with
standard Lua mechanisms. Do any of the anti-module proponents see any
problems with that design? Would you hate module() as much as you do
now if it behaved like that instead of the way it behaves in Lua 5.1?

-- Hisham
http://hisham.hm/ - http://luarocks.org/

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Geoff Leyland
On 19/10/2011, at 6:25 PM, Hisham wrote:

> In terms of "fixing module()", I think Fabio's implementation is right
> on the money:
>
> http://article.gmane.org/gmane.comp.lang.lua.general/84283
>
> It shows that a sane implementation of module() can be done with
> standard Lua mechanisms. Do any of the anti-module proponents see any
> problems with that design?

Would taking a shallow copy of the module to hide _G would make

-- mod.lua
module("mod")
count = 1
function increment() count = count + 1; print(count) end


-- main.lua
local mod = require("mod")
print(mod.count)
mod.increment()
print(mod.count)


print "1 2 1" rather than "1 2 2"?

Cheers,
Geoff

123456 ... 8