modules, require, magic

classic Classic list List threaded Threaded
160 messages Options
12345 ... 8
Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille

On Oct 17, 2011, at 11:50 PM, Thijs Schreijer wrote:

> And then stop this. (please)

Well, apologies from waking you up from bellow your rock :)


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Josh Simmons
In reply to this post by Pierre Chapuis
On Tue, Oct 18, 2011 at 8:39 AM, Pierre Chapuis <[hidden email]> wrote:

> On Mon, 17 Oct 2011 19:11:38 -0200, Luiz Henrique de Figueiredo wrote:
>
>> local M={}
>> M.version="1.0.2"
>> function M.f() ... end
>> ...
>> return M
>
>> There is no need for a module function and no need for syntax either.
>> No one needs to know about _ENV for that only for complicated things
>> like avoiding writing "M." (probably not a good idea as it serves as
>> documentation of what is exported).
>
> I use an alternative construction that does the same thing in the end:
>
> local f = function() ... end
> return {
>  version = "1.0.2",
>  f = f,
> }
>
> I also think modules shouldn't modify the global namespace.
> This solution doesn't allow concatenation of module files but this is
> a small price to pay for a simple, understandable module mechanism.
>
> --
> Pierre Chapuis
>
>

module...
Localise external functions so they remain accessible after the call
to module or use seeall to make _G the __index of the new
environment's metatable, call module creating a global table, making
it the environment of the current block so following global sets and
gets happen to the module table not the global environment.

not module...
return a table with exports.

I just don't understand _why_ you want the functional equivalent of
module 'foo' when all you ever need is trivial and much more obvious
without it.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille
In reply to this post by Pierre Chapuis

On Oct 17, 2011, at 11:39 PM, Pierre Chapuis wrote:

> This solution doesn't allow concatenation of module files

Nor naming of module, as now you rely on the require mechanism to do so. Nor environment isolation, as you are always working in the global one. Etc, ...

> but this is
> a small price to pay for a simple, understandable module mechanism.

Sure. If that works for you, then more power to you. But why would you want to take away module from everybody else, when you are not even using it yourself? Puzzling.


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

joao lobato
In reply to this post by Josh Simmons
On 10/17/11, Josh Simmons <[hidden email]> wrote:

> module...
> Localise external functions so they remain accessible after the call
> to module or use seeall to make _G the __index of the new
> environment's metatable, call module creating a global table, making
> it the environment of the current block so following global sets and
> gets happen to the module table not the global environment.
>
> not module...
> return a table with exports.
>
> I just don't understand _why_ you want the functional equivalent of
> module 'foo' when all you ever need is trivial and much more obvious
> without it.
>
>

Indeed. The Table(tm) helps here as well :-)

Actually, what I don't understand is dofile; I always find it wierd
that it can't be passed arguments...

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

troy wreford
In reply to this post by Petite Abeille

If you want a language that only adds features for everyone's tastes and never removes features, maybe Lua is not the answer for you.

On 2011-10-17 6:58 PM, "Petite Abeille" <[hidden email]> wrote:


On Oct 17, 2011, at 11:39 PM, Pierre Chapuis wrote:

> This solution doesn't allow concatenation of...

Nor naming of module, as now you rely on the require mechanism to do so. Nor environment isolation, as you are always working in the global one. Etc, ...


> but this is
> a small price to pay for a simple, understandable module mechanism.

Sure. If that works for you, then more power to you. But why would you want to take away module from everybody else, when you are not even using it yourself? Puzzling.


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille
In reply to this post by Josh Simmons

On Oct 18, 2011, at 12:37 AM, Josh Simmons wrote:

> I just don't understand _why_ you want the functional equivalent of
> module 'foo'

Lack of imagination? :))

> when all you ever need is trivial

Indeed the way 5.2 is heading. Trivialization.

Ironically enough, a while back, Mike Pall labelled 5.2 as the 'Vista' of Lua releases [1]. At the time, I though it was a bit cruel. But perhaps it was insightful instead.

Oh, well...

[1] http://article.gmane.org/gmane.comp.lang.lua.general/70563/

 
Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille
In reply to this post by troy wreford

On Oct 18, 2011, at 1:00 AM, troy wreford wrote:

> If you want a language that only adds features for everyone's tastes and
> never removes features, maybe Lua is not the answer for you.

Nah... lets not blow this out of proportion... Lua is a fine and cheerful language...

Still, some changes are smarter than others. And deprecating module is not in the smart move category.

But so it goes.

As Osgood Fielding III once famously said:

"Nobody's perfect!"
Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Luiz Henrique de Figueiredo
In reply to this post by joao lobato
> Actually, what I don't understand is dofile; I always find it wierd
> that it can't be passed arguments...

For that, use loadfile:
        assert(loadfile(x))(1,2,3,4)

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

troy wreford
In reply to this post by Petite Abeille

For the record, I agree that a standard module function majesty sense.

On 2011-10-17 7:34 PM, "Petite Abeille" <[hidden email]> wrote:


On Oct 18, 2011, at 1:00 AM, troy wreford wrote:

> If you want a language that only adds features ...

Nah... lets not blow this out of proportion... Lua is a fine and cheerful language...

Still, some changes are smarter than others. And deprecating module is not in the smart move category.

But so it goes.

As Osgood Fielding III once famously said:

"Nobody's perfect!"

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

troy wreford

Majesty = makes. Autocomplete...

On 2011-10-17 7:36 PM, "troy wreford" <[hidden email]> wrote:

For the record, I agree that a standard module function majesty sense.


>
> On 2011-10-17 7:34 PM, "Petite Abeille" <[hidden email]> wrote:
>

>
> On Oct 18, 2011, at 1:00 AM, troy wreford wrote:
>

> If you want a language that only adds features ...


>
> Nah... lets not blow this out of proportion... Lua is a fine and cheerful language...
>
> Still...

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

joao lobato
In reply to this post by Luiz Henrique de Figueiredo
On 10/18/11, Luiz Henrique de Figueiredo <[hidden email]> wrote:
>> Actually, what I don't understand is dofile; I always find it wierd
>> that it can't be passed arguments...
>
> For that, use loadfile:
> assert(loadfile(x))(1,2,3,4)
>
>

Yes, that's exactly why I think dofile feels like... well, cruft (of
course I can understand that its absence would break stuff).

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Paul E. Merrell, J.D.
In reply to this post by joao lobato
On Mon, Oct 17, 2011 at 3:59 PM, joao lobato <[hidden email]> wrote:
> On 10/17/11, Josh Simmons <[hidden email]> wrote:

> Actually, what I don't understand is dofile; I always find it wierd
> that it can't be passed arguments...

Might you enlighten me with a use case that can't be worked around via
dofile's ability to pass global variables?

(Not trying to be sarcastic; I'm close to being a novice scripter and
sincerely do not see a use case.)

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

David Manura
In reply to this post by Sam Roberts
On Mon, Oct 17, 2011 at 1:00 PM, Sam Roberts <[hidden email]> wrote:
> Now we are trying to support two ways, check out this code at about line 60:
>   https://github.com/dcurrie/lunit/blob/master/lunit.lua

That's ugly, though not necessary either.  I would, like Luiz, avoid
using both _ENV and 'module' for module definition.

> The occaisonal suggestion that lua programmers widely disliked
> module() is pretty galling, and I'm tempted to deface that wiki page
> people keep pointing to like it represents some kind of consensus
> opinion, when its only the opinion of a few.

The page (which is my fault ;) ) could use a little updating/more
balance indeed, starting, I believe, with Hisham's comments.

> I like module(), in particular, I like the behaviour that it sets up
> globally reachable names for modules, and I like that I can call it
> like module(...,package.seeall), and while containing no internal
> reference to its actual name, it will build itself into the global
> namespace depending on where its deployed in the filesystem.

I understand other pro-module users would not agree on those points.
Petite Abeille and Hisham said (or suggested) it's ok to get rid of
luaL_pushmodule, and Hisham argued the benefits of explicitly writing
out the name to module.

Personally, I've thought that if you do want to design the module
system to automatically set up global names, this responsibility
belongs more inside a function loading the module (like require) than
a function defining the module (like the module function).

> The require() function defines a namespace, a namespace that is global
> to the lua state, inherited from the filesystem, and in which
> conflicts must be avoided

Mark Hamburg had argued that the main problem is not conflicts but
rather hidden dependencies:
  http://lua-users.org/lists/lua-l/2006-04/msg00547.html
Improving visibility, I believe, follows in the same vein as some of
Hisham's recent comments concerning passing an explicit module name
and other things.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Dirk Laurie-2
In reply to this post by Petite Abeille
2011/10/18 Petite Abeille <[hidden email]>:

>
> On Oct 18, 2011, at 1:00 AM, troy wreford wrote:
>
>> If you want a language that only adds features for everyone's tastes and
>> never removes features, maybe Lua is not the answer for you.
>
> Nah... lets not blow this out of proportion... Lua is a fine and cheerful language...
>
> Still, some changes are smarter than others. And deprecating module is not in the smart move category.
>

One thing is obvious, and that is: some highly vocal people are
pro-module, other equally vocal people are anti-module.

Another thing should also be obvious, but those very vocal people seem
to regard it as unimportant: if three people agree on whether a
specific change to Lua is smart, their collective opinion matters if
and only if their names happen to be Luiz, Roberto and Waldemar.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Mark Hamburg
In reply to this post by David Manura
Let's run through what module does.

1. It handles constructing module tables including the fields _NAME, _M, and _PACKAGE. This is useful if one cares about these fields, but I note that they don't appear in the pre-defined modules like table and string.

2. It handles constructing submodules. I might question whether it makes sense to have socket.http available when socket is potentially just a stub table, but that's a more complicated matter.

3. It provides early registration for the module. This helps with mutually recursive requires, but it also means that require can return an incomplete module.

4. It adds the module to the global namespace. This, in my opinion, is a bad thing because it creates hidden dependencies -- i.e., code can use a module that it never required simply because some other code required it earlier and it became available in the global namespace.

5. It mucks with the environment to make it "easy" to export functions. But then to compensate for this, it offers package.seeall which results in a module that reveals a lot of globals in its table which have nothing to do with the module -- i.e., it pollutes the API for the module.

So, my evaluation on these is:

#1 is interesting but probably not all that useful.

#2 is potentially pretty useful if one thinks submodules are a useful construct. If we keep modules out of globals, however, I'm not sure the construction really matters. If I have to say "require 'socket.http'" why then does it matter that having said this I can also say "require 'socket'.http"?

#3 is useful at times but also dangerous. It might be better handled through a package.register function.

#4 is a bad thing.

#5 is a bad thing if you use package.seeall. It's a dubious savings if you don't use package.seeall.

Obviously others differ, but I think it would be useful to hear which of these behaviors is deemed so valuable that people are fighting for it.

Oh. And on the documentation front, the "module( ... )" idiom obviously fails to document the module name and comments are another good way to write documentation.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Sean Conner
It was thus said that the Great Mark Hamburg once stated:

> Let's run through what module does.
>
> 4. It adds the module to the global namespace. This, in my opinion, is a
> bad thing because it creates hidden dependencies -- i.e., code can use a
> module that it never required simply because some other code required it
> earlier and it became available in the global namespace.
>
> So, my evaluation on these is:
>
> #4 is a bad thing.
>
> Obviously others differ, but I think it would be useful to hear which of
> these behaviors is deemed so valuable that people are fighting for it.

  I'm still thinking on the other items, but for this one, I'm not sure if I
like module() adding to the global name space or not.  I can see why it one
would like it *not* to add to the global namespace, but on the other hand, I
can see a reason *for* adding a module to the global namespace.

  For the modules I've written, since they do fall into the global
namespace, is that I carved out a global namespace for myself.  All my
modules I've written are placed under the "org.conman" table (taking an idea
from the Java world) in order to prevent clashes.  So code that's expecting
DarkGod's DNS Lua module won't blow up with my DNS Lua module.  

  But a reason for a global module is as follows:  I have a Unix module
(org.conman.unix) that stores a few tables that list the users, groups and
programs on a Unix system.  For example:

[spc]lucy:~>lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> show = require("org.conman.table").show
> unix = require("org.conman.unix")
> show(unix)
paths         table: 0x874ae20                                            
_PACKAGE      "org.conman."                                                
_DESCRIPTION  "The list of users, groups and programs on this Unix system"
_NAME         "org.conman.unix"                                            
_COPYRIGHT    "Copyright 2010 by Sean Conner.  All Rights Reserved."      
groups        table: 0x8755ce0                                            
_M            table: 0x873d678                                            
users         table: 0x873d7a8                                            
_VERSION      "1.0"                                                        
> show(unix.users['spc'])
uid     500          
name    "Sean Conner"
gid     500          
userid  "spc"        
home    "/home/spc"  
shell   "/bin/bash"  
> print(unix.paths['firefox'])
/home/spc/bin/firefox
>

  Assume for a moment there are several other modules that require this
module.  Without a global namespace, each module will get their own copy of
this module, which would mean wasted space as there are multiple copies of
this data (and that paths table could be pretty big ... [1])

  "But module() and require() could keep a private table with references to
each module and not dump them into the global space," you say.  Okay, but
you *still* need a global namespace, because DarkGod has *his* DNS module
(named dns) and I have *my* DNS module (which would also be named "dns"), so
which one is returned when you require("dns")?  (assuming that I have
modules I wrote using my DNS module, and some other modules I want to use
might be using DarkGod's DNS module)

  Perhaps a question is in order:  has anyone experienced a module name
clash?

  -spc (I have, quite often.  Often enough that I'm using org.conman for
        my namespace ... )

[1] It's empty initially, until you try to index it; at that point it
        will look through $PATH for the executable, save the results to that
        table, and return it.  Next time it's indexed for the same name,
        it'll return the cached result.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Pierre Chapuis
In reply to this post by Petite Abeille
On Tue, 18 Oct 2011 00:38:52 +0200, Petite Abeille wrote:

> Sure. If that works for you, then more power to you. But why would
> you want to take away module from everybody else, when you are not
> even using it yourself? Puzzling.

Actually, I don't really care if module() stays in Lua or not.

What I care about is this:

     > mymodule = require "somemodule"
     > =somemodule
     nil

As long as module is changed to behave like that I'm happy if it
remains in the language!

--
Pierre Chapuis

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Josh Simmons
In reply to this post by Sean Conner
On Tue, Oct 18, 2011 at 6:20 PM, Sean Conner <[hidden email]> wrote:
>
>  "But module() and require() could keep a private table with references to
> each module and not dump them into the global space," you say.  Okay, but
> you *still* need a global namespace, because DarkGod has *his* DNS module
> (named dns) and I have *my* DNS module (which would also be named "dns"), so
> which one is returned when you require("dns")?  (assuming that I have
> modules I wrote using my DNS module, and some other modules I want to use
> might be using DarkGod's DNS module)
>

Require already does this, and you need to separate them at the
filesystem level irregardless of what scope they're imported into.

local dns = require 'crazy.weird.dns'
local dns2 = require 'somebody.elses.dns'

None of this has anything to do with global scope or module.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Josh Simmons
On Tue, Oct 18, 2011 at 6:28 PM, Josh Simmons <[hidden email]> wrote:

> On Tue, Oct 18, 2011 at 6:20 PM, Sean Conner <[hidden email]> wrote:
>>
>>  "But module() and require() could keep a private table with references to
>> each module and not dump them into the global space," you say.  Okay, but
>> you *still* need a global namespace, because DarkGod has *his* DNS module
>> (named dns) and I have *my* DNS module (which would also be named "dns"), so
>> which one is returned when you require("dns")?  (assuming that I have
>> modules I wrote using my DNS module, and some other modules I want to use
>> might be using DarkGod's DNS module)
>>
>
> Require already does this, and you need to separate them at the
> filesystem level irregardless of what scope they're imported into.
>
> local dns = require 'crazy.weird.dns'
> local dns2 = require 'somebody.elses.dns'
>
> None of this has anything to do with global scope or module.
>

Interestingly enough this is actually worse if you use module and they
both name themselves 'dns' in which case the value of the global just
depends on who got in last.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

David Manura
In reply to this post by Hisham Muhammad
Those are some good points:

On Mon, Oct 17, 2011 at 4:44 PM, Hisham <[hidden email]> wrote:
> [In Lua 5.0] require() didn't enforce the
> return of the module, so some packages would just add to the global
> namespace, some would return a table, etc.

Perl, for all its ability for "keeping easy things easy" has a
slightly annoying requirement that a module must explicitly return a
true value at the end of the file. [1]  However, if you omit the
return, then the function loading the module simply fails.  So, there
is not a risk that some users will omit the return value.  The lesson
I think is that the return can be enforced.

[1] http://stackoverflow.com/questions/5293246/why-the-1-at-the-end-of-each-perl-package

> The module() function gave us an easy answer for the question "how do
> I write a module" that mirror other languages that offer constructs for modularity.
> [...] If module() is broken beyond repair and there is no clean way to add a
> library function to do that, I'd dare to say that this is something
> important enough that a language construct would be justifiable

OTOH, there are multiple ways to use this function: The somewhat
canonical "easy way" involves package.seeall, and maybe we should all
just accept that, in the way in another life I just accept Perl 5.
Josh's principle I think is important though: "[it's about] making it
easier to do the right thing. With module it's easy to get things
wrong".

> But it's really nice to be able to open a source file and read something like " module('foo.bar.baz') "
> on top, which gives a clear indication of how this module should be loaded.

Yes.  Most recently, in packaging a certain project, it wasn't clear
to me where the modules should be installed or how they should be
loaded with require (though, thankfully, the author addressed these
concerns).  The "module(...)" trick doesn't help here, which perhaps
should be discouraged, or (as Mark just noted) at least accompanied
with good comments.

> requiring users to learn about _ENV in order to write a reusable
> module is, in my opinion, worse.

True, I don't plan to use _ENV for module definition in 5.2.

> that it won't pollute the global namespace,
> that you can assign the return of require() to a local.
> It's important to have
> those guarantees when writing code with reusable components.

IMO, it's a question of timing.  If the module is popular enough,
someone will eventually complain to the author to fix that.  Even
earlier, someone could complain to the author when the module was
submitted to LuaRocks.  Even earlier, Lua itself could complain (at
runtime or compile time).

~~~

Now, if one wanted to "fix" the module function, I think a solution
roughly along the following lines might be acceptable to all parties.
Modules could be written like this:

  -- foo.lua, to be loaded with foo = require 'loadmodule' 'foo'
  module "foo"
  local baz = require 'loadmodule' 'baz'
  function M.bar()
    return math.sqrt(16) + baz.qux()
  end

and the pluming might be defined more-or-less like this:

  -- loadmodule.lua
  return function(name)
    local M = {}
    local called
    local function module(name)
      M._NAME = name
      assert(not called, 'module function used twice')
      called = true
      return M
    end
    local env = setmetatable({M=M, module=module}, {__index = _G})
    local modpath = package.searchpath(name, package.path)
    local fh = io.open(modpath, 'rb')
    if not fh then error('could not open file', 2) end
    local source = fh:read'*a'
    fh:close()
    assert(load(source, modpath, 'bt', env))()
    if not called then
      error('"module" call expected in module.  Is this really a module?', 2)
    end
    return M
  end

12345 ... 8