modules, require, magic

classic Classic list List threaded Threaded
160 messages Options
1234 ... 8
Reply | Threaded
Open this post in threaded view
|

modules, require, magic

Eduardo Ochs
I think that this deserves a new thread.
This message is an answer to:


So:  in Lua 5.1 one can do this,

  module("mod1")
  ...
  module("mod2")
  ...

while in Lua 5.2 one has to do this,

  do local _ENV = mymodule("mod1")
     ...
  end
  do local _ENV = mymodule("mod2")
     ...
  end

where "mymodule" is a more or less trivial function, that can be
implemented in pure Lua.

In my opinion the two worst functions in Lua 5.1 are "require" and
"module". They contain too much magic, and - in my opinion again -
their implementation details are somewhat arbitrary. My suggestion
about the best way to cope with that is certainly going to be totally
different from Petite Abeille's... here it is: add to the "etc/"
directory of Lua 5.2 implementations of "require" and "mymodule" in
pure Lua, blessed by the Lua team, and equivalent to their
implementations in C. People who don't understand their behavior very
well ("idiots like me") will then be able to clarify things by just
adding "print"s and running tests.

By the way, how hard it would be implement something like the "module"
in the top chunk above in Lua 5.2? What are the difficulties? Would it
need to call the debug library? Would it need to do something that can
only be done from C? This is a honest, though maybe a dumb, question...
I am not a programmer - I am just a mathematician / logician /
category theorist - and I DID try to understand modules at one point,
but things didn't make enough sense to me, so I gave up and lived
happily ever after... 8-(

  Cheers,
    Eduardo Ochs

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Patrick Mc(avery-2
PIL is pretty easy to read and dofile, require and module all made sense
after I read about them but this was short lived.

The relentless debate on these topics has completely confused me, so
much so that I avoid all of these.

This problem, sounds complex but perhaps it could be chipped apart. One
simple function that would help people like me would be a sort of
"copy_in()" function whereby the contents enclosed would be would be
parsed as if they had been typed where the copy_in function was called
from, do - end blocks around it could be used also be used to control scope.

It's my understanding that this is not how dofile works, dofile has
scope side affects.

copy_in would be very simple to understand and could reduce the need for
require and could reduce these discussions a bit too-Patrick



Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille
In reply to this post by Eduardo Ochs

On Oct 16, 2011, at 11:39 PM, Eduardo Ochs wrote:

>  They contain too much magic

<interlude>

Hmmm... regarding that "magic" argument... this sounds like hogwash to me... who understands the intimate details of Lua implementation? Of how LuaJIT does its very own magic? Of how a relational database is implemented? Of how a phone work? I, for one, don't.

For all practical matter, my phone could be powered by  Hogwarts School of Witchcraft and Wizardry for all I know.

This doesn't prevent me, nor you, from making a phone call. Nor enjoying the benefit of all this "magic" surrounding us without a second thought.

So get a grip :)

</interlude>










Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Matthew Wild
On 17 October 2011 00:36, Petite Abeille <[hidden email]> wrote:
>
> On Oct 16, 2011, at 11:39 PM, Eduardo Ochs wrote:
>
>>  They contain too much magic
>
> <interlude>
>
> Hmmm... regarding that "magic" argument... this sounds like hogwash to me... who understands the intimate details of Lua implementation? Of how LuaJIT does its very own magic? Of how a relational database is implemented? Of how a phone work? I, for one, don't.
>

The implementation isn't a problem, it's where the implementation
interfaces with the user that counts. module() has too many hidden
caveats that are waiting to trip up users:
http://lua-users.org/wiki/LuaModuleFunctionCritiqued

Regards,
Matthew

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Roberto Ierusalimschy
In reply to this post by Eduardo Ochs
> By the way, how hard it would be implement something like the "module"
> in the top chunk above in Lua 5.2? What are the difficulties? Would it
> need to call the debug library? Would it need to do something that can
> only be done from C? This is a honest, though maybe a dumb, question...

It is quite easy to write a 'module' function in 5.2, using the debug
library. (But it will not allow multiple modules in a single file,
which is a kind of hack anyway ;)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille
In reply to this post by Matthew Wild

On Oct 17, 2011, at 1:42 AM, Matthew Wild wrote:

>  too many hidden caveats that are waiting to trip up users:

Who are these mythical users who need your active protection from tripping themselves with the treacherous  module? "People" cannot handle the intricacies of ipairs at first. Should we kill ipairs, as attempted in one of the 5.2 releases? What about _ENV? That sounds straightforward too. Ah, yes, goto will be a walk in the park as well.

So, please, lets stop infantilizing these mythical "users". Like with everything else, if you don't understand it, don't use it until you do.


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Matthew Wild
On 17 October 2011 00:58, Petite Abeille <[hidden email]> wrote:
>
> On Oct 17, 2011, at 1:42 AM, Matthew Wild wrote:
>
>>  too many hidden caveats that are waiting to trip up users:
>
> Who are these mythical users who need your active protection from tripping themselves with the treacherous  module?

The mythical users are the ones I regularly mythically help overcome
problems like 'attempt to index global 'table' (a nil value)', who I
then have to describe the hack known as package.seeall, or manually
importing everything they use into locals.

> So, please, lets stop infantilizing these mythical "users". Like with everything else, if you don't understand it, don't use it until you do.

That's an interesting approach... it means most people would never use
module(). I'd bet most people following this thread who have used it
don't even understand what it does fully.

I do understand it, which is why I don't like it.

Regards,
Matthew

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Josh Simmons
In reply to this post by Petite Abeille
On Mon, Oct 17, 2011 at 10:58 AM, Petite Abeille
<[hidden email]> wrote:

>
> On Oct 17, 2011, at 1:42 AM, Matthew Wild wrote:
>
>>  too many hidden caveats that are waiting to trip up users:
>
> Who are these mythical users who need your active protection from tripping themselves with the treacherous  module? "People" cannot handle the intricacies of ipairs at first. Should we kill ipairs, as attempted in one of the 5.2 releases? What about _ENV? That sounds straightforward too. Ah, yes, goto will be a walk in the park as well.
>
> So, please, lets stop infantilizing these mythical "users". Like with everything else, if you don't understand it, don't use it until you do.
>
>
>

It's not so much about saving mythical users, but about making it
easier to do the right thing. With module it's easy to get things
wrong by even a seasoned campaigner as the deeper semantics are
non-trivial and not obvious. When replaced by the arduous few lines
required in 5.2 the semantics are clear to anybody with a basic grasp
of Lua constructs.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille
In reply to this post by Matthew Wild

On Oct 17, 2011, at 2:19 AM, Matthew Wild wrote:

> The mythical users are the ones I regularly mythically help overcome
> problems like 'attempt to index global 'table' (a nil value)', who I
> then have to describe the hack known as package.seeall, or manually
> importing everything they use into locals.

I heard good things about the fine Lua manual. Concise, but to the point.

> That's an interesting approach... it means most people would never use
> module().

Mute point. I would posit that most "people" only use what they need. Not everything which is potentially available.

> I'd bet most people following this thread who have used it
> don't even understand what it does fully.

Back to "magic". I'm sure that hasn't prevented you from, say, driving. Even if, arguably,  you have most likely no ideas about how it all fits together :)

> I do understand it, which is why I don't like it.

Good. You don't have to. Feel free to reinvent your very own square wheel. As we will all have to do from now on.

In any case, lets simply agree to disagree as this is going strictly nowhere as usual.

Lua has decided to deprecate module in 5.2. 5.2 is not even released yet. Deprecated means its going to hang around until at least 5.3. At the current velocity, this is about 3-4 years after 5.2 is released.

So, no stress. Boneheaded decision nonetheless :))

 


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille
In reply to this post by Josh Simmons

On Oct 17, 2011, at 2:33 AM, Josh Simmons wrote:

>  When replaced by the arduous few lines
> required in 5.2 the semantics are clear to anybody with a basic grasp
> of Lua constructs.

Right. Looking forward to the avalanche of very pointed questions about what the (*&^(%*$%$# that magical _ENV thingy is. Remember, ipairs is apparently over the top already.


Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Josh Simmons
On Mon, Oct 17, 2011 at 11:43 AM, Petite Abeille
<[hidden email]> wrote:

>
> On Oct 17, 2011, at 2:33 AM, Josh Simmons wrote:
>
>>  When replaced by the arduous few lines
>> required in 5.2 the semantics are clear to anybody with a basic grasp
>> of Lua constructs.
>
> Right. Looking forward to the avalanche of very pointed questions about what the (*&^(%*$%$# that magical _ENV thingy is. Remember, ipairs is apparently over the top already.
>
>
>

Not sure why you think the problem with ipairs was magic, it was just
deemed (for a little while at least) unnecessary.

_ENV is a core construct of Lua, something you're going to have to
understand eventually anyway, it's not a restrictive single purpose
function replaceable with ~3 lines of trivial Lua in your module file
where the latter also provides more useful and obvious semantics.

I'd also note that it's not necessary or beneficial to touch _ENV at
all in your module unless you're doing something weird.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille

On Oct 17, 2011, at 3:07 AM, Josh Simmons wrote:

> Not sure why you think the problem with ipairs was magic,

Because, as we all know, ipairs doesn't know how to count.

> it was just
> deemed (for a little while at least) unnecessary.

Perhaps the same will happen to module.  Ain't over till the fat lady sings or something.

> _ENV is a core construct of Lua,

No. _ENV is the latest iteration of many attempt to get ride of setfenv/getfenv in 5.2. And 5.2 is a work of fiction as it hasn't been released.

> I'd also note that it's not necessary or beneficial to touch _ENV at
> all in your module unless you're doing something weird.

Creating one's own local namespace seems rather pedestrian to me, but to each its own.
Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Javier Guerra Giraldez
In reply to this post by Eduardo Ochs
On Sun, Oct 16, 2011 at 4:39 PM, Eduardo Ochs <[hidden email]> wrote:
> In my opinion the two worst functions in Lua 5.1 are "require" and
> "module". They contain too much magic, and - in my opinion again -
> their implementation details are somewhat arbitrary.

require() is quite good, it does a few things more than strictly
necessary, but that allows you some flexibility in how your modules
are defined but still can be require()ed

module(), on the other hand, is the one polluting the global
namespace, and also having to meddle around with the local
environment.

that's why when i saw that it was going to be deprecated, i simply switched to:

local Modulename = {}

.... populate Modulename

return Modulename


and it works perfectly well for my needs.  when I want a slightly more
OO-style, i might add a metatable to the module table, that's just a
couple lines extra. nothing to write home about.

module() is gone, the replacement is far simpler, the structure is
unchanged.  sounds good to me, but a very trivial adjustment.

I really don't get why to keep discussing about it.

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille

On Oct 17, 2011, at 4:28 PM, Javier Guerra Giraldez wrote:

> module() is gone,

It's not gone. It's alive and kicking in 5.1.

5.2 is not released. In 5.2, it might  be deprecated. Which means it's going to be there for years to come.

> the replacement is far simpler,

Bollox :)

> I really don't get why to keep discussing about it.

Well, is there a party line we have to toe? Or is it acceptable to voice discontent?
Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Sam Roberts
On Mon, Oct 17, 2011 at 9:07 AM, Petite Abeille
<[hidden email]> wrote:

>
> On Oct 17, 2011, at 4:28 PM, Javier Guerra Giraldez wrote:
>
>> module() is gone,
>
> It's not gone. It's alive and kicking in 5.1.
>
> 5.2 is not released. In 5.2, it might  be deprecated. Which means it's going to be there for years to come.
>
>> the replacement is far simpler,
>
> Bollox :)
>
>> I really don't get why to keep discussing about it.

Because module and require worked fine, and provided a common way to
implement modules.

Now we are trying to support two ways, check out this code at about line 60:

https://github.com/dcurrie/lunit/blob/master/lunit.lua

> Well, is there a party line we have to toe? Or is it acceptable to voice discontent?

Petite, probably its not useful to voice discontent... though I share
your pain, and I'm going to voice mine, too!

The occaisonal suggestion that lua programmers widely disliked
module() is pretty galling, and I'm tempted to deface that wiki page
people keep pointing to like it represents some kind of consensus
opinion, when its only the opinion of a few.

I like module(), in particular, I like the behaviour that it sets up
globally reachable names for modules, and I like that I can call it
like module(...,package.seeall), and while containing no internal
reference to its actual name, it will build itself into the global
namespace depending on where its deployed in the filesystem.

The require() function defines a namespace, a namespace that is global
to the lua state, inherited from the filesystem, and in which
conflicts must be avoided (you can't have two modules named "bit", not
with the old module(), and not with the new _ENV={}, return _ENV
idiom, either).

Given that the names cannot collide, injecting them as globals is handy.

However, this is clearly the losing side of the argument, and I won't
be porting our codebase to 5.2 for quite a while, anyway.

Having two ways to define modules is going to make it harder to keep
my published modules up to date and useable for both lua versions. Lua
is not exactly swimming in an ecosystem of useful libraries, having
most of the 5.1 ones not work in 5.2 until they are ported to the _ENV
system is a bit awkward, but so it goes.

Cheers,
Sam

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Hisham Muhammad
On Mon, Oct 17, 2011 at 3:00 PM, Sam Roberts <[hidden email]> wrote:

> On Mon, Oct 17, 2011 at 9:07 AM, Petite Abeille
> <[hidden email]> wrote:
>>
>> On Oct 17, 2011, at 4:28 PM, Javier Guerra Giraldez wrote:
>>
>>> module() is gone,
>>
>> It's not gone. It's alive and kicking in 5.1.
>>
>> 5.2 is not released. In 5.2, it might  be deprecated. Which means it's going to be there for years to come.
>>
>>> the replacement is far simpler,
>>
>> Bollox :)
>>
>>> I really don't get why to keep discussing about it.
>
> Because module and require worked fine, and provided a common way to
> implement modules.
>
> Now we are trying to support two ways, check out this code at about line 60:
>
> https://github.com/dcurrie/lunit/blob/master/lunit.lua
>
>> Well, is there a party line we have to toe? Or is it acceptable to voice discontent?
>
> Petite, probably its not useful to voice discontent... though I share
> your pain, and I'm going to voice mine, too!
>
> The occaisonal suggestion that lua programmers widely disliked
> module() is pretty galling, and I'm tempted to deface that wiki page
> people keep pointing to like it represents some kind of consensus
> opinion, when its only the opinion of a few.

Yes, you are not alone. I agree that it is not a consensus opinion.

Even though that "Lua Module Function Critiqued" page (mostly written
by David Manura?) is sponsored by some influential members of the Lua
community (David, Mike Pall, etc.) and the points written there are
indeed valid, I think that those problems are often overrated, and we
shouldn't forget about the good aspects of module() as well.

Let's start by remembering the circumstances that brought us module().
For long, Lua has lacked behind other languages in its ability to
write reusable, modular code. First we got require() in Lua 5.0, which
allowed us to abstract away the some of the filesystem specifics in
scripts (but lacking, AFAIR, submodules). require() didn't enforce the
return of the module, so some packages would just add to the global
namespace, some would return a table, etc.

The module() function gave us an easy answer for the question "how do
I write a module" that mirror other languages that offer constructs
for modularity. Yes, the implementation has the pitfalls listed in the
"Critiqued" page. Yes, having to add something like "package.seeall"
to get something similar to what would be the default behavior in most
other languages is unfortunate. But it's really nice to be able to
open a source file and read something like " module('foo.bar.baz') "
on top, which gives a clear indication of how this module should be
loaded.

I've always seen the evolution of the Lua modules system (require in
Lua 5.0, module in Lua 5.1, compat-5.1 for Lua 5.0) as an attempt to
provide some standardization in order to promote code interoperability
and reusability. As Roberto says in PiL:

"Usually, Lua does not set policies. Instead, Lua provides mechanisms
that are powerful enough for groups of developers to implement the
policies that best suit them. However, this approach does not work
well for modules. One of the main goals of a module system is to allow
different groups to share code. The lack of a common policy impedes
this sharing."

The implementation of module() in Lua 5.1 may not be ideal, but
requiring users to learn about _ENV in order to write a reusable
module is, in my opinion, worse. Yes, it's a mechanism that's powerful
enough to get the job done, but don't we risk to go back to the Lua
5.0 days in terms of code sharing?

My opinion is that modules are important enough so that there should
be a "blessed" way of writing modules, and preferrably a somewhat
obvious one. There has to be a policy.

Looking at modules that are contributed to LuaRocks, I see that
module() is popular among developers and promotes a clean, uniform way
of describing modules, whereas among those who don't use it you see
all styles of coding. It's nice to look at module() on top and see
that the file was designed to be require()'d, that it won't pollute
the global namespace beyond the name of the module (yes, would be even
nicer if it didn't pollute the environment at all), that you can
assign the return of require() to a local. It's important to have
those guarantees when writing code with reusable components.

If module() is broken beyond repair and there is no clean way to add a
library function to do that, I'd dare to say that this is something
important enough that a language construct would be justifiable (along
the lines of ":" being justifiable for the sake of OOP). (Perhaps even
the "in ... end" that was once proposed for environments? (But please,
let's stick to discussing the merits/problems of a canonical module
construct, rather than diving in yet another
let's-discuss-syntax-details thread)).

-- Hisham
http://hisham.hm/ - http://luarocks.org/

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Luiz Henrique de Figueiredo
> My opinion is that modules are important enough so that there should
> be a "blessed" way of writing modules, and preferrably a somewhat
> obvious one. There has to be a policy.

The recommended way of writing modules is the trivial way: a module should
return a table with its data and functions. It'd be nice if it also did
not pollute the global environment. In Lua, this is done trivially with:

local M={}
M.version="1.0.2"
function M.f() ... end
...
return M

This way works fine for both 5.1 and 5.2

There is no need for a module function and no need for syntax either.
No one needs to know about _ENV for that only for complicated things
like avoiding writing "M." (probably not a good idea as it serves as
documentation of what is exported).

All my C modules follow this (but do create a global as well). Changing
them to 5.2 is really easy, even removing the creation of the global.

I do not see any big deal here.

Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Petite Abeille

On Oct 17, 2011, at 11:11 PM, Luiz Henrique de Figueiredo wrote:

> In Lua, this is done trivially with:

Luiz, while you are at it, why don't you demonstrate how to produce the functional equivalent of that single line of code in 5.2:

module( 'foo' )

And please, no trivialization, nor hand waving :)

> I do not see any big deal here.

Indeed. Perhaps this is the issue as well.



Reply | Threaded
Open this post in threaded view
|

Re: modules, require, magic

Pierre Chapuis
In reply to this post by Luiz Henrique de Figueiredo
On Mon, 17 Oct 2011 19:11:38 -0200, Luiz Henrique de Figueiredo wrote:

> local M={}
> M.version="1.0.2"
> function M.f() ... end
> ...
> return M

> There is no need for a module function and no need for syntax either.
> No one needs to know about _ENV for that only for complicated things
> like avoiding writing "M." (probably not a good idea as it serves as
> documentation of what is exported).

I use an alternative construction that does the same thing in the end:

local f = function() ... end
return {
   version = "1.0.2",
   f = f,
}

I also think modules shouldn't modify the global namespace.
This solution doesn't allow concatenation of module files but this is
a small price to pay for a simple, understandable module mechanism.

--
Pierre Chapuis

Reply | Threaded
Open this post in threaded view
|

RE: modules, require, magic

Thijs Schreijer
In reply to this post by Petite Abeille
This thread starts to look like some major code duplication, can anyone come
up with a way to package this into something re-usable?

Something like;

    module("myOpinionAllOverThePlace",
package.show_everyone_how_great_it_is)
    return 42

And then stop this. (please)




1234 ... 8