Re: Private modules, was Calling module from a C function

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Jim Whitehead II
For the moment, if some particular Lua users replace "module" with a
custom version more adapted to its environment (for example in some
project I'd like "module" not to set the global tables, only register
the module in package.loaded), Lua modules will still work with that
environment, but binary modules may fail to. Calling "module" will
ensure that the code provided by the user get called, the code necessary
to adapt the module to the new environment "from the inside".

I've been meaning to ask this for some time, but there are a number of
concerns on the http://lua-users.org/wiki/LuaDesignPatterns page,
namely the pollution of the global namespace by the module system, and
the package.seeall flag which gives us the odd effect:

local someModule = require("someModule")
local one = someModule.math.floor(1.4)

Is there any chance of these being made a bit more friendly, perhaps
not setting the globals unless specified somewhere?

Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Roberto Ierusalimschy
> I've been meaning to ask this for some time, but there are a number of
> concerns on the http://lua-users.org/wiki/LuaDesignPatterns page,
> namely the pollution of the global namespace by the module system, and
> the package.seeall flag which gives us the odd effect:
> 
> local someModule = require("someModule")
> local one = someModule.math.floor(1.4)
> 
> Is there any chance of these being made a bit more friendly, perhaps
> not setting the globals unless specified somewhere?

I don't think so. As far as I know, the "pollution" of the global
namespace by one name per module is quite theoretical. The problem
created by 'seeall' is trivially fixed by not using 'seeall'. At the
same time, both global names and 'seeall' modules are quite convenient
for a "more relaxed" programming approach.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Thomas Lauer-3
[hidden email] (Roberto Ierusalimschy) wrote:
> I don't think so. As far as I know, the "pollution" of the global
> namespace by one name per module is quite theoretical. The problem
> created by 'seeall' is trivially fixed by not using 'seeall'.

I like these simple solutions:-)

However, an official function in module package to *selectively* import
stuff from other modules would be a welcome addition to the module
system, IMHO. I know that's easy to write, but as usual everybody will
have his or her own version and it all won't play nicely together.
-- 
cheers  thomasl

web : http://thomaslauer.com/start


Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Tomás Guisasola-2
On Wed, 25 Apr 2007, Thomas Lauer wrote:
> [hidden email] (Roberto Ierusalimschy) wrote:
> > I don't think so. As far as I know, the "pollution" of the global
> > namespace by one name per module is quite theoretical. The problem
> > created by 'seeall' is trivially fixed by not using 'seeall'.
> 
> I like these simple solutions:-)
> 
> However, an official function in module package to *selectively* import
> stuff from other modules would be a welcome addition to the module
> system, IMHO. I know that's easy to write, but as usual everybody will
> have his or her own version and it all won't play nicely together.
	Have you wrote your own version?  I think a copy function
is easy to write, but I'm not certain on how to handle function
dependencies when importing a function f from module A to module B.
What to do if f uses a global g and B already has a member g?
	For my purposes, I wrote a substitute for module:

function addtomodule (modname, requiredname)
    local _M = require(modname)
    if requiredname then
        package.loaded[requiredname] = _M
    end
    setfenv (2, _M)
end

	It uses an already defined module instead of creating a new
table.  In fact, I don't like it.  I would prefer to let the user
decide if he/she wants to open a module inside another one, instead
of the module's developer.  I would like to write:

local table = require("table.extra", "table")

	The `table/extra.lua' file could be written:

local modname = select(2, ...) or select(1, ...)
module (modname)

	But `require' only passes its first argument to the script :-(
		Tomás


Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Thomas Lauer-3
Tomas Guisasola Gorham <[hidden email]> wrote:
> On Wed, 25 Apr 2007, Thomas Lauer wrote:
> > However, an official function in module package to *selectively* import
> > stuff from other modules would be a welcome addition to the module
> > system, IMHO. I know that's easy to write, but as usual everybody will
> > have his or her own version and it all won't play nicely together.
>
> 	Have you wrote your own version?  I think a copy function
> is easy to write, but I'm not certain on how to handle function
> dependencies when importing a function f from module A to module B.
> What to do if f uses a global g and B already has a member g?

Hm... this should only be of concern if the modules in question use
'seeall'. Don't use it and this sort of problem can't happen: all
accesses to functions are then "hard-wired" in their respective modules
and as such won't go through an __index metacall. (Of course, it is
still perfectly possible to willy-nilly change any and all addresses in
the module tables, but this has to be done explicitly.)

But perhaps I have misunderstood your question.

> 	But `require' only passes its first argument to the script :-(

Yeah, that's one of my pet peeves as well. It would be a really great
addition if ll_require() would push further arguments before it calls
the loaded module. ll_module() could then simply access these as further
parameters. This should even work with C libraries.

Alas, doing this would break most existing module code.



Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Gé Weijers

On Apr 26, 2007, at 5:03 AM, Thomas Lauer wrote:


But `require' only passes its first argument to the script :-(

Yeah, that's one of my pet peeves as well. It would be a really great
addition if ll_require() would push further arguments before it calls
the loaded module. ll_module() could then simply access these as further
parameters. This should even work with C libraries.

Alas, doing this would break most existing module code.


The arguments passed would come from the 'chunk' that happens to import the module first, and the correct functioning of your code would then depend on the order in which you directly or indirectly import modules. I'm not sure that that's what you'd want from a module system.


--
Gé Weijers



Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Thomas Lauer-3
Gé Weijers <[hidden email]> wrote:
> On Apr 26, 2007, at 5:03 AM, Thomas Lauer wrote:
> >
> > Yeah, that's one of my pet peeves as well. It would be a really great
> > addition if ll_require() would push further arguments before it calls
> > the loaded module. ll_module() could then simply access these as  
> > further
> > parameters. This should even work with C libraries.
> >
> > Alas, doing this would break most existing module code.
> >
> 
> The arguments passed would come from the 'chunk' that happens to  
> import the module first, and the correct functioning of your code  
> would then depend on the order in which you directly or indirectly  
> import modules. I'm not sure that that's what you'd want from a  
> module system.

Well, it depends. Such a feature would be completely optional to start
with. Someone who doesn't need (or doesn't want) this wouldn't use it in
his or her modules.

Second, any parameter beyond the first (the name) would be optional as
well, much like you can call functions with varying parameter lists and
create some sensible defaults if a parameter is nil. Writers of such
module(s) would structure their code in such a way that they take this
into account.

Third, module parameters might make handling complex systems with many
internal interdependencies and one or two controlling main modules
easier, exactly because it would give module users more control about
how they use a module and modules writers more control about how their
modules are used. I think what you see as a disadvantage can actually be
an advantage. At least in a properly designed system.

Which brings me to point 4: this feature would of course only be used by
people who know what they're doing, like all these dangerous features:-)

-- 
cheers  thomasl

web : http://thomaslauer.com/start


Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Tomás Guisasola-2
In reply to this post by Thomas Lauer-3
	Hi Thomas

> > 	Have you wrote your own version?  I think a copy function
> > is easy to write, but I'm not certain on how to handle function
> > dependencies when importing a function f from module A to module B.
> > What to do if f uses a global g and B already has a member g?
> 
> Hm... this should only be of concern if the modules in question use
> 'seeall'. Don't use it and this sort of problem can't happen: all
> accesses to functions are then "hard-wired" in their respective modules
> and as such won't go through an __index metacall. (Of course, it is
> still perfectly possible to willy-nilly change any and all addresses in
> the module tables, but this has to be done explicitly.)
	I mean:

== A.lua ==
module"A"
g = 10
function f (x) return x+g end

== B.lua ==
local A = require"A"
module"B"
g = 20
import_f_from_A

	What will be the result of B.f(2) ?

> > 	But `require' only passes its first argument to the script :-(
> 
> Yeah, that's one of my pet peeves as well. It would be a really great
> addition if ll_require() would push further arguments before it calls
> the loaded module. ll_module() could then simply access these as further
> parameters. This should even work with C libraries.
> 
> Alas, doing this would break most existing module code.
	Why?

	Regards,
		Tomás


Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Thomas Lauer-3
Tomas Guisasola Gorham <[hidden email]> wrote:
> 	I mean:
> 
> == A.lua ==
> module"A"
> g = 10
> function f (x) return x+g end
> 
> == B.lua ==
> local A = require"A"
> module"B"
> g = 20
> import_f_from_A
> 
> 	What will be the result of B.f(2) ?

Did you try?

Simulating a Lua VM I would say that B.f() calls A.f() which has a
hard-coded reference to its "local" global g. It doesn't know the first
thing about B's environment and its g (that's assuming your opaque
import_f_from_A call hasn't inadvertently mixed up environments).

So this call returns 12. If it doesn't either Lua or I am broken.

> > > 	But `require' only passes its first argument to the script :-(
> > 
> > Yeah, that's one of my pet peeves as well. It would be a really great
> > addition if ll_require() would push further arguments before it calls
> > the loaded module. ll_module() could then simply access these as further
> > parameters. This should even work with C libraries.
> > 
> > Alas, doing this would break most existing module code.
> 	Why?

I can't see how the current module system would cleanly refer to more
than one parameter, given that module()'s list of optional parameters
(after the name) is a list of functions to be applied over the module.

Modules are strange hybrids: module() is a function call in its own
right but a module is also, as it were, on the receiving end of a call.
Try this:

-- in a.lua
local g=_G
print(...)
module(...)
g.print(_M._NAME)

Next copy a.lua to b.lua and then run this file:

-- test.lua
require("a")
require("b")

This feature allows a certain degree of hacking.

(Small aside: I've just hacked the sources to try multiple parameters
for require(). It's possible to change ll_require() such that it pushes
more parameters and ll_module() such that a module accepts them in (...)
without crashing lua. However, I don't know enough about the inner
workings of those functions (and their support functions) to be sure
what goes on. Time permitting, I will investigate.)

-- 
cheers  thomasl

web : http://thomaslauer.com/start


Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Tomás Guisasola-2
	Hi Thomas

On Thu, 26 Apr 2007, Thomas Lauer wrote:

> Tomas Guisasola Gorham <[hidden email]> wrote:
> > 	I mean:
> > 
> > == A.lua ==
> > module"A"
> > g = 10
> > function f (x) return x+g end
> > 
> > == B.lua ==
> > local A = require"A"
> > module"B"
> > g = 20
> > import_f_from_A
> > 
> > 	What will be the result of B.f(2) ?
> 
> Did you try?
	Surelly not.  There is no `import_f_from_A'.  I was just
trying to understand the semantics of this import function you
were proposing.

> > > Alas, doing this would break most existing module code.
> > 	Why?
> 
> I can't see how the current module system would cleanly refer to more
> than one parameter, given that module()'s list of optional parameters
> (after the name) is a list of functions to be applied over the module.
	Yes, code like `module(...)' will not work if required
with more than one argument (supposing `require' will pass them
to the module).

	Regards,
		Tomás


Reply | Threaded
Open this post in threaded view
|

RE: Private modules, was Calling module from a C function

Jerome Vuarand-2
Tomas Guisasola wrote:
> On Thu, 26 Apr 2007, Thomas Lauer wrote:
>> I can't see how the current module system would cleanly refer to more
>> than one parameter, given that module()'s list of optional parameters
>> (after the name) is a list of functions to be applied over the
>> module. 
> 	Yes, code like `module(...)' will not work if required with more
> than one argument (supposing `require' will pass them to the module). 

There's another issue with multiple parameters to require. Require
parameter got passed not only to the module itself, but also to the
module loader which loads the module. It would be nice to pass optionnal
parameters to the module loader. For example imagine you have a loader
that will retrieve modules from an online module repository (through
http), it would be nice to pass the url of the server as the require
second argument. From my point of view that would be much more useful to
parameterize the loaders than the module themselves.

So how would you determine which arguments of require goes to the
loader, and which goes to the module ? With the fact that many modules
beginning with module(...) instead of module((...)) will fail to work if
the module chunk get more parameter, I think it's clear that adding more
parameters to require needs at least a partial rethink of the module
system before it becomes a standard.

On the other hand, for a given application, you can easily and safely
modify require, module and the loaders to manage these extra arguments.
But you won't be able to use third-party modules.


Reply | Threaded
Open this post in threaded view
|

RE: Private modules, was Calling module from a C function

Tomás Guisasola-2
	Hi Jerome

> There's another issue with multiple parameters to require. Require
> parameter got passed not only to the module itself, but also to the
> module loader which loads the module. It would be nice to pass optionnal
> parameters to the module loader. For example imagine you have a loader
> that will retrieve modules from an online module repository (through
> http), it would be nice to pass the url of the server as the require
> second argument. From my point of view that would be much more useful to
> parameterize the loaders than the module themselves.
	Maybe I am not following your thoughts but I think that this
special loader might be configured to work properly in a particular
environment and this should not be the case of the programmer that is
trying to load a module.  Don't you agree?

> So how would you determine which arguments of require goes to the
> loader, and which goes to the module ? With the fact that many modules
> beginning with module(...) instead of module((...)) will fail to work if
> the module chunk get more parameter, I think it's clear that adding more
> parameters to require needs at least a partial rethink of the module
> system before it becomes a standard.
	Sure, but I think this discussion could help :-)

> On the other hand, for a given application, you can easily and safely
> modify require, module and the loaders to manage these extra arguments.
> But you won't be able to use third-party modules.
	In this case it is not a simple task, because you will
have to rewrite `require' -- which is not a simple function --
to change only the call of the module function!

	Regards,
		Tomás


Reply | Threaded
Open this post in threaded view
|

RE: Private modules, was Calling module from a C function

Jerome Vuarand-2
Tomas Guisasola wrote:
>> There's another issue with multiple parameters to require. Require
>> parameter got passed not only to the module itself, but also to the
>> module loader which loads the module. It would be nice to pass
>> optionnal parameters to the module loader. For example imagine you
>> have a loader that will retrieve modules from an online module
>> repository (through http), it would be nice to pass the url of the
>> server as the require second argument. From my point of view that
>> would be much more useful to parameterize the loaders than the
>> module themselves. 
> 	Maybe I am not following your thoughts but I think that this
special
> loader might be configured to work properly in a particular
> environment and this should not be the case of the programmer that is
> trying to load a module.  Don't you agree?   

An example may be clearer. Suppose that you have a loader that get Lua
rocks from the net:

-- Install the loader
require("rocks.loaders.http")

-- Get some modules from the official rocks website (fictionnal url)
require("lxp", "http://rocks.lua.org/";)
require("pdf", "http://rocks.lua.org/";)

-- Get some modules from your corporate intranet quality assurance site
require("syslog", "http://qa/";)

And from here you can use modules lxp, pdf, and syslog as normal
modules. There are other solutions to the problem, I could for example
embed the url in the module name like require("org.lua.rocks.lxp"), or
as methods of the loader module, but that can be done to modify normal
modules too. My point is that extra parameters to require may be used to
configure how module are loaded rather than how modules will work.

>> On the other hand, for a given application, you can easily and safely
>> modify require, module and the loaders to manage these extra
>> arguments. 
>> But you won't be able to use third-party modules.
> 	In this case it is not a simple task, because you will have to
> rewrite `require' -- which is not a simple function -- to change only
> the call of the module function!  

'require' is not that complicated. It's clearly organized around a
loader concept, and it's very generic and flexible. The apparent
complexity shown in its manual entry comes from the fact that there are
4 different loaders for only 2 module types in stock Lua, but each
loader independently is also very simple. They are ordered in such a
clever way as to handle several special cases.


Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Thomas Lauer-3
"Jerome Vuarand" <[hidden email]> wrote:
> An example may be clearer. Suppose that you have a loader that get Lua
> rocks from the net:
> 
> -- Install the loader
> require("rocks.loaders.http")
> 
> -- Get some modules from the official rocks website (fictionnal url)
> require("lxp", "http://rocks.lua.org/";)
> require("pdf", "http://rocks.lua.org/";)
> 
> -- Get some modules from your corporate intranet quality assurance site
> require("syslog", "http://qa/";)
> 
> And from here you can use modules lxp, pdf, and syslog as normal
> modules. There are other solutions to the problem, I could for example
> embed the url in the module name like require("org.lua.rocks.lxp"), or
> as methods of the loader module, but that can be done to modify normal
> modules too. My point is that extra parameters to require may be used to
> configure how module are loaded rather than how modules will work.

Such an extension to the current model could perhaps be implemented with
a table as require()'s parameter. Most, if not all, fields would be
predefined, so as to standardise usage. Ideas would include:

require("syslog")       -- would still work, with same defaults as now
-- or alternatively
syslog.name="syslog"
syslog.loadmethod="lua" -- could also support a pointer to a function
syslog.location="http://localsite/";
syslog.param={"whatever"}  -- could be used for module initialisation
require(syslog)

There are certainly other things one could think of.

Such an approach would basically parameterise the "monolithic"
require(). And if the loaded module gets this table as well, it could
get information about how and where it was loaded, what parameters were
used etc.

-- 
cheers  thomasl

web : http://thomaslauer.com/start


Reply | Threaded
Open this post in threaded view
|

Re: Private modules, was Calling module from a C function

Mike Schmitz
Thomas Lauer wrote:
>
> Such an extension to the current model could perhaps be implemented with
> a table as require()'s parameter. Most, if not all, fields would be
> predefined, so as to standardise usage. Ideas would include:
>
> require("syslog")       -- would still work, with same defaults as now
> -- or alternatively
> syslog.name="syslog"
> syslog.loadmethod="lua" -- could also support a pointer to a function
> syslog.location="http://localsite/";
> syslog.param={"whatever"}  -- could be used for module initialisation
> require(syslog)
>
> There are certainly other things one could think of.
>
> Such an approach would basically parameterise the "monolithic"
> require(). And if the loaded module gets this table as well, it could
> get information about how and where it was loaded, what parameters were
> used etc.
>
>   

Maybe I am missing something, but can't this be done w/the existing
require?  Like this:

syslog = require("syslog")
syslog.location = "http://localsite/";
syslog.param = {"whatever"}



Reply | Threaded
Open this post in threaded view
|

RE: Private modules, was Calling module from a C function

Tomás Guisasola-2
In reply to this post by Jerome Vuarand-2
	Hi Jerome

> An example may be clearer. Suppose that you have a loader that get Lua
> rocks from the net:
> 
> -- Install the loader
> require("rocks.loaders.http")
> 
> -- Get some modules from the official rocks website (fictionnal url)
> require("lxp", "http://rocks.lua.org/";)
> require("pdf", "http://rocks.lua.org/";)
> 
> -- Get some modules from your corporate intranet quality assurance site
> require("syslog", "http://qa/";)
> 
> And from here you can use modules lxp, pdf, and syslog as normal
> modules. There are other solutions to the problem, I could for example
> embed the url in the module name like require("org.lua.rocks.lxp"), or
> as methods of the loader module, but that can be done to modify normal
> modules too. My point is that extra parameters to require may be used to
> configure how module are loaded rather than how modules will work.
	Why don't you use a `package.urlpaths' to configure that?

package.urlpaths = {
	lxp = "http://rocks.lua.org/";,
	pdf = "http://rocks.lua.org/";,
}

	You already could implement a loader like that, don't you?

	In fact, I think my proposal is not good enough, but I don't
have a better one.  Maybe someone else has :-)

	Regards,
		Tomás


Reply | Threaded
Open this post in threaded view
|

RE: Private modules, was Calling module from a C function

Jerome Vuarand-2
In reply to this post by Mike Schmitz
Mike Schmitz wrote:
> Maybe I am missing something, but can't this be done w/the existing
> require?  Like this: 
> 
> syslog = require("syslog")
> syslog.location = "http://localsite/";
> syslog.param = {"whatever"}

No, syslog.lua doesn't exist on the local computer, the url is not a
parameter of the module itself but of the loader of the module. In my
example I used a ficionnal module that modify loader table to add one
that gets modules from the network.