Packaging and importing

classic Classic list List threaded Threaded
66 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Packaging and importing

Chris Marrin

I have been trying to learn about Lua 5.1's new packaging mechanism. I have two main questions. First, there seems to be some contradictions to how it works. In looking at the file lom.lua in the luaexpat package, it looks like you can go:

    require "foo"
    local a = foo.bar(5)

This is nice and simple. But in reading the message that describes how it works at:

    http://lua-users.org/lists/lua-l/2004-09/msg00342.html

There are a couple of things that don't seem to allow this. First of all, it looks like the spec says you have to go:

    local foo = require "foo"
    local a = foo.bar(5)

In other words, does the package get put into the global namespace or not? Also, the spec seems to say that it searches for files of the form ?.so or ?.dll, but it says nothing about packages written in Lua itself. In looking at lom.lua it looks like this is certainly possible.

Is my problem that the feature is not yet well documented? If so, that is fine. I just want a clear idea of what the functionality will be, because that leads me to my second question:

Does Lua have any capability for importing names from a package into the local namespace, like you see in Python? In that language, you have this:

    import foo
    from foo import bar
    from foo import *

In the first form, this is pretty much equivalent to require in Lua. It puts a symbol called "foo" in the namespace and all its functionality can be accessed from that. So you would go:

    a = foo.bar(5)

just like in Lua. The second form looks at the foo package, extracts its bar symbol and adds that to the namespace. So you can now go:

    a = bar(5)

The last form looks into foo and extracts ALL its symbols and adds them to the namespace.

Has this issue been discussed? I have found a few threads discussing import and require in general (I assume that these are what resulted in the current spec) but I never saw a resolution about the "from...import" issue. This looked like a promising comment from Mike Pall:

    http://lua-users.org/lists/lua-l/2004-06/msg00069.html

but the "from...import" argument didn't really go anywhere.

One other detail of this has to do with specific namespaces. On the list I have seen concern about putting package names into the GLOBAL namespace. Because then some module could rely on a package getting loaded by another. But Python has the concept of module specific namespaces. So that, if file A imported package foo, the name foo would be available throughout A, but not to another file B. I know Lua has the concept of environments (which I am still trying to understnad fully) so maybe that deals with this issue as well.

Anyway, just trying to understand how all this packaging stuff works...

--
chris marrin                    "As a general rule,don't solve puzzles
[hidden email]                 that open portals to Hell" ,,.


Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Petite Abeille

On Aug 13, 2005, at 17:18, Chris Marrin wrote:

In other words, does the package get put into the global namespace or not?

It depends.

In theory, they shouldn't. In practice, any package can stick whatever it feels like in _G, itself included. Or highjack _G altogether. To complicate matters further more, a package doesn't have to return itself either.

For example, most of Luiz packages don't play well with 'require':

local md5 = require( "md5" )

print( type( md5 ), md5 )

> string  md5

Instead of returning itself, the md5 package returns its name.

Diego favors modules which register themselves with _G no matter what. But at least they return themselves as well:

local socket = require( "socket" )

print( type( socket ), socket )

> table   table: 0x118480

In other words, anything goes :)

It would be nice though to have a consistent behavior, e.g. 'require' always returning a table of functions, instead of the current unpredictable situation...

On the other hand, well, it's not such a big issue in practice as you are most likely in full control of your environment and can impose whatever policy you see fit.

For example, LU never defines anything in _G and always returns its functionalities as a table of functions:

-- import dependencies
local LUObject = require( "LUObject" )

-- define the class
local self = LUObject()

...

return self

http://dev.alt.textdrive.com/browser/lu/

Cheers

--
PA, Onnay Equitursay
http://alt.textdrive.com/


Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Chris Marrin
PA wrote:


...
In other words, anything goes :)

It would be nice though to have a consistent behavior, e.g. 'require' always returning a table of functions, instead of the current unpredictable situation...

On the other hand, well, it's not such a big issue in practice as you are most likely in full control of your environment and can impose whatever policy you see fit.

Right, but this does not scale well. It requires that anyone using any package has to understand the rules of that package. That does not bode well for wide adoption of Lua and its very nice array of supporting packages. Seems like it would be better if require did some checks to ensure that rules were being followed. For instance, it could prevent the package being loaded from putting things into the global table. And it could throw an error if the return value was not a table.

And hopefully the doc describing how to create a package will lay out some conventions, so at least there will be guidelines on how to conform.

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Diego Nehab-3
In reply to this post by Petite Abeille
Hi,

Diego favors modules which register themselves with _G no matter what. But at least they return themselves as well:

I actually would rather they didn't register themselves with _G. What I
favour is the mechanism setup by module() that makes any globals
declared by the module fall inside the namespace. I also favour each
module returning its namespace.

[]s,
Diego.

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Petite Abeille
In reply to this post by Chris Marrin

On Aug 13, 2005, at 19:29, Chris Marrin wrote:

It requires that anyone using any package has to understand the rules of that package.

Yep.

That does not bode well for wide adoption of Lua and its very nice array of supporting packages.

Well, perhaps. It depend of what you are looking for. Lua imposes very few, if any, "policies" to speak of. It's more of a "do-it-yourself" type of environment.

Seems like it would be better if require did some checks to ensure that rules were being followed. For instance, it could prevent the package being loaded from putting things into the global table. And it could throw an error if the return value was not a table.

Perhaps.

On the other hand, Lua seems to be very much against imposing any kind of policies anywhere. As mentioned somewhere in "Programming in Lua":

"Lua gives you the power, you build the mechanisms"

While nice sounding as a mission statement, the above philosophy would also explain the paucity of reusable Lua code. There is just not enough common ground to build any reusable corpus given that no one can/want/need to agree on a minimum set of standard policies which would make it possible in the first place.

This is not necessarily a bad thing in and by itself... but it's most definitively something to get use to as this approach greatly differs from most other language environments which offer not only a programming language but an entire set of rules and regulations as well as libraries covering everything and the kitchen sink.

Cheers

--
PA, Onnay Equitursay
http://alt.textdrive.com/


Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Lisa Parratt
On 13 Aug 2005, at 19:32, PA wrote:
While nice sounding as a mission statement, the above philosophy would also explain the paucity of reusable Lua code.

It is a shame - the first thing I had to do when deploying Lua for my day job was formulate a whole set of policies and frameworks so that Lua code *will* be reusable and predictable.

Unfortunately, the side effect of this is that none of the source will ever make it to the outside world, since it's all so dependent on said core policies and frameworks.

--
Lisa

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Luiz Henrique de Figueiredo
In reply to this post by Petite Abeille
> For example, most of Luiz packages don't play well with 'require':

They will be fixed for Lua 5.1. --lhf

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Diego Nehab-3
In reply to this post by Chris Marrin
Hi,

The package proposal allows for a lot of freedom. This is because Lua is
used embeded in applications that might have especial needs.

But does suggests the standard way of writing a package. This is with
the use of the module() function.  This function has a very well
specified behaviour. If your package uses module() (and it should use
it), then both

    local foo = require"foo"
    a = foo.bar(5)

and

    require"foo"
    a = foo.bar(5)

will work. That's it. This is true of all libraries that don't try to be
creative by not using module().

The proposal also specifies search order and search locations for all
types of libraries. It also specifies how to override many details that
can be specific to your deployment platform, doing so in such a way that
will make other people's libraries work in your system anyways.

At the moment, we are in transition. In the future, all libraries should
be be consistent. Please consider using the package proposal, as it will
make everyone's life easier in the long run.

If there is anything you would like to be able to do when you package
your libraries, but feel you can't do with our proposal, this is the
right venue to talk about it. Chances are we can figure out how to do it
so that everything is compatible.

Regards,
Diego.

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Chris Marrin
Diego Nehab wrote:
Hi,

The package proposal allows for a lot of freedom. This is because Lua is
used embeded in applications that might have especial needs.

But does suggests the standard way of writing a package. This is with
the use of the module() function.  This function has a very well
specified behaviour. If your package uses module() (and it should use
it), then both

    local foo = require"foo"
    a = foo.bar(5)

and

    require"foo"
    a = foo.bar(5)

will work. That's it. This is true of all libraries that don't try to be
creative by not using module().

That sounds good. I think library writers (as all programmers, and oh what the hell everyone) are basically lazy. If there is an easy way to do something they will use it. That's great because it means consistency will be the easy thing to do. Of couse, some library writers will have to be shamed into updating their libraries to use module :-)

Looking at the implementation of module it looks like it actually does place the module name in the global namespace. But that seems problematic for the reason mentioned here before. If I <<require "a">> then <<require "b">>, if "a" requires "c" and then "b" requires something different called "c" (because it fiddled with the path and looked someplace different), now "a" is using the wrong "c", right? Or am I missing something about how setfenv() works? I admit that I am having trouble understanding how setfenv() helps setting up an environment for an entire module when you just set it on a single function.


The proposal also specifies search order and search locations for all
types of libraries. It also specifies how to override many details that
can be specific to your deployment platform, doing so in such a way that
will make other people's libraries work in your system anyways.

At the moment, we are in transition. In the future, all libraries should
be be consistent. Please consider using the package proposal, as it will
make everyone's life easier in the long run.

Yes, it is a very good step toward consistency!


If there is anything you would like to be able to do when you package
your libraries, but feel you can't do with our proposal, this is the
right venue to talk about it. Chances are we can figure out how to do it
so that everything is compatible.

Thanks for the help so far. My big issues still have to do with isolating namespaces. For instance, what about the problem I mention above? Are modules required by "b" isolated from those required in "a", like they are in Python? Also, has anyone made a proposal for how you can promote all the members of "a" into the namespace of the module that requires "a"?

Thanks again...

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Diego Nehab-3
Hi,

Looking at the implementation of module it looks like it actually does place the module name in the global namespace. But that seems problematic for the reason mentioned here before. If I <<require "a">> then <<require "b">>, if "a" requires "c" and then "b" requires something different called "c" (because it fiddled with the path and looked someplace different), now "a" is using the wrong "c", right? Or am I missing something about how setfenv() works? I admit that I am having trouble understanding how setfenv() helps setting up an environment for an entire module when you just set it on a single function.

Underlying the package proposal, there is the assumption that there is a
one-to-one mapping between names such as "a.b" and modules residing
somewhere (such as "/usr/share/lua/5.0/a/b.lua", or
package.preload["a.b"]). There is also a caching mechanism, so that
requiring the same module a second time will be very efficient. Usually,
paths are set at startup. Fiddling with the path can have weird effects,
such as what you mention. However, it is "b" is the one that will use
the wrong "c", because "c" is already loaded and require() will not try
to load the new "c" from the new path.

In any case, the standard way to use require is to have a sequence of calls in your module

    local a = require"a"
    local b = require"b"
    ...
    module("foo")
    ...
    function bar()
    ...
    end

That way, no matter what happens, foo has a local reference to each of
the modules it needs. The setfenv simply causes bar() to end up in the
module namespace, instead of in the globals' table.

If there is anything you would like to be able to do when you package
your libraries, but feel you can't do with our proposal, this is the
right venue to talk about it. Chances are we can figure out how to do it
so that everything is compatible.

Thanks for the help so far. My big issues still have to do with isolating namespaces. For instance, what about the problem I mention above? Are modules required by "b" isolated from those required in "a", like they are in Python?

No. All modules are available globally. If both "a" and "b" require"c",
both will share the same module. That said, Lua is powerful enough for
you to override this behavior. I believe VEnv, used with CGILua, does that. Once again, Lua gives you the power. If you need a sandbox, it can be done.

Also, has anyone made a proposal for how you can promote all the members of "a" into the namespace of the module that requires "a"?

There is no standard way of doing this. The naive idea of doing
something like

    local a = require"a"
    for i,v in pairs(a) do
        _G[i] = v
    end

does not work. That is because within "a", functions might be
referencing symbols through the namespace table directly, so that if you
change one of the exported symbols, "a" might not see the change. This
is not true of all modules, but I wouldn't bet on it working everywhere.

On the other hand, it would be trivial to change the behavior of
module() so that, instead of seting the environment to the namespace
table, it used the globals' table instead.

The problem is moving a module from it's namespace to the globals'
table...

[]s,
Diego.





Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Tomas-14
	Hi,

Thanks for the help so far. My big issues still have to do with isolating namespaces. For instance, what about the problem I mention above? Are modules required by "b" isolated from those required in "a", like they are in Python?

No. All modules are available globally. If both "a" and "b" require"c",
both will share the same module. That said, Lua is powerful enough for
you to override this behavior. I believe VEnv, used with CGILua, does that. Once again, Lua gives you the power. If you need a sandbox, it can be done.
	What I have to add to Diego's explanation is that a module
that needs a certain global could store it in a local not only to gain
some speed, but also to guarantee that this global will be exactly the
same during its execution.  For example:

require"c"
local f = c.f
module"a"

function x (s)
	f(s)
	c.g(s)
end

	`f' is the local copy of `c.f', made after the require"c"
(this required library could be an already loaded library, as Diego
told); `c.g' will be _M.c.g (_M is the module's global table created
by `module()'), but as `c' is inherited from the original globals table,
it will be _M._G.c.g, which could be easily changed outside the library.

Also, has anyone made a proposal for how you can promote all the members of "a" into the namespace of the module that requires "a"?
	Why don't you inherit the symbols from "a" with something
like:

require"a"
_M = {}
setmetatable (_M, { __index = a }) -- inherit from "a"
setfenv (1, _M) -- new environment (global table)

	If "a" inherits from the global table you'll get all global
through it!

	Tomas

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Chris Marrin
In reply to this post by Diego Nehab-3
Diego Nehab wrote:

...
Thanks for the help so far. My big issues still have to do with isolating namespaces. For instance, what about the problem I mention above? Are modules required by "b" isolated from those required in "a", like they are in Python?


No. All modules are available globally. If both "a" and "b" require"c",
both will share the same module. That said, Lua is powerful enough for
you to override this behavior. I believe VEnv, used with CGILua, does that. Once again, Lua gives you the power. If you need a sandbox, it can be done.

Yes, I agree that Lua has the power to do all this.

We are actually wrapping raw Lua in a declarative language called Emma (.ema), which looks a bit like VRML, but essentially generates Lua source, which then is loaded and run. The language lets you declare objects and their hierachical relationship. The resultant scene graph is just a big hierarchy of these objects. Each .ema file defines a new, top-level object, which can be embedded in other files, which form other objects, and so on.

What I want is to have each of these files form a namespace, which can have its own imported modules that are known in that file, but not outside. I am thinking that Lua environments are what I need to do this. But I am having trouble wrapping my head around them. For instance, I have a Lua function whose job it is to return a prototype object for a class. So Emma generates something like this:

    return ProtoNode {
        foo = function(self,a) self.b = a; globalThing = a end,
        b = 0
    }

When I execute this, I get back a ProtoNode object, which is essentially a Lua table with a C++ class embedded in it. This table also has a metatable with a __call metamethod to allow me to construct instances using the call syntax. I might put this proto into the global table called 'MyProto' so I can then say:

    local p = MyProto();

and I have an instance of MyProto. When I say instance, I mean that in a Javascript/Self prototype delegation way. 'p' has a 'prototype' property which points to MyProto. The metatable also has an __index metamethod which allows me to go:

    p.foo(12)
    print(p.b)

and have it print '12'. 'p' does not have a 'foo' or a 'b' property, but the __index metamethod looks in MyProto if it doesn't find them in 'p'. All pretty standard stuff. But this function also sets the global 'globalThing'.

The question is this. If I go:

    setfenv(1, { })
    return ProtoNode {
        foo = function(self,a) self.b = a end,
        b = 0
    }

the returned function will have new, empty environment. But will the 'foo' function have that same environment? Will 'globalThing' be set in my local environment instead? Or am I missing the point of setfenv()?

I think my brain is thinking about environments for Objects, so having environments on functions doesn't click with me.

Please forgive my naive questions. Any insights would be great...

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Chris Marrin
In reply to this post by Tomas-14
Tomas wrote:

...
Also, has anyone made a proposal for how you can promote all the members of "a" into the namespace of the module that requires "a"?

    Why don't you inherit the symbols from "a" with something
like:

require"a"
_M = {}
setmetatable (_M, { __index = a }) -- inherit from "a"
setfenv (1, _M) -- new environment (global table)

    If "a" inherits from the global table you'll get all global
through it!

Ok, I'm liking this. Now I think my only question is whether all functions defined after the setfenv() inherit the same environment. If so, then I think I can make nice isolated objects this way.

Of course, I think I might have to do something a bit more complex with __index, if I want this module to be able to access the real global table as well, right? I would have to make __index a function which first checked in 'a' then in _G. Or maybe I just want to go through and copy each member of 'a' (or some subset) into _M. then I could do:

    setmetatable (_M, { __index = _G })
    ... add some or all of the members of 'a' to _M ...

then I would be able to see everything I need, wouldn't I???

--
chris marrin              ,""$, "As a general rule,don't solve puzzles
[hidden email]        b`    $  that open portals to Hell" ,,.
        ,.`           ,b`    ,`                            , 1$'
     ,|`             mP    ,`                              :$$'     ,mm
   ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
  m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
 b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
 ```"```'"    `"`         `""`        ""`                          ,P`

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Tomas-14
	Hi Chris,

On Sun, 14 Aug 2005, Chris Marrin wrote:

Tomas wrote:

...
Also, has anyone made a proposal for how you can promote all the members of "a" into the namespace of the module that requires "a"?

    Why don't you inherit the symbols from "a" with something
like:

require"a"
_M = {}
setmetatable (_M, { __index = a }) -- inherit from "a"
setfenv (1, _M) -- new environment (global table)

    If "a" inherits from the global table you'll get all global
through it!

Ok, I'm liking this. Now I think my only question is whether all functions defined after the setfenv() inherit the same environment. If so, then I think I can make nice isolated objects this way.

Of course, I think I might have to do something a bit more complex with __index, if I want this module to be able to access the real global table as well, right?
	I was assuming that module "a" inherits from the global env
so you'll have a chain: _M inherits from a which inherits from _G.

I would have to make __index a function which first checked in 'a' then in _G. Or maybe I just want to go through and copy each member of 'a' (or some subset) into _M. then I could do:

   setmetatable (_M, { __index = _G })
   ... add some or all of the members of 'a' to _M ...

then I would be able to see everything I need, wouldn't I???
	I do think so (if I'm understanding what you mean).
	Another approach will be to add "a" as a local:

local a = a

	Then you'll have access to the table "a" instead of access to
'a table called "a"'.

		Tomas

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Mark Hamburg-4
One problem with having module auto-populate the global namespace is that it
encourages the creation of inefficient and fragile Lua code.

The efficient path:

    local foo = require "foo"

    for k = 1, 10 do
        foo.bar( k )
    end

The inefficient path:

    require "foo"

    for k = 1, 10 do
        foo.bar( k )
    end

The fragile path:

    -- Fortunately, someone else seems to have always required foo.

    for k = 1, 10 do
        foo.bar( k )
    end

And, to end on a positive note, the really efficient path:

    local foo = require "foo"

    local bar = foo.bar

    for k = 1, 10 do
        bar( k )
    end

I don't have a good answer for how to encourage people to use the last form,
but it's a simple change to module to get them to use the first form. If
we're going to worry about Lua efficiency and how well it does on shootouts,
it might be a good idea to try to promote the writing of efficient Lua code.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Tomas-14
	Hi Mark,

I don't have a good answer for how to encourage people to use the last form,
but it's a simple change to module to get them to use the first form.
	Please, could you elaborate on that?  What change on module()
are you proposing?

	Tomas

Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Mark Hamburg-4
on 8/15/05 5:50 AM, Tomas at [hidden email] wrote:

>> I don't have a good answer for how to encourage people to use the last form,
>> but it's a simple change to module to get them to use the first form.
> Please, could you elaborate on that?  What change on module()
> are you proposing?

Don't have it populate the global namespace. It did last time I looked and
the discussion in this thread suggests that it still does. (If that's
changed, then this is all irrelevant and I apologize for consuming
bandwidth. I don't have w6 handy on my e-mail machine at the moment.)

This change "fixes" two things:

1. It prevents using require and then referring to the required entity via
the global instead of a local.

2. It discourages relying on some other piece of code doing the require that
populated the global namespace.

The downside is that it makes command line coding more of a pain. If table
were a module for example, you would have to write:

    require"table".foreach( t, print )

In order to print the contents of t. Rather than:

    table.foreach( t, print )

Assigning into locals at the command-line level is of limited value because
the locals don't last past the end of the chunk. (See some of the
discussions about local v global for related issues.)

One way to deal with this would be to have the environment for the command
line do something like the following:

    __index = function( t, k )
        local v = _G[ k ]
        if v ~= nil then return v end
        v = require( k )
        t[ k ] = v
        return v
    end

This automatically tries require when the name isn't found within the
environment and adds the results to the environment, but it is only intended
for use on the sandbox environment for the command line.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: Packaging and importing

Wim Couwenberg-2
In reply to this post by Diego Nehab-3
> If there is anything you would like to be able to do when you package
> your libraries, but feel you can't do with our proposal, this is the
> right venue to talk about it. Chances are we can figure out how to do it
> so that everything is compatible.

I've dug in to the proposed require implementation since we needed it
for deployment of our app.  I have some remarks (aka questions :-) ).

1.  For developing extension modules in C we apply a scheme where part
of the module is in C (let's say a DLL, but could also be statically
linked) and another part is in an accompanying Lua script.  So we have
basically two files (DLL, Lua) per module.  Under the current proposal
it is not possible to cleanly deploy such a module:

  1.a  The lua part in itself is *not* a module so should not be
       loaded through "require".  (It is a "private" script.)

  1.b  The entry point in the C module that gets called by require
       does not know the path from which it could luaL_loadfile the
       Lua script.

  1.c  I'd rather not put the Lua part inside the DLL (as resource),
       partly because we'll move to Linux with our app but mostly
       because it defies easy scripting during development and adds
       unnecessary steps to the build process.

For example, my recent Fiber module is set up as a single C file,
without accompanying Lua script, because of this deployment issue.  I
see two options for require that would cleanly allow to load such
dichotomous modules:

  1.a  Pass the exact path to the module as a second argument to the
       module entry point (the required name being the first), which
       can then luaL_loadfile the Lua part from the same location.

  1.b  Let each entry point decide for itself which arguments it likes
       to receive.  loader_C (or any other loader) could return a
       closure with appropriate upvalues.  Or a loader could return
       not only the module entry point but also any arguments (to save
       us an auxiliary closure) like so (pseudo code snippet of
       "require"):

           local function call_opt(entry_point, ...)
             if entry_point then
               return true, entry_point(...)
             end
           end

           local done, result
           for _, loader in ipairs(package.loaders) do
             done, result = call_opt(loader(name))
             if done then break end
           end

           
2.  I find it odd that the call <module(modname)> stores its table (if it
didn't exist already) in package.loaded[modname], since this name is not
necessarily "require"-ed anywhere.  Suppose we do <require(name)> and
this leads to a <module(modname)> call.  Then I'd expect that module
would store the table at package.loaded[name] (i.e. the inner-most
required name).  As an example, suppose we develop a package "complex"
but decide to see it as an extension of "math".  Then the complex
package could look something like:

    module "math"

    function complex(re, im)
      -- return a complex number
    end

    I = complex(0, 1)
    
    -- etc. etc.

Under the current implementation, <require "complex"> would simply
return true instead of the math table.  The other scheme *does* need
some rule to enforce that a <module(modname)> call consistently
returns the same table.  We could use the globals (_G) for that or
something like a package.modules table.  (Or, better still, a table
that is private to the "module" function, since "require" is the
public interface to load a package/module.)


3.  I use the opposite order in calling "require" and "module", so
instead of

    local aap = require "aap"
    module "noot"

I use

    module "noot"
    local aap = require "aap"

because it handles possible cyclic dependencies better (aap could
simply be "true" instead of the aap package table in the first snippet
above).  Are there any unexpected (=unwanted) consequences?


4.  (Maybe this should be item 1. :-) ).  For deployment of stand
alone apps, the require philosophy of simply trying a list of loaders
is very simple and flexible, especially since you can control the
specific loaders that will be used.  For example, in a release
deployment we package many C modules into a single DLL simply by
adding a loader that checks for some fixed prefix (say "aap.noot.") in
the required name to a select the appropriate DLL.  Couldn't be much
simpler than that...  :-)  (Of course, we also pass the full package
path to the entry point in this custom built loader, as suggested in
1. above)

HTH (in one way or other :-D ),
Wim



Reply | Threaded
Open this post in threaded view
|

Stripping HTML tags

Florian Berger
In reply to this post by Mark Hamburg-4

Hi.

I thought that stripping HTML tags was easy until I saw something like this:
<a href="http://www.example.com"; alt="> example"> example </a>

My code was:
local s = '<a href="http://www.example.com"; alt="> example"> example </a>'
s = string.gsub(s, '<.->', ' ')
print(string.gsub(s, '<.->', ' '))

-> example"> example


I have seen some examples using PHP and regular expressions. Programming in Lua 20.1 says that Lua cannot do all what POSIX implementation does (http://www.lua.org/pil/20.1.html). Can this be done in Lua? All that come to my mind are captures but I'm not sure if they help at all. Of course my example works in most of the cases, but it would be nice to have it work even better.

f

Reply | Threaded
Open this post in threaded view
|

Re: Stripping HTML tags

Roberto Ierusalimschy
> <a href="http://www.example.com"; alt="> example"> example </a>

Maybe you could preprocess the string, finding all substrings inside
quotes and escaping "dangerous" characters to something else;
something like this (untested):

  s = string.gsub(s, '".-"', function (p)
        return string.gsub(p, "[<>%]", function (c)
                 return string.format("%02x", string.byte(c))
               end)
      end)

-- Roberto

1234