Lua 5.2 beta: default module path incompatibility

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Lua 5.2 beta: default module path incompatibility

Dirk Laurie
It just took me 30 minutes of debugging time to realize that the reason
my enhancements to mboxparser.lua worked under Lua 5.1.4 but not under
Lua 5.2 was that the current version was not being used at all.  

Reason:

  The default set in luaconf.h for both package.path and package.cpath
  in Lua 5.2.0 beta puts the present working directory last, whereas in
  Lua 5.1.4 it was put first.

The change seems not to be documented under either "Changes since Lua 5.1"
or in the reference manual itself.

It is difficult to understand why this change has been made.  It causes
any application distributed with customized versions of packages that
happen to installed system-wide to break unless the application explicitly
sets the path.

Since the change is undocumented, I sincerely hope it is a bug.

Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.2 beta: default module path incompatibility

liam mail
On 20 September 2011 12:04, Dirk Laurie <[hidden email]> wrote:

> It just took me 30 minutes of debugging time to realize that the reason
> my enhancements to mboxparser.lua worked under Lua 5.1.4 but not under
> Lua 5.2 was that the current version was not being used at all.
>
> Reason:
>
>  The default set in luaconf.h for both package.path and package.cpath
>  in Lua 5.2.0 beta puts the present working directory last, whereas in
>  Lua 5.1.4 it was put first.
>
> The change seems not to be documented under either "Changes since Lua 5.1"
> or in the reference manual itself.
>
> It is difficult to understand why this change has been made.  It causes
> any application distributed with customized versions of packages that
> happen to installed system-wide to break unless the application explicitly
> sets the path.
>
> Since the change is undocumented, I sincerely hope it is a bug.
>
> Dirk
>
>

I would think this is a bug otherwise some of the documentation [1][2]
could be seen as misleading if one _assumes_ the stated path is the
default.

[1] http://www.lua.org/work/doc/manual.html#pdf-package.searchers
[2] http://www.lua.org/work/doc/manual.html#pdf-package.searchpath

Reply | Threaded
Open this post in threaded view
|

pcall fails

Thijs Schreijer
Probably a stupid thingy, but can't seem to get it;

This code (part of a larger object)
20 -- add a filter to my list, no duplicates will be added, flt is
string or table
21 add = function (self, flt)
22 if type(flt) == "string" then
23 flt = self:split(flt)
24 end
25 assert(type(flt) == "table", "cannot add filter, string or
table expected, got " .. type (flt))
26 if not flt.filter then
27 flt.filter = string.concat(flt, ".")
28 end
29 if not self.filters[flt.filter] then -- only add if not in
the list already
30 self.filters[flt.filter] = flt
31 end
32 end,

When called with this test;
133 local s = pcall(filters:add(123))
134 assert( not s, "error expected because of a number")

Gives me this error;
lua: xplfilter.lua:25: cannot add filter, string or table expected, got
number
stack traceback:
        [C]: in function 'assert'
        xplfilter.lua:25: in function 'add'
        xplfilter.lua:133: in main chunk
        [C]: ?

Question is; The error is obvious, but why doesn't the pcall on line 133
catch the error and return true in local s?

Lua for windows, code uses only 1 external require; "loop.simple" for the
object creation.

Any help is appreciated.
Thijs


Reply | Threaded
Open this post in threaded view
|

Re: pcall fails

Robert Raschke

On Tue, Sep 20, 2011 at 1:22 PM, Thijs Schreijer <[hidden email]> wrote:
Probably a stupid thingy, but can't seem to get it;

This code (part of a larger object)
20      -- add a filter to my list, no duplicates will be added, flt is
string or table
21      add = function (self, flt)
22              if type(flt) == "string" then
23                      flt = self:split(flt)
24              end
25              assert(type(flt) == "table", "cannot add filter, string or
table expected, got " .. type (flt))
26              if not flt.filter then
27                      flt.filter = string.concat(flt, ".")
28              end
29              if not self.filters[flt.filter] then -- only add if not in
the list already
30                      self.filters[flt.filter] = flt
31              end
32      end,

When called with this test;
133     local s = pcall(filters:add(123))
134     assert( not s, "error expected because of a number")

Gives me this error;
lua: xplfilter.lua:25: cannot add filter, string or table expected, got
number
stack traceback:
       [C]: in function 'assert'
       xplfilter.lua:25: in function 'add'
       xplfilter.lua:133: in main chunk
       [C]: ?

Question is; The error is obvious, but why doesn't the pcall on line 133
catch the error and return true in local s?

Lua for windows, code uses only 1 external require; "loop.simple" for the
object creation.


You are invoking the function before allowing pcall() to do its thing.

You probably want this:

   local s = pcall(add, filters, 123)

This invokes the function add on the remaining arguments in a protected manner.

Robby

Reply | Threaded
Open this post in threaded view
|

Re: pcall fails

steve donovan
In reply to this post by Thijs Schreijer
On Tue, Sep 20, 2011 at 2:22 PM, Thijs Schreijer
<[hidden email]> wrote:
> 133     local s = pcall(filters:add(123))

Ah, pcall() is passed a _function_, so you need to do this:

local ok, s = pcall(function() return filters:add(123) end)

Here ok will be true or false depending on whether the code threw an
error; s will then be the error, or the result of the function if
successful.

In your case, could also do

local ok, s = pcall(filters.add,filters,123)

since pcall() can take arguments.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: pcall fails

Michal Kottman
In reply to this post by Robert Raschke
On Tue, 2011-09-20 at 13:31 +0100, Robert Raschke wrote:

>        
>         When called with this test;
>         133     local s = pcall(filters:add(123))
>         134     assert( not s, "error expected because of a number")
>
> You are invoking the function before allowing pcall() to do its thing.
> You probably want this:
>
>    local s = pcall(add, filters, 123)
>
> This invokes the function add on the remaining arguments in a
> protected manner.

That would work if 'add' was global or a local variable. Assuming only
'filters' is visible, this should be better:

    local s = pcall(filters.add, filters, 123)

filters:add(123) is sugar for filters.add(filters, 123)



Reply | Threaded
Open this post in threaded view
|

RE: pcall fails

Thijs Schreijer
In reply to this post by Robert Raschke

Thx everyone! How stupid can one be…

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Robert Raschke
Sent: dinsdag 20 september 2011 14:31
To: Lua mailing list
Subject: Re: pcall fails

 

 

On Tue, Sep 20, 2011 at 1:22 PM, Thijs Schreijer <[hidden email]> wrote:

Probably a stupid thingy, but can't seem to get it;

This code (part of a larger object)
20      -- add a filter to my list, no duplicates will be added, flt is
string or table
21      add = function (self, flt)
22              if type(flt) == "string" then
23                      flt = self:split(flt)
24              end
25              assert(type(flt) == "table", "cannot add filter, string or
table expected, got " .. type (flt))
26              if not flt.filter then
27                      flt.filter = string.concat(flt, ".")
28              end
29              if not self.filters[flt.filter] then -- only add if not in
the list already
30                      self.filters[flt.filter] = flt
31              end
32      end,

When called with this test;
133     local s = pcall(filters:add(123))
134     assert( not s, "error expected because of a number")

Gives me this error;
lua: xplfilter.lua:25: cannot add filter, string or table expected, got
number
stack traceback:
       [C]: in function 'assert'
       xplfilter.lua:25: in function 'add'
       xplfilter.lua:133: in main chunk
       [C]: ?

Question is; The error is obvious, but why doesn't the pcall on line 133
catch the error and return true in local s?

Lua for windows, code uses only 1 external require; "loop.simple" for the
object creation.


You are invoking the function before allowing pcall() to do its thing.

You probably want this:

   local s = pcall(add, filters, 123)

This invokes the function add on the remaining arguments in a protected manner.

Robby

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.2 beta: default module path incompatibility

David Manura
In reply to this post by liam mail
On Tue, Sep 20, 2011 at 7:26 AM, liam mail <[hidden email]> wrote:
> On 20 September 2011 12:04, Dirk Laurie <[hidden email]> wrote:
>>  The default set in luaconf.h for both package.path and package.cpath
>>  in Lua 5.2.0 beta puts the present working directory last, whereas in
>>  Lua 5.1.4 it was put first.[...]
>> It is difficult to understand why this change has been made.  It causes
>> any application distributed with customized versions of packages that
>> happen to installed system-wide to break unless the application explicitly
>> sets the path.

Arguments for the new behavior were raised in [1,2].

Note that the path "./?.lua" is relative to the current directory, but
you'd probably rather search relative to the Lua executable (i.e. "!"
style LUA_PATH in Windows / LuaDist setprogdir [3]) or relative to the
current script [4-8].  Perl's FindBin and "use lib" [9] do this well,
and I've contemplated writing the analogue of both modules for Lua,
and submitting it to distributions like LuaForWindows, because the
need is so common.  IMO, "./?.lua" is a hack that introduces some
modes of failure.

[1] http://lua-users.org/lists/lua-l/2009-07/msg00106.html
[2] http://lua-users.org/lists/lua-l/2009-05/msg00559.html

[3] https://github.com/LuaDist/lua/blob/master/src/loadlib_rel.c#L68
[4] http://lua-users.org/lists/lua-l/2011-01/msg00920.html
[5] http://lua-users.org/lists/lua-l/2010-02/msg00683.html
[6] http://lua-users.org/lists/lua-l/2011-02/msg01364.html
[7] http://lua-users.org/lists/lua-l/2006-10/msg00189.html
[8] http://lua-users.org/lists/lua-l/2003-06/msg00399.html

[9] http://search.cpan.org/~jesse/perl/lib/FindBin.pm

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.2 beta: default module path incompatibility

Wesley Smith
Here's what I use with scripts running from the lua command line
interpreter.  This has worked quite well for me so far on OSX.

local
        function exec(cmd, echo)
                echo = echo or true
                if(echo) then
                        print(cmd)
                        print("")
                end
                local res = io.popen(cmd):read("*a")
                return res:sub(1, res:len()-1)
        end
       
        local
        function stripfilename(filename)
                return string.match(filename, "(.+)/[^/]*%.%w+$")
        end
       
        local
        function strippath(filename)
                return string.match(filename, ".+/([^/]*%.%w+)$")
        end
       
        local
        function stripextension(filename)
                local idx = filename:match(".+()%.%w+$")
                if(idx) then
                        return filename:sub(1, idx-1)
                else
                        return filename
                end
        end
       
        function addmodulepath(path)
                -- add to package paths (if not already present)
                if not string.find(package.path, path, 0, true) then
                        package.path = string.format("%s/?.lua;%s", path, package.path)
                        package.path = string.format("%s/?/init.lua;%s", path, package.path)
                        package.cpath = string.format("%s/?.so;%s", path, package.cpath)
                end
        end
       
        local
        function setup_path()
       
                local pwd = exec("pwd")
                local root = arg[0]
                if(root and stripfilename(root)) then
                        root = stripfilename(root) .. "/"
                else
                        root = ""
                end
               
                local script_path
                local path
       
                if(root:sub(1, 1) == "/") then
                        script_path = root
                        path = string.format("%s%s", root, "modules")
                else
                        script_path = string.format("%s/%s", pwd, root)
                        path = string.format("%s/%s%s", pwd, root, "modules")
                end
                return script_path:sub(1, script_path:len()-1)
        end
        ---------------------------------------------------------------
        -- Script Initialization
        script = {}
        script.path = setup_path()

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.2 beta: default module path incompatibility

Dirk Laurie
In reply to this post by David Manura
On Thu, Sep 22, 2011 at 02:39:31AM +0200, David Manura wrote:

> On Tue, Sep 20, 2011 at 7:26 AM, liam mail <[hidden email]> wrote:
> > On 20 September 2011 12:04, Dirk Laurie <[hidden email]> wrote:
> >>  The default set in luaconf.h for both package.path and package.cpath
> >>  in Lua 5.2.0 beta puts the present working directory last, whereas in
> >>  Lua 5.1.4 it was put first.[...]
> >> It is difficult to understand why this change has been made.  It causes
> >> any application distributed with customized versions of packages that
> >> happen to installed system-wide to break unless the application explicitly
> >> sets the path.
>
> Arguments for the new behavior were raised in [1,2].
>
> Note that the path "./?.lua" is relative to the current directory, but
> you'd probably rather search relative to the Lua executable (i.e. "!"
> style LUA_PATH in Windows / LuaDist setprogdir [3]) or relative to the
> current script [4-8].  Perl's FindBin and "use lib" [9] do this well,
> and I've contemplated writing the analogue of both modules for Lua,
> and submitting it to distributions like LuaForWindows, because the
> need is so common.  IMO, "./?.lua" is a hack that introduces some
> modes of failure.
>
> [1] http://lua-users.org/lists/lua-l/2009-07/msg00106.html
> [2] http://lua-users.org/lists/lua-l/2009-05/msg00559.html
>

Maybe an even better solution would thus be to omit the current directory
altogether from the default path.  It's easy enough to concatenate
"./?.lua;" in front or ";./?.lua" at the back of package.path, and then
the code contains an explicit indication that one is doing so.

But '"./?.lua" is a hack' is too harsh.  It is a typical Lua compromise
between security and ease of use, while still offering the user ways
to enhance either at the expense of the other.

Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.2 beta: default module path incompatibility

David Manura
On Thu, Sep 22, 2011 at 1:33 AM, Dirk Laurie <[hidden email]> wrote:
> Maybe an even better solution would thus be to omit the current directory
> altogether from the default path.

That could be more correct.  Some programs like LuaRocks do things
like `local lfs_ok, lfs = pcall(require, "lfs")` to test whether a
module exists and conditionally use it [1].  If luafilesystem is not
installed and there happens to be a file named "lfs.lua" in the
current directory (even if it's intended to map to another module name
like "foo.bar.lfs"), then LuaRocks fails:

  $ touch lfs.lua
  $ luarocks
  /usr/bin/lua: <...>/luarocks/fs/lua.lua:161: attempt to index
upvalue 'lfs' (a boolean value)

[1] https://github.com/keplerproject/luarocks/blob/master/src/luarocks/fs/lua.lua

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.2 beta: default module path incompatibility

liam mail
In reply to this post by David Manura
On 22 September 2011 01:39, David Manura <[hidden email]> wrote:

> On Tue, Sep 20, 2011 at 7:26 AM, liam mail <[hidden email]> wrote:
>> On 20 September 2011 12:04, Dirk Laurie <[hidden email]> wrote:
>>>  The default set in luaconf.h for both package.path and package.cpath
>>>  in Lua 5.2.0 beta puts the present working directory last, whereas in
>>>  Lua 5.1.4 it was put first.[...]
>>> It is difficult to understand why this change has been made.  It causes
>>> any application distributed with customized versions of packages that
>>> happen to installed system-wide to break unless the application explicitly
>>> sets the path.
>
> Arguments for the new behavior were raised in [1,2].
>
> Note that the path "./?.lua" is relative to the current directory, but
> you'd probably rather search relative to the Lua executable (i.e. "!"
> style LUA_PATH in Windows / LuaDist setprogdir [3]) or relative to the
> current script [4-8].  Perl's FindBin and "use lib" [9] do this well,
> and I've contemplated writing the analogue of both modules for Lua,
> and submitting it to distributions like LuaForWindows, because the
> need is so common.  IMO, "./?.lua" is a hack that introduces some
> modes of failure.
>
> [1] http://lua-users.org/lists/lua-l/2009-07/msg00106.html
> [2] http://lua-users.org/lists/lua-l/2009-05/msg00559.html
>
> [3] https://github.com/LuaDist/lua/blob/master/src/loadlib_rel.c#L68
> [4] http://lua-users.org/lists/lua-l/2011-01/msg00920.html
> [5] http://lua-users.org/lists/lua-l/2010-02/msg00683.html
> [6] http://lua-users.org/lists/lua-l/2011-02/msg01364.html
> [7] http://lua-users.org/lists/lua-l/2006-10/msg00189.html
> [8] http://lua-users.org/lists/lua-l/2003-06/msg00399.html
>
> [9] http://search.cpan.org/~jesse/perl/lib/FindBin.pm
>
>

The only argument there seems to be
"That is, above being a security risk, and not in the
control of the script, but the caller...what am I missing? I have to
parse package.path to remove those entries"

To be honest I do not understand this logic. If you are wanting to
create a sandbox then you have to modify Lua anyway and unless you do
so there is nothing I can see from stopping the caller from
reintroducing dot slash to that start of the search paths.

Liam