Lua-users standard libraries project: anyone game?

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Lua-users standard libraries project: anyone game?

Reuben Thomas-5
Following the release of a new standard libraries snapshot, I've rewritten
the Wiki pages

http://lua-users.org/wiki/StandardLibraries

and

http://lua-users.org/wiki/StandardLibraryProposal

to bring things up to date and reflect the work that has been done. To
emphasise this:

   * There are now about 20 modules totalling about 2000 lines of code,
     all of it documented in a luadoc-like fashion (I haven't yet settled
     on a satisfactory documentation tool, though a simple one is
     included)

   * The code is written in a consistent and clear style

   * You can get a recent and fairly stable snapshot from
     http://sourceforge.net/project/showfiles.php?group_id=32250

   * The libraries have been used in production code

   * There's lots of useful stuff from primitives for working with tables
     (e.g. inverting keys and values, making the list of keys or list of
     values), through a simple and powerful prototype-based object
     implementation, to a full-blown getopt module, a shell for writing
     small CLI programs, and a polymorphic implementation of the longest
     common subsequence algorithm---the sort of thing you might use to
     write diff.

I'm looking now for interest from three main classes of people:

1. Users. If the libraries are useless, tell me why. If they're buggy,
tell me what doesn't work. If they don't do what you want, tell me what
you want.

2. System integrators. I'm thinking of the Lua team and LuaCheia people
principally. Are these libraries useful with your distribution? If not,
why not? (I'm still not clear on why no Lua code is distributed with the
core distribution: it can be 100% portable, just like C code, and just as
useful and fundamental as the standard C libraries.)

3. Contributors. It's easy to add code to these libraries. I'm not
insisting on a particular layout or coding style (though that would be
nice); functionality is much more important. In particular, other than by
avoiding namespace clashes, I don't even think it's essential to make sure
that individual modules work well together (though in the long run it's
certainly desirable). I'd love to hear from anyone else who would like to
get involved (at the moment it's just me, John Belmonte and a few others
who've contributed code).

Thanks for listening!

-- 
http://www.mupsych.org/~rrt/ | The only person worth beating is yourself

Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

Luiz Henrique de Figueiredo
>I'm still not clear on why no Lua code is distributed with the
>core distribution: it can be 100% portable, just like C code, and just as
>useful and fundamental as the standard C libraries.

Because adding things to the package means more things to worry about,
things that must be kept up to date and that must work etc.

Following our usual inclinations, we've been trying to reduce the number of
extras, which are the things included in etc/ and test/ in the distribution.
(Now is a good time for people to tell us which of these extras are useful and
which are not and can go.)

Anyway, what Lua code did you have in mind?
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

Reuben Thomas-5
> >I'm still not clear on why no Lua code is distributed with the
> >core distribution: it can be 100% portable, just like C code, and just as
> >useful and fundamental as the standard C libraries.
>
> Because adding things to the package means more things to worry about,
> things that must be kept up to date and that must work etc.

Seems reasonable.

> Anyway, what Lua code did you have in mind?

Perhaps I'm being really unclear. I meant the code in the standard
libraries project, as advertised. I hope that was clear, that was the
whole point of the announcement!

-- 
http://www.mupsych.org/~rrt/ | wisdom, n.  knowing when to be foolish

Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

D Burgess-2
In reply to this post by Reuben Thomas-5
>(Now is a good time for people to tell us which of these extras are useful and
>which are not and can go.)
>--lhf
My useful votes go to everything in /etc

especially

bin2c.c
compat.lua
min.c
noparser.c
trace.c





Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

Reuben Thomas-5
>(Now is a good time for people to tell us which of these extras are useful and
>which are not and can go.)

The stuff in /etc is crucial (and in some cases, like bin2c and
compat.lua, should be more central).

The stuff in test that is for demonstration purposes is less interesting,
but should stay until such time as there's a decent tutorial on lua.org
(and I mean there, not lua-users). The stuff in test that is more
to do with language tricks, such as luac.lua and env.lua is much more
interesting, as it shows tricks that are harder to learn (and, being more
language-specific, harder to find in other places).

-- 
http://www.mupsych.org/~rrt/ | The only person worth beating is yourself

Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

Martin Spernau
In reply to this post by Reuben Thomas-5
Reuben Thomas wrote:

2. System integrators. I'm thinking of the Lua team and LuaCheia people
principally. Are these libraries useful with your distribution? If not,
why not? (I'm still not clear on why no Lua code is distributed with the
core distribution: it can be 100% portable, just like C code, and just as
useful and fundamental as the standard C libraries.)
Hi Reuben!
We have the integration of your standard libs as a very high priority on the ToDo for LuaCheia. Especially we need to explore how the standard libiry code fits into the module-concept of LuaCheia (namespaces etc.) Having high quality extemsions in Lua code is very important for a project like LuaCheia I think.
Currently it's mainly a problem of free time.

-Martin


Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

Andreas Stenius-3
In reply to this post by Reuben Thomas-5
Reuben Thomas wrote:

[snip]

I'm looking now for interest from three main classes of people:

1. Users.

[snip]

2. System integrators.

[snip]

3. Contributors. It's easy to add code to these libraries. I'm not
insisting on a particular layout or coding style (though that would be
nice); functionality is much more important. In particular, other than by
avoiding namespace clashes, I don't even think it's essential to make sure
that individual modules work well together (though in the long run it's
certainly desirable). I'd love to hear from anyone else who would like to
get involved (at the moment it's just me, John Belmonte and a few others
who've contributed code).

I'll be part of a project, where lua will be a major part of the functionality, using alot of libraries to manage different tasks involved. As each module becomes usefull, I would be happy to contribute our achievements to a larger library.


Thanks for listening!


Thanks for the initiative!

//Andreas


Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

Reuben Thomas-5
> I'll be part of a project, where lua will be a major part of the
> functionality, using alot of libraries to manage different tasks
> involved. As each module becomes usefull, I would be happy to contribute
> our achievements to a larger library.

Great. It looks as though the libraries will be going into LuaCheia
(whether you like LuaCheia or not, you'll probably agree it's the best
place for them!) and so it's probably best to get involved through
LuaCheia. I hope to be popping up there soon once I've read up a bit on
LuaCheia's organisation.

-- 
http://www.mupsych.org/~rrt/ | The Next Station Is Oval (Anon)

Reply | Threaded
Open this post in threaded view
|

Bugs.. ? (was: Re: Lua-users standard libraries project: anyone game?)

Andreas Stenius-2
I'm running into some issues with the library code.
So far it is:
o std.bit.lua:12: attempt to index global `bit' (a nil value)
o std.rex.lua:4: bad argument #1 to `setmetatable' (table expected, got nil)

Hmm... begin to see a pattern here.. I miss the coresponding c libs
needed (like bits and rex etc). Maybe the lib should check wheter these
libraries are available or not, before doing their stuff :)

o std.table.lua:122 shouldn't the metamethod be __index?
o std.base.lua:tostring need to check for self referencing tables, or it
goes down an endless circle of tostring calls.

That's what I've found so far.
Oh, and I used the latest sources from cvs (few hours ago), not the zip
you released.

Regards,
Andreas




Reply | Threaded
Open this post in threaded view
|

Re: Bugs.. ? (was: Re: Lua-users standard libraries project: anyone game?)

Reuben Thomas-5
> Hmm... begin to see a pattern here.. I miss the coresponding c libs
> needed (like bits and rex etc). Maybe the lib should check wheter these
> libraries are available or not, before doing their stuff :)

Yes, that's true. I should put in a test, and have now. It's a while since
I thought about it, but I think the reason I didn't do anything was that
it's not entirely obvious what my approach should be if the library isn't
loaded. Should I complain? Silently fail to compile the requisite
functions? Load them anyway (but they won't work unless/until you load the
C part)? And what about extensions to the standard C libraries (table,
string &c.)? What if they're not there?

For the moment, I've made the loading of std.bit and std.rex conditional
on the existence of tables bit and rex *in std.lua*. So if you say
require "std" and you are lacking bit or rex, it won't complain, but if
you say require "std.bit" and you don't have bit, you'll get the error you
just got. That's the simplest solution I can think of that definitely
improves what I have at the moment.

I'd be interested to hear ideas on solving the more general problem. I
guess this is something that has to be solved for integration into
LuaCheia.

> o std.table.lua:122 shouldn't the metamethod be __index?

It should, thanks. I clearly haven't tested this since Lua 4.

> o std.base.lua:tostring need to check for self referencing tables, or it
> goes down an endless circle of tostring calls.

Thanks. I had fun trying to fix this. I have committed the result, which
seems to work for things like

a = {}; a[1]=a; print(a)
a = {{}}; a[1][1]=a; print(a)
a = {}; b = {a}; a[1] = b; print (a)
a = {}; b = {a,a}; print (b)

I don't claim to have got it right first time, though.

-- 
http://www.mupsych.org/~rrt/
motive, n.  a mental wolf in moral wool (Bierce)

Reply | Threaded
Open this post in threaded view
|

Re: Bugs.. ? (was: Re: Lua-users standard libraries project: anyone game?)

Thatcher Ulrich
On Jan 28, 2004 at 09:37 +0100, Reuben Thomas wrote:
> > Hmm... begin to see a pattern here.. I miss the coresponding c libs
> > needed (like bits and rex etc). Maybe the lib should check wheter these
> > libraries are available or not, before doing their stuff :)
> 
> Yes, that's true. I should put in a test, and have now. It's a while since
> I thought about it, but I think the reason I didn't do anything was that
> it's not entirely obvious what my approach should be if the library isn't
> loaded. Should I complain? Silently fail to compile the requisite
> functions? Load them anyway (but they won't work unless/until you load the
> C part)? And what about extensions to the standard C libraries (table,
> string &c.)? What if they're not there?
> 
> For the moment, I've made the loading of std.bit and std.rex conditional
> on the existence of tables bit and rex *in std.lua*. So if you say
> require "std" and you are lacking bit or rex, it won't complain, but if
> you say require "std.bit" and you don't have bit, you'll get the error you
> just got. That's the simplest solution I can think of that definitely
> improves what I have at the moment.
> 
> I'd be interested to hear ideas on solving the more general problem. I
> guess this is something that has to be solved for integration into
> LuaCheia.

What we typically have been doing in luacheia is something simple like
this:

assert(cheia.load("bit"))  -- if it's already loaded, this is a no-op
assert(cheia.load("rex"))

-- ...code using bit and rex here...

Obviously that's luacheia-specific, doesn't do any versioning, and
throws an error if the libraries can't be loaded.  I think throwing an
error is a good default policy; for certain cases it might be
desirable to have a fallback instead, which is straightforward to
check for:

if not cheia.load("bit") then
   -- fallback
end
-- etc

I'm not sure of the right way to do this in a distro-neutral way.  We
could imagine a more generic-sounding loader function, e.g.:

function std.load(module)
	-- distro-specific stuff in here...
	if cheia and cheia.load then return cheia.load(module) end
	-- ... other ways of loading ...

	-- perhaps also check module against a list of statically
	-- linked modules

	return false, "we have no method of loading modules"
end

As far as namespaces go, we have been using a single-level namespace.
Modules are certainly not prohibited from defining their own
sub-namespaces, but cheia.load() and the like don't understand
hierarchies themselves.  E.g.:

cheia.load("std")	 -- OK
cheia.load("std.bit")	 -- not understood in a meaningful way

We originally had a configurable, hierarchical organization, but it
created all kinds of headaches, and seemed to offer little gains.  One
of the bigger headache is that luaL_openlib() expects a global table
name, so supporting sub-namespaces in other people's libraries
involved grafting in a bunch of external code.

So the policy is that module tables live in the global table.

-- 
Thatcher Ulrich
http://tulrich.com

Reply | Threaded
Open this post in threaded view
|

RE: Bugs.. ? (was: Re: Lua-users standard libraries project: anyone game?)

Virgil Smith
In reply to this post by Reuben Thomas-5
> > o std.base.lua:tostring need to check for
> > self referencing tables, or it goes down
> > an endless circle of tostring calls.
>
> Thanks. I had fun trying to fix this.
> I have committed the result, which
> seems to work for things like
>
> a = {}; a[1]=a; print(a)
> a = {{}}; a[1][1]=a; print(a)
> a = {}; b = {a}; a[1] = b; print (a)
> a = {}; b = {a,a}; print (b)

I've hit the cyclical reference issue before (in a utility for saving "data"
from scripts).  What I came up with was making a temporary table to hold
references to any reference based objects (i.e. tables, userdata, functions,
and threads).  In this table the references are the keys and I used an
incremented integer for assigning values.  Thus when you come to a table (or
other object) all you need to do is check for it in the table.

In my "data" file I'd then just record the integer ID "reference" in place
of the whole object.  I'm not sure what you would present best in "print"
function, but at least you can avoid the endless recursion.

Ain't it great that you can key off of <anything> in Lua tables. :-)



Reply | Threaded
Open this post in threaded view
|

RE: Bugs.. ? (was: Re: Lua-users standard libraries project: anyone game?)

Reuben Thomas-5
> I've hit the cyclical reference issue before (in a utility for saving "data"
> from scripts).  What I came up with was making a temporary table to hold
> references to any reference based objects (i.e. tables, userdata, functions,
> and threads).

I and Andreas seemed to do the same thing.

> In my "data" file I'd then just record the integer ID "reference" in place
> of the whole object.  I'm not sure what you would present best in "print"
> function, but at least you can avoid the endless recursion.

I used the table address as returned by the original tostring. Handy for
debugging and guaranteed to be unique, so would work for serialisation as
well.

> Ain't it great that you can key off of <anything> in Lua tables. :-)

Yup.

-- 
http://www.mupsych.org/~rrt/
Quidquid latine dictum sit, altum viditur (Anon)

Reply | Threaded
Open this post in threaded view
|

Re: Bugs.. ? (was: Re: Lua-users standard libraries project: anyone game?)

Reuben Thomas-5
In reply to this post by Thatcher Ulrich
> cheia.load("std")	 -- OK
> cheia.load("std.bit")	 -- not understood in a meaningful way

Actually, I have no hierarchy, only convention.

require "foo.bar.baz"

just looks for file "foo.bar.baz" on LUA_PATH.

> One of the bigger headache is that luaL_openlib() expects a global table
> name, so supporting sub-namespaces in other people's libraries involved
> grafting in a bunch of external code.

But this isn't a problem for Lua libraries, only C libraries, and in a
large library you might well want a hierarchical namespace (but I agree
that hierarchies are more popular with designers than useful to
programmers).

-- 
http://www.mupsych.org/~rrt/ | robber, n.  a candid man of affairs (Bierce)

Reply | Threaded
Open this post in threaded view
|

Re: Bugs.. ? (was: Re: Lua-users standard libraries project: anyone game?)

Thatcher Ulrich
On Jan 28, 2004 at 10:36 +0100, Reuben Thomas wrote:
> > cheia.load("std")	 -- OK
> > cheia.load("std.bit")	 -- not understood in a meaningful way
> 
> Actually, I have no hierarchy, only convention.
> 
> require "foo.bar.baz"
> 
> just looks for file "foo.bar.baz" on LUA_PATH.

Hm, luacheia might be able to just piggyback on require().  Needs some
looking into...

-- 
Thatcher Ulrich
http://tulrich.com

Reply | Threaded
Open this post in threaded view
|

Re: Bugs.. ?

Andreas Stenius-2
In reply to this post by Thatcher Ulrich
Thatcher Ulrich wrote:
[snip]

As far as namespaces go, we have been using a single-level namespace.
Modules are certainly not prohibited from defining their own
sub-namespaces, but cheia.load() and the like don't understand
hierarchies themselves.  E.g.:

cheia.load("std")	 -- OK
cheia.load("std.bit")	 -- not understood in a meaningful way

[snip]

When loading lua files (e.g. with require) you could take the approach mentioned in Roberto's book: to have a specific file as the last entry in LUA_PATH, so all required modules not found will run this named file, which in turn could inspect the global _REQUIREDNAME to do the actual loading. I don't know about your previous headaches vs. gains, but this wouls work for dyn libs as well, taken that there is a table defined that maps the lib with a name and what else is needed to require it.

I'm not sure if what I just said is understandable, so I provide a small example: (note: I have no prior experience with loadlib, so I'm just tossing about to make my point somewhat clear, if possible)

LUA_PATH="?;?.lua;/usr/local/lua/cust_require.lua" -- example lua path

-- lib_defs.lua
libraries = {
 { "name" = "rex", "libname" = "/.../lib_rex.so", "func" = "open_rex" },
{ "name" = "std.foo.bit", "libname" = "/.../std/foo/bit.so", "func" = "open_bit" }
}
-- EOF lib_defs

-- cust_require.lua
-- check for library
for lib in ipairs( libraries ) do
	if lib.name == _REQUIREDNAME then
		-- load library with some func
		return loadlib( lib.libname, lib.func )
	end
end

-- not a dyn lib, check for lua file
-- some searching in ordinary ways..

-- EOF cust_require


Only make sure that the libraries table is defined, and the cust_require is in LUA_PATH. Not sure what headaches comes with this, and might not be worth it, or even something you've already tried.. I've had fun writing it (not tested) any how ;)

Comments?

//Andreas



Reply | Threaded
Open this post in threaded view
|

Re: Lua-users standard libraries project: anyone game?

Steve Elkins
In reply to this post by Luiz Henrique de Figueiredo
On Tue, 27 Jan 2004 15:17:38 -0200
Luiz Henrique de Figueiredo <[hidden email]> wrote:

> (Now is a good time for people to tell us which of these extras are
> useful and which are not and can go.)

I realize that "now" was long ago.  ;) I tweak test/table.lua to make
variables_like_this behave when I use it as directed in globals.lua

   luac -p -l file.lua | lua globals.lua | sort | lua table.lua

I have profited from almost all of the code in etc/ and test/ earlier
and still refer to some of it now and then.

Thanks,
Steve

--- table.lua.ORIG      Wed Aug  7 10:16:42 2002
+++ table.lua   Tue Feb  3 11:49:26 2004
@@ -5,7 +5,7 @@
 while 1 do
  local l=io.read()
  if l==nil then break end
- local _,_,a,b=string.find(l,'"?(%w+)"?%s*(.*)$')
+ local _,_,a,b=string.find(l,'"?([_%w]+)"?%s*(.*)$')
  if a~=A then A=a io.write("\n",a,":") end
  io.write(" ",b)
 end

Reply | Threaded
Open this post in threaded view
|

Calling a chunk by file name

jose marin2
Hello.

How do I call a compiled chunk by the original file
name?

For example, I have the files:

Enemy1.lua
  some code..
  

Enemy2.lua
  some code..


etc...


I compile the files:

luac -o enemies.dat Enemy1.lua Enemy2.lua ...

Then, in C or Lua code, I would like to call the code
in some archive, for example, Enemy1.lua.

I wouldn't like to create a function for each enemy,
like this:

function Enemy1()
  some code..
end
  

function Enemy2()
  some code..
end


then call Enemy1().




______________________________________________________________________

Yahoo! GeoCities: 15MB de espaço grátis para criar seu web site!
http://br.geocities.yahoo.com/

Reply | Threaded
Open this post in threaded view
|

RE: Calling a chunk by file name

Kevin Baca-2
This is untested code:

----- code -----

local chunkCache = {}

function callChunk( name )
    local chunk = chunkCache[ name ]

    if( not chunk ) then
        local msg

        chunk, msg = loadfile( name )

        if( msg ) then error( msg ) end

        chunkCache[ name ] = chunk
    end

    local success

    success, msg = pcall( chunk )

    if( not success ) then error( msg ) end
end

----- end code -----

> -----Original Message-----
> From: [hidden email] 
> [[hidden email]] On Behalf Of Jose Marin
> Sent: Thursday, February 05, 2004 4:34 AM
> To: Lua list
> Subject: Calling a chunk by file name
> 
> 
> Hello.
> 
> How do I call a compiled chunk by the original file
> name?
> 
> For example, I have the files:
> 
> Enemy1.lua
>   some code..
>   
> 
> Enemy2.lua
>   some code..
> 
> 
> etc...
> 
> 
> I compile the files:
> 
> luac -o enemies.dat Enemy1.lua Enemy2.lua ...
> 
> Then, in C or Lua code, I would like to call the code
> in some archive, for example, Enemy1.lua.
> 
> I wouldn't like to create a function for each enemy,
> like this:
> 
> function Enemy1()
>   some code..
> end
>   
> 
> function Enemy2()
>   some code..
> end
> 
> 
> then call Enemy1().
> 
> 
> 
> 
> ______________________________________________________________________
> 
> Yahoo! GeoCities: 15MB de espaço grátis para criar seu web 
> site! http://br.geocities.yahoo.com/
> 


Reply | Threaded
Open this post in threaded view
|

RE: Calling a chunk by file name

jose marin2
Good idea, but what I trying to do is to run a code 
in a compiled file.

this file i's build like this

luac -o files.dat prog1.lua prog2.lua prog3.lua

that generates a file "files.dat".

What I'm trying to do is to call, say, "prog2.lua",
that it's in "files.dat". 

It's that possible?

______________________________________________________________________

Yahoo! GeoCities: 15MB de espaço grátis para criar seu web site!
http://br.geocities.yahoo.com/

12