Mac OS X dynamic loading and Lua5.0.2

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Mac OS X dynamic loading and Lua5.0.2

Peter Colson
Just want to check since I've seen plenty of references to dynamic loading and Max OS X on the list whether this is a simple 'loadlib(path/lib.dylib, initfunc)' process or is there still some trickery in getting Lua to dynamically load OS X libs.

Are there particular options I need to be aware of when building the dynamic library to allow it to load and expose it's init function (I saw something about prepending function names with '_c' or similar).

So far I keep getting a Nil return from loadlib.


Regards,
Peter Colson.


Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Asko Kauppi-3

If you just need Lua and extension modules, I'd strongly recommend running LuaX or Luacheia on the Mac. They provide a framework for building your modules, and take care of details like that.

Or.. at least you could have a look at their code, to see how the things have been done.

o.t.o.h:

Shouldn't there be a "sample" dynamic module included with Lua core tgz now. I mean, if loadlib is supported, why not make a small and simple "start here" sample (that would compile on several architectures) to show how it's used. I think this shold go in the 'etc' directory, or somewhere. (no, wiki is no good for these, it needs to have the 'official' status stamped on it!) :)

-ak

22.3.2005 kello 06:18, Peter Colson kirjoitti:

Just want to check since I've seen plenty of references to dynamic loading and Max OS X on the list whether this is a simple 'loadlib(path/lib.dylib, initfunc)' process or is there still some trickery in getting Lua to dynamically load OS X libs.

Are there particular options I need to be aware of when building the dynamic library to allow it to load and expose it's init function (I saw something about prepending function names with '_c' or similar).

So far I keep getting a Nil return from loadlib.


Regards,
Peter Colson.



Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Petite Abeille
In reply to this post by Peter Colson

On Mar 22, 2005, at 05:18, Peter Colson wrote:

Are there particular options I need to be aware of when building the dynamic library to allow it to load and expose it's init function (I saw something about prepending function names with '_c' or similar).

Mike Pall provided the following incantation to compile modules on Mac OS X:

local CompileCommand = "gcc -O2 -fno-common -c -o"
local LinkCommand = "gcc -bundle -undefined dynamic_lookup -o"

Works like a charm.

Here is a little Lua script I use instead of make to compile stuff on Mac OS X:

http://dev.alt.textdrive.com/file/lu/lub.lua

The script expects to find a "lubfile" in the directory you are compiling. The "lubfile" is a Lua table with two keys:

"module.name" = the name of the module to build
"object.names" = the source files to compile and link

Attached is an example "lubfile" to compile zlib with its lzlib bindings.

Attachment: lubfile
Description: Binary data




Cheers

--
PA, Onnay Equitursay
http://alt.textdrive.com/

Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Mike Pall-52
Hi,

PA wrote:
> Mike Pall provided the following incantation to compile modules on Mac 
> OS X:
> 
> local CompileCommand = "gcc -O2 -fno-common -c -o"
> local LinkCommand = "gcc -bundle -undefined dynamic_lookup -o"
> 
> Works like a charm.

Yes, but on Lua 5.1 only!

The original posting asked about Lua 5.0.2. But this one doesn't have any
support for Mac OS X. You need to patch it. Look into src/lib/loadlib.c
at the bottom. However this patch requires you to compile your modules
as dylibs (and not as bundles).

Peter Colson wrote:
> So far I keep getting a Nil return from loadlib.

The second return value should tell you what went wrong. Try:

lua -e 'print(loadlib("xyz.so", "_luaopen_xyz"))'

For Lua 5.0.2 "xyz.so" has to be in the current dir.

With Lua 5.1 you should use require("xyz") and put the module into the
proper path ( /usr/local/lib/lua/5.1/xyz.so ).

Asko Kauppi wrote:
> Shouldn't there be a "sample" dynamic module included with Lua core tgz 
> now.

Yes, a hello world would suffice. But the Makefile that needs to
accompany it is the real problem. There are about a dozen common
variations for creating a shared library, depending on the OS and
the toolchain.

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Peter Colson
On 22/03/2005, at 9:37 PM, Mike Pall wrote:

Hi,

PA wrote:
Mike Pall provided the following incantation to compile modules on Mac
OS X:

local CompileCommand = "gcc -O2 -fno-common -c -o"
local LinkCommand = "gcc -bundle -undefined dynamic_lookup -o"

Works like a charm.

Yes, but on Lua 5.1 only!

The original posting asked about Lua 5.0.2. But this one doesn't have any
support for Mac OS X. You need to patch it. Look into src/lib/loadlib.c
at the bottom. However this patch requires you to compile your modules
as dylibs (and not as bundles).

I'm quite happy to go to 5.1 since it offers better OS X support. I'm using Lua under OS X more as a quick test/prototype environ for another platform.

Tried making 5.1 (work5) and this line:

gcc -o lua  lua.o -L. -llua -llualib -lm -ldl -Wl,-E

fails with OS X ld not supporting the '-E' flag. Any hints as to what might work? I'll keep looking at this end, but I'm also learning the ropes with OS X build procedures as well.

Peter Colson wrote:
So far I keep getting a Nil return from loadlib.

The second return value should tell you what went wrong. Try:

lua -e 'print(loadlib("xyz.so", "_luaopen_xyz"))'

For Lua 5.0.2 "xyz.so" has to be in the current dir.

Tried this under 5.0.2 and it said 'loadlib not supported'. That makes sense. Then tried it on what I think was an old version of my test dylib using Lua5.1 libs that had been built without the ld '-E' param and it actually got into the luaopen_test (once I added '_' to the func name) before failing with a segmentation fault. Rebuilding my test module and linking to the new Lua5.1 libs produced heaps of undefined symbols which I think is due to removing '-E' and not adding the correct OS X options. Also had to re-ranlib the Lua5.1 libs according to another message that appeared.


With Lua 5.1 you should use require("xyz") and put the module into the
proper path ( /usr/local/lib/lua/5.1/xyz.so ).

Under OS X is the extension typically ',dylib' or '.so' or doesn't it really matter?


Regards,
Peter Colson.


Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Peter Colson
In reply to this post by Petite Abeille

On 22/03/2005, at 8:56 PM, PA wrote:


On Mar 22, 2005, at 05:18, Peter Colson wrote:

Are there particular options I need to be aware of when building the dynamic library to allow it to load and expose it's init function (I saw something about prepending function names with '_c' or similar).

Mike Pall provided the following incantation to compile modules on Mac OS X:

local CompileCommand = "gcc -O2 -fno-common -c -o"
local LinkCommand = "gcc -bundle -undefined dynamic_lookup -o"

Works like a charm.

Here is a little Lua script I use instead of make to compile stuff on Mac OS X:

http://dev.alt.textdrive.com/file/lu/lub.lua

The script expects to find a "lubfile" in the directory you are compiling. The "lubfile" is a Lua table with two keys:

"module.name" = the name of the module to build
"object.names" = the source files to compile and link

Attached is an example "lubfile" to compile zlib with its lzlib bindings.

<lubfile>


Thanks for the script. I'll try it again once I get my 5.1 build problems sorted. Did note that it gave this error, which I also see when running your commands above directly:

(info) compile: 'gcc -O2 -fno-common -c -o testlua.o testlua.c'
(info) link: 'gcc -bundle -undefined dynamic_lookup -o testlua.so testlua.o' ld: flag: -undefined dynamic_lookup can't be used with MACOSX_DEPLOYMENT_TARGET environment variable set to: 10.1

I'll look at it some more here, but if it's obvious what's wrong let me know.


Regards,
Peter Colson.


Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Mike Pall-52
In reply to this post by Peter Colson
Hi,

Peter Colson wrote:
> gcc -o lua  lua.o -L. -llua -llualib -lm -ldl -Wl,-E
> 
> fails with OS X ld not supporting the '-E' flag.

Just comment out the whole "DL= ..." line in src/Makefile for OSX.

Since Lua 5.1 uses the native dynamic library functions, -ldl is
not required. Symbols are exported by default on OSX. But do not
use 'strip' on the executable ('strip -x' is ok, though).

> >With Lua 5.1 you should use require("xyz") and put the module into the
> >proper path ( /usr/local/lib/lua/5.1/xyz.so ).
> 
> Under OS X is the extension typically ',dylib' or '.so' or doesn't it 
> really matter?

It doesn't matter and Lua uses '.so' for consistency with other
POSIX machines (other ported apps do that, too).

> ld: flag: -undefined dynamic_lookup can't be used with 
> MACOSX_DEPLOYMENT_TARGET environment variable set to: 10.1

MACOSX_DEPLOYMENT_TARGET="10.3"
export MACOSX_DEPLOYMENT_TARGET
gcc -O2 -fno-common -c -o module.o module.c
gcc -bundle -undefined dynamic_lookup -o module.so module.o

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Peter Colson
Fantastic! It's all working now. I'll commit these details to memory...

Thanks for your help (and others too). This is a good list for getting prompt, useful replies.

Regards,
Peter Colson.

On 23/03/2005, at 1:38 PM, Mike Pall wrote:

Hi,

Peter Colson wrote:
gcc -o lua  lua.o -L. -llua -llualib -lm -ldl -Wl,-E

fails with OS X ld not supporting the '-E' flag.

Just comment out the whole "DL= ..." line in src/Makefile for OSX.

Since Lua 5.1 uses the native dynamic library functions, -ldl is
not required. Symbols are exported by default on OSX. But do not
use 'strip' on the executable ('strip -x' is ok, though).

With Lua 5.1 you should use require("xyz") and put the module into the
proper path ( /usr/local/lib/lua/5.1/xyz.so ).

Under OS X is the extension typically ',dylib' or '.so' or doesn't it
really matter?

It doesn't matter and Lua uses '.so' for consistency with other
POSIX machines (other ported apps do that, too).

ld: flag: -undefined dynamic_lookup can't be used with
MACOSX_DEPLOYMENT_TARGET environment variable set to: 10.1

MACOSX_DEPLOYMENT_TARGET="10.3"
export MACOSX_DEPLOYMENT_TARGET
gcc -O2 -fno-common -c -o module.o module.c
gcc -bundle -undefined dynamic_lookup -o module.so module.o

Bye,
     Mike




Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Luiz Henrique de Figueiredo
In reply to this post by Mike Pall-52
> Just comment out the whole "DL= ..." line in src/Makefile for OSX.

Or just do "make DL=" at the top level...

Does that warrant a top-level "macosx" target just for that? Or perhaps
a note in INSTALL suffices?
--lhf

Reply | Threaded
Open this post in threaded view
|

Lua Makefile (Re: Mac OS X dynamic loading and Lua5.0.2)

Asko Kauppi-3

Or perhaps conditional operation to adopt to Linux, OS X etc. specially.

I think it'd be less nuisance for everyone to have the most common platforms supported directly. Then, if someone has problems with the conditionals, they could simply remove them. Now, every non-Linux user is expected to edit.

...
# If conditionals cause you a problem, simply remove the block below
ifneq "$(windir)" ""
  MYLIBS=-lm
else
 ifeq "$(shell uname)" "Linux"
  MYLDFLAGS= -Wl,-E		# enable dynamic loading in Linux
  MYLIBS= -lm -ldl		# enable dynamic loading in Linux
 endif
 ifeq "$(shell uname)" "Darwin"
  MYLIBS=
 endif
 #..
endif


23.3.2005 kello 05:22, Luiz Henrique de Figueiredo kirjoitti:

 Just comment out the whole "DL= ..." line in src/Makefile for OSX.

Or just do "make DL=" at the top level...

Does that warrant a top-level "macosx" target just for that? Or perhaps
a note in INSTALL suffices?
--lhf



Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X dynamic loading and Lua5.0.2

Mike Pall-52
In reply to this post by Luiz Henrique de Figueiredo
Hi,

Luiz Henrique de Figueiredo wrote:
> Or just do "make DL=" at the top level...
> 
> Does that warrant a top-level "macosx" target just for that? Or perhaps
> a note in INSTALL suffices?

Some BSD-based systems do not need -ldl, but need -Wl,-E.
Others use gcc, but not the GNU linker and need other options to
export all symbols. Others provide both their own compiler and gcc.


Maybe it would suffice to add targets for a few of the most common
OS/toolchain combinations. This is obviously highly subjective,
but my list would be:
  Linux, FreeBSD, NetBSD, OpenBSD, Mac OS X, Solaris
Maybe also:
  HP-UX, AIX
(but you can guess where they are heading).

[Non-POSIX operating systems and cross-compilers do not work well
with the provided Makefiles, anyway.]

These differ mostly in CFLAGS, LDFLAGS and DL. I can provide these
settings for all of them (except for AIX and OpenBSD).

Maybe the top-level Makefile should print a list of the possible targets
by default and then require you to type (say) "make linux".


Some systems may need changes to luaconf.h, too. E.g. setjmp vs. _setjmp 
makes a huge performance difference on *BSD and Mac OS X when you are
using coroutines.

Bye,
     Mike