Part of the build process for Lua MIDI builds an executable and
stitches a .lua file into the executable so that the executable is
standalone. Part of the application is written in Lua but the .lua
files do not need to copied separately; the entire application can be
distributed as a single binary file.
There are plenty of ways of doing this (for example translate
the .lua files into .c files that have the contents in a C string), I
think mostly the reason why I chose this way is that I happened to be
reading about the Mach-O loader whilst I was needing this feature.
Because I use the Mach-O loader interface this technique only works
on OS X; though the general principles will be similar on other
operating systems, most of the work is in the platform-specific
details. Since the work was non-trivial I thought I'd jot down a few
notes in case anyone else wanted to do something similar
The technique works by creating a Mach-O Segment Searcher, MOSS.
MOSS has the following features:
- Include one extra C file, the MOSS implementation, and call one
function to enable MOSS in your application.
- require'foo' will search for foo in the Mach-O executable.
- Embed Lua code in your Mach-O executable at link time by adding a
few options ("just link-and-go").
- External foo.lua file will override internal foo, easing the
editing and development of pure Lua modules whilst the application is
running (no relink required).
Anyone who wants the gory details (and code!) i urged to look at code/
lmoss.h and code/lmoss.c in the recent Lua MIDI release: http://
sourceforge.net/project/showfiles.php?group_id=131184 . I didn't
really think it was worth unbundling.
moss_loader is the function installed in the package.loaders array.
It allows modules to be found in the Mach-O executable. moss_loader
works by grovelling over the structures in the Mach-O executable
header, using the not-very-well-documented structures in <mach-o/
loader.h>. Here's a typical extract of the code (this finds the
command segment called "Lua", it is used when MOSS is initialised):
Later, when require is called, the moss_loader function will search
this segment for a section of the required name.
Conveniently the Mach-O file structure allows for a named segment to
contain several named sections. So each Lua file can go in a section
of its own name (foo.lua goes in section foo, say), and all those
sections go in a single segment called "Lua".
Here's how you include a file in a named segment at link time
(further reading in ld(1)):
Notice the -sectcreate option, "-sectcreate Lua lmidi ../../code/
lmidi.lua" includes the contents of the file "lmidi.lua" in the
section named "lmidi" in the segment named "Lua". MOSS can now find
this Lua file if you use "require'lmidi'".
The "just link-and-go" feature means that in principle an entire
application consisting of Lua and C code can be built with one
invocation of the compiler:
cc -sectcreate Lua foo foo.lua [repeat-for-each-Lua file] *.c
On Mon, 3 Mar 2008, David Jones wrote:
> There are plenty of ways of doing this (for example translate the .lua files
> into .c files that have the contents in a C string), I think mostly the reason
> why I chose this way is that I happened to be reading about the Mach-O loader
> whilst I was needing this feature.
> Because I use the Mach-O loader interface this technique only works on OS X;
> though the general principles will be similar on other operating systems, most
> of the work is in the platform-specific details. Since the work was
> non-trivial I thought I'd jot down a few notes in case anyone else wanted to
> do something similar
Interesting. The easy linking is really nice.
On an ELF system with gcc and GNU binutils, you can turn a (source or
compiled) lua script into a linkable object as follows. The key magic is
the -b option which specifies the input file format, in this case, any old
ld -b binary -r -o MYSCRIPT.o MYSCRIPT.lua
When you link your program you must specify -export-dynamic so that the
binary file's symbols are available to dynamic objects, including the
gcc -export-dynamic -o MYPROG MYPROG.c MYSCRIPT.o
Then MYPROG can find the location of MYSCRIPT in memory using dlsym().
void *start = dlsym(NULL, "_binary_MYSCRIPT_lua_start");
void *end = dlsym(NULL, "_binary_MYSCRIPT_lua_end");
luaL_loadbuffer(L, start, end-start, "MYSCRIPT");
You can also turn MYSCRIPT into a shared object using:
ld -shared -o MYSCRIPT.so MYSCRIPT.o
Then MYPROG can use dlopen() and pass the resulting handle into the
dlsym() calls in the loadbuffer() code, or you can use the LD_PRELOAD
environment variable to link MYPROG and MYSCRIPT together at run-time.
f.a.n.finch <[hidden email]> http://dotat.at/
FISHER: NORTHERLY 5 OR 6, INCREASING 6 TO GALE 8. ROUGH BECOMING VERY ROUGH.
WINTRY SHOWERS. MODERATE OR GOOD.