extension for lua bytecode files

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

extension for lua bytecode files

Tony Finch
Is there a standard extension for Lua bytecode files? I've been using .luo
(short for Lua object code).

I'm writing a program that embeds Lua and which will have significant
parts written in Lua. In order to make it easier to deploy I'm embedding
the bytecodes in the executable, which I am doing by a .lua -> .luo -> .c
translation. The C file just declares an array and initializes it with
the contents of the Lua bytecodes so it can be passed to luaL_loadbuffer.
I guess lots of people do something like this?

Tony.
-- 
f.a.n.finch  <[hidden email]>  http://dotat.at/
SOUTH UTSIRE: EASTERLY 4 OR 5, OCCASIONALLY 6. SLIGHT TO MODERATE,
OCCASIONALLY ROUGH. OCCASIONAL RAIN. MODERATE OR GOOD.

Reply | Threaded
Open this post in threaded view
|

Re: extension for lua bytecode files

Thomas Lauer-3
Tony Finch <[hidden email]> wrote:
> Is there a standard extension for Lua bytecode files? I've been using .luo
> (short for Lua object code).

I don't think there's a standard. FWIW, I use .lbc but .luo sounds
nicer.

> I'm writing a program that embeds Lua and which will have significant
> parts written in Lua. In order to make it easier to deploy I'm embedding
> the bytecodes in the executable, which I am doing by a .lua -> .luo -> .c
> translation. The C file just declares an array and initializes it with
> the contents of the Lua bytecodes so it can be passed to luaL_loadbuffer.
> I guess lots of people do something like this?

This works very well. I have a 'shell' along the lines of lua.c which
loads three or four byte-code libraries either from internal byte arrays
or from external .lbc files.

Just watch out for strlen() and friends ... there's a function in lua.c
that calls this before loading the buffer. That's fine if the string is
text but if it's precompiled byte code strlen() just won't cut the
mustard.

-- 
cheers  thomasl

web : http://thomaslauer.com/start


Reply | Threaded
Open this post in threaded view
|

embedded bytecode, was Re: extension for lua bytecode files

Tony Finch
On Sat, 16 Jun 2007, Thomas Lauer wrote:
>
> This works very well. I have a 'shell' along the lines of lua.c which
> loads three or four byte-code libraries either from internal byte arrays
> or from external .lbc files.

I have been using a helper program to make this kind of array, but I would
have thought that it's a common enough requirement to build in to luac:

--- luac.c.orig	Fri Jun  2 17:37:11 2006
+++ luac.c	Sat Jun 16 18:26:12 2007
@@ -30,6 +30,7 @@
 static int dumping=1;			/* dump bytecodes? */
 static int stripping=0;			/* strip debug information? */
 static char Output[]={ OUTPUT };	/* default output file name */
+static const char* Ccode;		/* C wrapper name */
 static const char* output=Output;	/* actual output file name */
 static const char* progname=PROGNAME;	/* actual program name */

@@ -55,6 +56,7 @@
  "usage: %s [options] [filenames].\n"
  "Available options are:\n"
  "  -        process stdin\n"
+ "  -c name  output C wrapper\n"
  "  -l       list\n"
  "  -o name  output to file " LUA_QL("name") " (default is \"%s\")\n"
  "  -p       parse only\n"
@@ -84,6 +86,11 @@
   }
   else if (IS("-"))			/* end of options; use stdin */
    break;
+  else if (IS("-c"))			/* output C wrapper */
+  {
+   Ccode=argv[++i];
+   if (Ccode==NULL || *Ccode==0) usage(LUA_QL("-c") " needs argument");
+  }
   else if (IS("-l"))			/* list */
    ++listing;
   else if (IS("-o"))			/* output file */
@@ -144,6 +151,33 @@
  }
 }

+static void Cprologue(FILE *D)
+{
+ fprintf(D,"/* auto-generated by luac */\n"
+           "void luaload_%s(lua_State *L) {\n"
+           " static const char bytecode[] = {\n  ", Ccode);
+}
+
+static void Cepilogue(FILE *D)
+{
+ fprintf(D,"};\n if(luaL_loadbuffer(L,bytecode,sizeof(bytecode),"
+           "\"%s\"))\n  lua_error(L);\n}\n", Ccode);
+}
+
+static int Cwriter(lua_State* L, const void* p, size_t size, void* u)
+{
+ UNUSED(L);
+ while (size > 0) {
+  if (fprintf((FILE*)u, "%d,", *(unsigned char *)p) < 0)
+   return 1;
+  p++; size--;
+  if (size % 20 == 0)
+   if (fprintf((FILE*)u, "\n  ") < 0)
+    return 1;
+ }
+ return 0;
+}
+
 static int writer(lua_State* L, const void* p, size_t size, void* u)
 {
  UNUSED(L);
@@ -174,9 +208,11 @@
  {
   FILE* D= (output==NULL) ? stdout : fopen(output,"wb");
   if (D==NULL) cannot("open");
+  if (Ccode != NULL) Cprologue(D);
   lua_lock(L);
-  luaU_dump(L,f,writer,D,stripping);
+  luaU_dump(L,f, (Ccode != NULL) ? Cwriter : writer, D,stripping);
   lua_unlock(L);
+  if (Ccode != NULL) Cepilogue(D);
   if (ferror(D)) cannot("write");
   if (fclose(D)) cannot("close");
  }


Tony.
-- 
f.a.n.finch  <[hidden email]>  http://dotat.at/
FAEROES: NORTHEASTERLY 3 OR 4, INCREASING 5 OCCASIONALLY 6 LATER. SLIGHT OR
MODERATE. RAIN OR SHOWERS. MODERATE OR GOOD.

Reply | Threaded
Open this post in threaded view
|

Re: embedded bytecode, was Re: extension for lua bytecode files

Luiz Henrique de Figueiredo
> I have been using a helper program to make this kind of array, but I would
> have thought that it's a common enough requirement to build in to luac:

Under Linux, you can use "xxd -i".

Reply | Threaded
Open this post in threaded view
|

Re: embedded bytecode, was Re: extension for lua bytecode files

Tony Finch
On Sat, 16 Jun 2007, Luiz Henrique de Figueiredo wrote:
>
> Under Linux, you can use "xxd -i".

Thanks for pointing that out. Sadly it doesn't exist on all unixes.

Tony.
-- 
f.a.n.finch  <[hidden email]>  http://dotat.at/
SOUTHEAST ICELAND: EAST OR SOUTHEAST 4 OR 5. SLIGHT OR MODERATE. SHOWERS.
MODERATE OR GOOD.

Reply | Threaded
Open this post in threaded view
|

Re: embedded bytecode, was Re: extension for lua bytecode files

Rici Lake-2

On 16-Jun-07, at 1:44 PM, Tony Finch wrote:

On Sat, 16 Jun 2007, Luiz Henrique de Figueiredo wrote:

Under Linux, you can use "xxd -i".

Thanks for pointing that out. Sadly it doesn't exist on all unixes.


It's part of the vim distribution.


Reply | Threaded
Open this post in threaded view
|

Re: extension for lua bytecode files

Ashwin Hirschi-3
In reply to this post by Tony Finch
> I'm writing a program that embeds Lua and which will have significant
> parts written in Lua. In order to make it easier to deploy I'm embedding
> the bytecodes in the executable, which I am doing by a .lua -> .luo -> .c
> translation. The C file just declares an array and initializes it with
> the contents of the Lua bytecodes so it can be passed to luaL_loadbuffer.
> I guess lots of people do something like this?

Instead of embedding Lua byte code you might consider keeping all your Lua logic separate from your (compiled) C program.

If indeed a significant part of your application is written in Lua and you have a fair number of scripts, you could compile & bundle them in, for instance, an archive or database format [preferably one you can easily access from your C program... ;-)].

My company has been deploying software this way for years. Mostly we use "application packages" based on zip archives or SQLite databases. And this has worked very well for is.

The approach makes a lot of sense, especially if you have a fairly stable Lua-host/runtime and are developing the better part of your solution in Lua.

For such a scenario updates are often mostly enhancements/tweaks to your Lua logic. So, all you need to do to deploy any changes is recompile and send out the repackaged Lua scripts, without the need to rebuild your entire solution and distribute executables (and such).

Anyways, consider it some food for thought on a Sunday afternoon.

Ashwin.
-- 
no signature is a signature.

Reply | Threaded
Open this post in threaded view
|

Re: embedded bytecode, was Re: extension for lua bytecode files

Mark Edgar-2
In reply to this post by Tony Finch
On 6/16/07, Tony Finch <[hidden email]> wrote:
I have been using a helper program to make this kind of array, but I would
have thought that it's a common enough requirement to build in to luac:

There's bin2c.c in Lua 5.0 distributions, and this one for Lua 5.1:
http://lua-users.org/wiki/BinTwoCee

    -Mark

Reply | Threaded
Open this post in threaded view
|

Re: embedded bytecode, was Re: extension for lua bytecode files

Sherry Zhang
On 6/17/07, Mark Edgar <[hidden email]> wrote:
On 6/16/07, Tony Finch <[hidden email]> wrote:
> I have been using a helper program to make this kind of array, but I would
> have thought that it's a common enough requirement to build in to luac:

There's bin2c.c in Lua 5.0 distributions, and this one for Lua 5.1:
http://lua-users.org/wiki/BinTwoCee

    -Mark

well, use the oringinal bin2c, then change lua_dobuffer to
luaL_loadbuffer, then pcall it will solve the problem.

Reply | Threaded
Open this post in threaded view
|

Re: extension for lua bytecode files

Doug Rogers-4
In reply to this post by Thomas Lauer-3
Thomas Lauer wrote:
> I don't think there's a standard. FWIW, I use .lbc but .luo sounds
> nicer.

Long ago I used .lc, but I like Tony's .luo better. If I ever embed byte
codes directly again, I'll use .lua.

Nowadays I put the .lua files on a flash drive and just recompile them
when the code loads. Long ago the byte code was portable so I could
compile using Solaris/sparc and include it in a cross-compile to
LynxOS/PowerPC and also for my x86 applications. But now I just include
the Lua source and recompile it.

In the future I may go with Ashwin Hirschi's method of zipping
everything up in a package.

Doug

-- 
Innovative Concepts, Inc. www.innocon.com 703-893-2007 x220