string.dump strip option (5.2 request)

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

string.dump strip option (5.2 request)

Daan Nusman
Hi,

1) I was thinking, for 5.2, why not add a boolean option string.dump so that to string.dump(f, true) produces bytecode without debug information.

Right now it is impossible to create stripped bytecode from within a normally compiled Lua interpreter (not counting luac of course), not even with a custom-made C function, because lua_dump also does not expose a "strip" parameter, so

2) let's change lua_dump to include the strip parameter too :)

--
Kind regards,

Daan Nusman, M.Sc.
re-lion
+31 53 4356687




Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Thomas Harning Jr.
Daan Nusman wrote:
Hi,

1) I was thinking, for 5.2, why not add a boolean option string.dump so that to string.dump(f, true) produces bytecode without debug information.

Right now it is impossible to create stripped bytecode from within a normally compiled Lua interpreter (not counting luac of course), not even with a custom-made C function, because lua_dump also does not expose a "strip" parameter, so

2) let's change lua_dump to include the strip parameter too :)

Simple patch and equally simply tested...
> function test()
>> error("BAD")
>> end
> x1 = string.dump(test)
> =#x1
123
> x2 = string.dump(test,true)
> =#x2
100
> loadstring(x1)()
stdin:2: BAD
stack traceback:
   [C]: in function 'error'
   stdin:2: in function <stdin:1>
   stdin:1: in main chunk
   [C]: ?
> loadstring(x2)()
BAD
stack traceback:
   [C]: in function 'error'
   ?: in function <?:1>
   stdin:1: in main chunk
   [C]: ?

diff -u -ru lua-5.1.3/src/lapi.c lua-5.1.3-dumpstrip/src/lapi.c
--- lua-5.1.3/src/lapi.c	2008-01-28 10:12:52.000000000 -0500
+++ lua-5.1.3-dumpstrip/src/lapi.c	2008-02-26 11:06:29.000000000 -0500
@@ -873,14 +873,14 @@
 }
 
 
-LUA_API int lua_dump (lua_State *L, lua_Writer writer, void *data) {
+LUA_API int lua_dump (lua_State *L, lua_Writer writer, void *data, int strip) {
   int status;
   TValue *o;
   lua_lock(L);
   api_checknelems(L, 1);
   o = L->top - 1;
   if (isLfunction(o))
-    status = luaU_dump(L, clvalue(o)->l.p, writer, data, 0);
+    status = luaU_dump(L, clvalue(o)->l.p, writer, data, strip);
   else
     status = 1;
   lua_unlock(L);
diff -u -ru lua-5.1.3/src/lstrlib.c lua-5.1.3-dumpstrip/src/lstrlib.c
--- lua-5.1.3/src/lstrlib.c	2007-12-28 10:32:23.000000000 -0500
+++ lua-5.1.3-dumpstrip/src/lstrlib.c	2008-02-26 11:08:43.000000000 -0500
@@ -145,10 +145,12 @@
 
 static int str_dump (lua_State *L) {
   luaL_Buffer b;
+  int strip = 0;
   luaL_checktype(L, 1, LUA_TFUNCTION);
+  strip = lua_toboolean(L, 2);
   lua_settop(L, 1);
   luaL_buffinit(L,&b);
-  if (lua_dump(L, writer, &b) != 0)
+  if (lua_dump(L, writer, &b, strip) != 0)
     luaL_error(L, "unable to dump given function");
   luaL_pushresult(&b);
   return 1;
diff -u -ru lua-5.1.3/src/lua.h lua-5.1.3-dumpstrip/src/lua.h
--- lua-5.1.3/src/lua.h	2008-01-28 10:12:52.000000000 -0500
+++ lua-5.1.3-dumpstrip/src/lua.h	2008-02-26 11:08:55.000000000 -0500
@@ -204,7 +204,7 @@
 LUA_API int   (lua_load) (lua_State *L, lua_Reader reader, void *dt,
                                         const char *chunkname);
 
-LUA_API int (lua_dump) (lua_State *L, lua_Writer writer, void *data);
+LUA_API int (lua_dump) (lua_State *L, lua_Writer writer, void *data, int strip);
 
 
 /*
Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Ignacio Burgueño
In reply to this post by Daan Nusman
Daan Nusman wrote:
Hi,

1) I was thinking, for 5.2, why not add a boolean option string.dump so that to string.dump(f, true) produces bytecode without debug information.

Right now it is impossible to create stripped bytecode from within a normally compiled Lua interpreter (not counting luac of course), not even with a custom-made C function, because lua_dump also does not expose a "strip" parameter, so

2) let's change lua_dump to include the strip parameter too :)


This was requested on 2006. Don't know if the discussion arrived to any conclusion.

http://lua-users.org/lists/lua-l/2006-10/msg00013.html

Regards,
Ignacio Burgueño



Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Luiz Henrique de Figueiredo
> This was requested on 2006. Don't know if the discussion arrived to any 
> conclusion.
> 
> http://lua-users.org/lists/lua-l/2006-10/msg00013.html

I think the conclusion was that it would complicate the API.

Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Ignacio Burgueño
Luiz Henrique de Figueiredo wrote:
This was requested on 2006. Don't know if the discussion arrived to any conclusion.

http://lua-users.org/lists/lua-l/2006-10/msg00013.html

I think the conclusion was that it would complicate the API.



Good. Just a quick question (haven't read the above thread thoroughly myself). Is possible to strip the debug information off the generated bytecode? Like

withDbgInfo = string.dump(aFunction)
noDbgInfo = strip_debuf_info(withDbgInfo)

just by manipulating the byte stream?

Regards,
Ignacio Burgueño



Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Luiz Henrique de Figueiredo
> Good. Just a quick question (haven't read the above thread thoroughly 
> myself). Is possible to strip the debug information off the generated 
> bytecode?

There's C code and instructions for that in
	http://lua-users.org/lists/lua-l/2006-10/msg00011.html

> withDbgInfo = string.dump(aFunction)
> noDbgInfo = strip_debuf_info(withDbgInfo)
> 
> just by manipulating the byte stream?

You could do it in Lua but you'd have to parse the bytecode stream. Not fun.
On the other hand, in the current bytecode format, the debug info is the last
section dumped, and so if aFunction does not include other functions, it's
a matter of removing the tail of the bytecode string. The problem, of course,
is finding out where that tail starts...
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

D Burgess-4
In reply to this post by Luiz Henrique de Figueiredo
Can we levaluate that  again? I think it is a valuable feature.

Davud B

On Wed, Feb 27, 2008 at 4:40 AM, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
> > This was requested on 2006. Don't know if the discussion arrived to any
>  > conclusion.
>  >
>  > http://lua-users.org/lists/lua-l/2006-10/msg00013.html
>
>  I think the conclusion was that it would complicate the API.
>

Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Peter Cawley
Strip in Lua:

-- Strips debug information from a dumped chunk
function strip(dump)
  local version, format, endian, int, size, ins, num = dump:byte(5, 11)
  local subint
  if endian == 1 then
    subint = function(dump, i, l)
      local val = 0
      for n = l, 1, -1 do
          val = val * 256 + dump:byte(i + n - 1)
      end
      return val, i + l
    end
  else
    subint = function(dump, i, l)
            local val = 0
            for n = 1, l, 1 do
                val = val * 256 + dump:byte(i + n - 1)
            end
            return val, i + l
        end
    end
	local strip_function
  strip_function = function(dump)
    local count, offset = subint(dump, 1, size)
    local stripped, dirty = string.rep("\0", size), offset + count
    offset = offset + count + int * 2 + 4
    offset = offset + int + subint(dump, offset, int) * ins
    count, offset = subint(dump, offset, int)
    for n = 1, count do
      local t
      t, offset = subint(dump, offset, 1)
      if t == 1 then
        offset = offset + 1
      elseif t == 4 then
        offset = offset + size + subint(dump, offset, size)
      elseif t == 3 then
        offset = offset + num
      end
    end
    count, offset = subint(dump, offset, int)
    stripped = stripped .. dump:sub(dirty, offset - 1)
    for n = 1, count do
      local proto, off = strip_function(dump:sub(offset, -1))
      stripped, offset = stripped .. proto, offset + off - 1
    end
    offset = offset + subint(dump, offset, int) * int + int
    count, offset = subint(dump, offset, int)
    for n = 1, count do
      offset = offset + subint(dump, offset, size) + size + int * 2
    end
    count, offset = subint(dump, offset, int)
    for n = 1, count do
      offset = offset + subint(dump, offset, size) + size
    end
    stripped = stripped .. string.rep("\0", int * 3)
    return stripped, offset
  end
	return dump:sub(1,12) .. strip_function(dump:sub(13,-1))
end

-- Test function
local function test()
  local alpha, beta = "world", "hello"
  alpha, beta = beta, alpha
  local table, mod = {1,2,3,4,5,6,7}, 3
  _G.table.sort(table, function(a,b) return (a % mod) > (b % mod) end)
  return alpha .. " " .. beta .. " " .. _G.table.concat(table,",")
end

print("Normal: ", test())

local dumped = string.dump(test)
test = loadstring(dumped)

print("Dumped #: ", #dumped)
print("After dump: ", test())

dumped = strip(dumped)
test = loadstring(dumped)

print("Stripped #: ", #dumped)
print("After strip: ", test())

-- Strip the strip function
dumped = string.dump(strip)
print("Dumped strip #: ", #dumped)
dumped = strip(dumped)
print("Stripped strip #: ", #dumped)
strip = loadstring(dumped)

dumped = strip(string.dump(function() print"Strip still works!" end))
print(assert(loadstring(dumped))())

On 26/02/2008, David Burgess <[hidden email]> wrote:
> Can we levaluate that  again? I think it is a valuable feature.
>
>  Davud B
>
>
>  On Wed, Feb 27, 2008 at 4:40 AM, Luiz Henrique de Figueiredo
>  <[hidden email]> wrote:
>  > > This was requested on 2006. Don't know if the discussion arrived to any
>  >  > conclusion.
>  >  >
>  >  > http://lua-users.org/lists/lua-l/2006-10/msg00013.html
>  >
>  >  I think the conclusion was that it would complicate the API.
>  >
>

Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Luiz Henrique de Figueiredo
> Strip in Lua:

Wow!

Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

David Jones-2

On 28 Feb 2008, at 11:13, Luiz Henrique de Figueiredo wrote:

Strip in Lua:

Wow!

Writing Lua code to manipulate the dumped representation of Lua functions is pretty fun.

Probably quite a few people have a couple of skeletons like this lurking in their cupboards.

Peter Cawley's is pretty neat!

drj

Reply | Threaded
Open this post in threaded view
|

Re: string.dump strip option (5.2 request)

Luiz Henrique de Figueiredo
> Writing Lua code to manipulate the dumped representation of Lua  
> functions is pretty fun.

Shameless plug: see my lbci at http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/#lbci

If there is interest, lbci could be extended to strip functions without
having to dump them first.