dostring and precompiled code

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

dostring and precompiled code

Luiz Henrique de Figueiredo
>PS: Btw, why does dostring rejects precompiled code at the moment?  Is it
>considered too dangerous?  Afaics it should work.

Besides being a little dangerous, there's no way to get precompiled code
except from a file. In that case, why not use dofile instead?
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Reuben Thomas-3
> Besides being a little dangerous, there's no way to get precompiled code
> except from a file. In that case, why not use dofile instead?

But that argument is (sensibly) not applied to lua_dobuffer...and Lua
strings can have embedded NULs, and contain the result of reading a file
(in the case that you want to execute some precompiled code repeatedly).

-- 
http://sc3d.org/rrt/ | maxim, n.  wisdom for fools


Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Cary O'Brien
In reply to this post by Luiz Henrique de Figueiredo
> >PS: Btw, why does dostring rejects precompiled code at the moment?  Is it
> >considered too dangerous?  Afaics it should work.
> 
> Besides being a little dangerous, there's no way to get precompiled code
> except from a file. In that case, why not use dofile instead?

We are looking at using lua in an embedded environment where there is no
file system.  I can imagine situations where we might want to send precompiled
code to the system over the communications link.

Another example might be a workflow system where trigger functions were
stored in a database.

-- cary


> --lhf
> 


Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Adolf Mathias
My two cents to this precompiled code issue:

The compiler is in a position to (theoretically) construct VM code that
is safe. Safe in a sense that it doesn't, say, do things or call
functions that it's not supposed to. This is not the case with
precompiled code in a string, since it can contain about anything that
the VM can do. 
<paranoia mode on>
The only place for precompiled code that resides in memory is in a data
type that has no other access possibilities but execution and might have
some cryptographic signature in order to prove that it has not been
altered in some way.
<paranoia mode off>
Other than that, precompiled code in memory is something that I would
have found useful some time ago.

Dolfi

ot
Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

ot
--- In lua-l@y..., Adolf Mathias <dolfi@z...> wrote:
> The compiler is in a position to (theoretically) construct VM code 
> that is safe.Safe in a sense that it doesn't, say, do things or call
> functions that it's not supposed to. This is not the case with
> precompiled code in a string, since it can contain about anything 
> that the VM can do. 
> <paranoia mode on>
> The only place for precompiled code that resides in memory is in a 
> data type that has no other access possibilities but execution and 
> might have some cryptographic signature in order to prove that it  
> has not been altered in some way.
> <paranoia mode off>

Beg to disgree here. Security vetting of code does not belong in the 
compiler. It belongs in the VM, or more properly, in the external 
functions available to the VM. If you want secure file IO for 
instance, the way to implement it is not to add an extra verification 
pass to the interpreter (what for?) but to ensure that the only Lua-
callable file IO functions available to that particular Lua instance 
contain verification routines (eg user/password checks). This is one 
of the strong points of Lua as it exists, that the Lua core comes 
with very few intrinsic functions; everything including IO is located 
in external libraries. 

The only difference between bytecode and source code is (or should 
be) compactness of representation, and differences due to 
optimization, macro expansion etc. Source code is basically human-
readable bytecode. 

Cheers, Elliott


Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Cary O'Brien
[Charset ISO-8859-1 unsupported, filtering to ASCII...]
> --- In lua-l@y..., Adolf Mathias <dolfi@z...> wrote:
> > The compiler is in a position to (theoretically) construct VM code 
> > that is safe.Safe in a sense that it doesn't, say, do things or call
> > functions that it's not supposed to. This is not the case with
> > precompiled code in a string, since it can contain about anything 
> > that the VM can do. 
> > <paranoia mode on>
> > The only place for precompiled code that resides in memory is in a 
> > data type that has no other access possibilities but execution and 
> > might have some cryptographic signature in order to prove that it  
> > has not been altered in some way.
> > <paranoia mode off>
> 
> Beg to disgree here. Security vetting of code does not belong in the 
> compiler. It belongs in the VM, or more properly, in the external 
> functions available to the VM. If you want secure file IO for 
> instance, the way to implement it is not to add an extra verification 
> pass to the interpreter (what for?) but to ensure that the only Lua-
> callable file IO functions available to that particular Lua instance 
> contain verification routines (eg user/password checks). This is one 
> of the strong points of Lua as it exists, that the Lua core comes 
> with very few intrinsic functions; everything including IO is located 
> in external libraries. 
> 

As a point of external reference, python has something called "bytecode-hacks"
that allows manipulation of bytecodes.  Zope uses this for a feature called
"python scripts" which allows somewhat untrusted users to enter python
code through a web interface.  Zope then (I think) uses bytecode-hacks
to restrict what code can be executed.  The point?  I guess you could
scan Lua bytecode before execution.

Another question (ok, I know 10 minutes with the source code and I could
figure this out myself): are all sequences of bytecodes safe?  At least
in terms of keeping the interpreter running?  No HCF or FOOF [1] instructions?

-- cary

[1] HCF - halt catch fire - an instruction on a particular single-chip
    micro that would turn on opposing output buffers.  Chip would melt.

[2] Pentium opcode that caused lockup.


> The only difference between bytecode and source code is (or should 
> be) compactness of representation, and differences due to 
> optimization, macro expansion etc. Source code is basically human-
> readable bytecode. 
> 
> Cheers, Elliott
> 


ot
Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

ot
--- In lua-l@y..., "Cary O'Brien" <cobrien@R...> wrote:
> Another question (ok, I know 10 minutes with the source code and I 
> could figure this out myself): are all sequences of bytecodes      
> safe?  At least in terms of keeping the interpreter running?  No   
> HCF or FOOF [1] instructions?
> [1] HCF - halt catch fire - an instruction on a particular single-  
> chip micro that would turn on opposing output buffers.  Chip would 
> melt. 
> [2] Pentium opcode that caused lockup.

You have to distinguish between a number of "safety" issues. 

One is the "sandbox" issue: is a program running in the Lua VM able 
to crash the host program? The most common cause is a memory access 
violation (segfault), or an untrapped fatal signal (untrapped divide-
by-zero errors, for instance). I haven't examined the Lua VM source 
code minutely, but from what I have seen it is safe in this respect. 
Source code that exceeds array bounds, or tries a = 1/0 should not 
crash the Lua host.

Then there is the question of whether a program running in the Lua VM 
is able to halt the VM itself. For instance, a sequence of random 
bytes interpreted as bytecode may cause the VM to grind to a halt and 
refuse to continue execution. This can happen with Lua. This is not 
neccessarily a problem if the host program is designed to handle the 
possibility of invalid code being offered. This is not merely limited 
to byte code; it applies to invalid or buggy source code too. 

Finally there is the question of whether a Lua program can cause a 
Denial of Service problem. For instance, causing an endless loop, 
overutilising CPU resources, holding files open in write mode 
unneccessarily, allocating ever-increasing number of objects, 
blocking sockets, etc. The answer is, definitely. Solving this is a 
lot more difficult, as the implementation has to be able to 
differentiate between genuine loads and hostile attacks. I think the 
only way to resolve this kind of problem satisfactorily is by careful 
program design, and I don't think any one single fit-all solution 
exists.

There may be other bugs in the Lua VM eg a random byte sequence that 
results in the host machine turning into a purple crab, spitting on 
you, and waddling away. Little can be done about this class of bugs, 
other than hoping that Luis, Roberto or Waldemar fix it as soon as it 
is sighted :-)

Cheers, Elliott


Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Luiz Henrique de Figueiredo
In reply to this post by Luiz Henrique de Figueiredo
>Another question (ok, I know 10 minutes with the source code and I could
>figure this out myself): are all sequences of bytecodes safe?  At least
>in terms of keeping the interpreter running?  No HCF or FOOF [1] instructions?

The VM should be completely safe, in the sense that no sequence of bytecodes
should dump core or something like that. Of course, there can be no guarantee
that the VM will not be locked into an infinite loop or such. Back in 3.2,
luac had a -t option that tested the integrity of bytecodes, but I didn't
find the time to make it work in 4.0 (my fault not 4.0...). Il try to have it
running for 4.1.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Steve Dekorte-4
In reply to this post by Luiz Henrique de Figueiredo
Cary O'Brien wrote:
> We are looking at using lua in an embedded environment where there is no 
> file system.  I can imagine situations where we might want to send precompiled 
> code to the system over the communications link. 

That's why we wanted that functionality for yindo as well. Why download code 
across the net to memory and then have to put it in a temp file, read it back into
memory and delete the file?

Steve

Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Jean-Claude Wippler
In reply to this post by Luiz Henrique de Figueiredo
On Mon, Feb 5, 2001, Luiz Henrique de Figueiredo <[hidden email]>
wrote:

>[...] If dostring accepted precompiled code, it might be possible to crash 
>the host by feeding dostring with malicious code [...]
>
>Anyway, it's simple to change dostring to allow precompiled code: Just remove
>the two lines marked below from lbaselib.c:
>
> static int luaB_dostring (lua_State *L) {
>   int oldtop = lua_gettop(L);
>   size_t l;
>   const char *s = luaL_check_lstr(L, 1, &l);
>|  if (*s == '\27')  /* binary files start with ESC... */
>|    lua_error(L, "`dostring' cannot run pre-compiled code");
>   return passresults(L, lua_dobuffer(L, s, l, luaL_opt_string(L, 2, s)),
>oldtop)
> ;
>}

Suggestion: please move those two lines to lua_dostring in lapi.c, since
that call cannot handle precompiled code with embedded null bytes.

In terms of what to do in C and what to do in Lua, my mantra is:
    Make POLICY decisons scripted, write just INTERFACES in C

The reason is that this way one can provide maximum functionality, yet
use the scripting layer to alter/restrict the API when needed.  An
(untested) example:

  function dostring(s,...)
    if strbyte(s)==27 then
      error("`dostring' cannot run pre-compiled code")
    end
    return %dostring(s,arg[1])
  end

-jcw

Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Edgar Toernig
In reply to this post by Luiz Henrique de Figueiredo
Luiz Henrique de Figueiredo wrote:
> If dostring accepted
> precompiled code, it might be possible to crash the host by feeding dostring
> with malicious code, created on the fly in Lua, although I almost sure
> that badly built precompiled code is flagged; the only problems would be
> runtime errors, which have been discussed here in previous messages, and are
> hard to handle.

I'm not sure if I understand you right, but malicious code is definitely able
to generate a segfault.  In fact, the virtual machine just assumes "correct"
code.  It will happy execute a pushnil(999999).

> |  if (*s == '\27')  /* binary files start with ESC... */
> |    lua_error(L, "`dostring' cannot run pre-compiled code");

And if you keep it in, better make that '\33' ;-)

Ciao, ET.


Reply | Threaded
Open this post in threaded view
|

Re: dostring and precompiled code

Luiz Henrique de Figueiredo
In reply to this post by Luiz Henrique de Figueiredo
>I'm not sure if I understand you right, but malicious code is definitely able
>to generate a segfault.  In fact, the virtual machine just assumes "correct"
>code.  It will happy execute a pushnil(999999).

Quite right.

>> |  if (*s == '\27')  /* binary files start with ESC... */
>> |    lua_error(L, "`dostring' cannot run pre-compiled code");
>
>And if you keep it in, better make that '\33' ;-)

Oops, this is a bug then. It's 27 not '\27'. Thanks.
So, after all dostring in 4.0 does run precompiled code!!
--lhf