[ANN] New project: Safe Lua

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] New project: Safe Lua

Stefan Reich
Hello Lua people!

I am currently realizing some ideas I have had for a long time
regarding fine-grained customizable sandboxes.

I had an earlier research project called "Imaginary Microcomputers"
that explored this (parts of it still online) plus a few other
projects. These projects yielded insights and prototypes; but what
they lacked was the right programming language for the job.
(Noteworthy language candidates included: Assembly, Java, Python, and
"E".)

There are very interesting applications for customizable, light-weight
sandboxes that, to my knowledge, have not been realized anywhere yet.

It all starts with a system that can run untrusted code safely. And
then, optionally, you go on to connect running programs to each other
in a well-defined, restricted way.

I am happy to report that I think I have now finally found a language
that is suitable for implementing this. As you may have guessed by
now: It's Lua :)

Lua seems to provide all the necessary means to create real sandboxes
and extend/modify them the way I want. Even CPU and memory consumption
can be limited which is an important feature that many other candidate
languages I looked at did not provide.

Here's the project homepage: http://safelua.sf.net

I made a first release with a very simple script runner (safelua.lua)
and two examples, downloadable from the project page.

A general note: I don't intend to really "own" this project. I do want
to maintain my own page about it. And maybe maintain some sort of
steering oversight because I have a vision I want to see realized.
Other than that, I really do welcome any and all collaboration here.
And of course, you can always fork the thing if you feel that your
vision is somehow cooler (hotter?) than mine :)

In fact, if a better system exists that suits all my needs, I will be
happy to throw mine away and use that system instead. However, I don't
know of any such system yet.

So, it does look like we're building something new here.

Many components will want to be realized. A language definition for
Safe Lua (quite simple really, it's just Lua with less globals and a
bit of a new API). Safe Lua script runners, textual as well as
graphical. Some simple means to combine scripts with each other.
Standard components that take other scripts as input and/or output
(this is where the real power of the approach begins).

As for possible applications, here's a few:

-Safe, portable, mobile agents
-Execution of untrusted code without worries
-Migrating running code from one machine to another with a single click
-Cloning running programs with equally little effort
-Orthogonal or semi-orthogonal persistence
-Logging of each and all activity, including full replayability - live
or post-portem
-Self-unpacking data with arbitrary algorithms (procedural compression)
-A complete "Safe Lua OS" could be developed, providing perfect
portability and much better and easier to handle security than
traditional OSes

So... well well. As I said before: Contributions, questions or ideas
will be very appreciated. (Don't flame me though... I might flame
back! *grins broadly*)

Best regards to you all,

Stefan Reich
Software enthusiast / Activist of the German revolution

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Josh Simmons
On Wed, Aug 3, 2011 at 4:30 AM, Stefan Reich
<[hidden email]> wrote:

> Hello Lua people!
>
> I am currently realizing some ideas I have had for a long time
> regarding fine-grained customizable sandboxes.
>
> I had an earlier research project called "Imaginary Microcomputers"
> that explored this (parts of it still online) plus a few other
> projects. These projects yielded insights and prototypes; but what
> they lacked was the right programming language for the job.
> (Noteworthy language candidates included: Assembly, Java, Python, and
> "E".)
>
> There are very interesting applications for customizable, light-weight
> sandboxes that, to my knowledge, have not been realized anywhere yet.
>
> It all starts with a system that can run untrusted code safely. And
> then, optionally, you go on to connect running programs to each other
> in a well-defined, restricted way.
>
> I am happy to report that I think I have now finally found a language
> that is suitable for implementing this. As you may have guessed by
> now: It's Lua :)
>
> Lua seems to provide all the necessary means to create real sandboxes
> and extend/modify them the way I want. Even CPU and memory consumption
> can be limited which is an important feature that many other candidate
> languages I looked at did not provide.
>
> Here's the project homepage: http://safelua.sf.net
>
> I made a first release with a very simple script runner (safelua.lua)
> and two examples, downloadable from the project page.
>
> A general note: I don't intend to really "own" this project. I do want
> to maintain my own page about it. And maybe maintain some sort of
> steering oversight because I have a vision I want to see realized.
> Other than that, I really do welcome any and all collaboration here.
> And of course, you can always fork the thing if you feel that your
> vision is somehow cooler (hotter?) than mine :)
>
> In fact, if a better system exists that suits all my needs, I will be
> happy to throw mine away and use that system instead. However, I don't
> know of any such system yet.
>
> So, it does look like we're building something new here.
>
> Many components will want to be realized. A language definition for
> Safe Lua (quite simple really, it's just Lua with less globals and a
> bit of a new API). Safe Lua script runners, textual as well as
> graphical. Some simple means to combine scripts with each other.
> Standard components that take other scripts as input and/or output
> (this is where the real power of the approach begins).
>
> As for possible applications, here's a few:
>
> -Safe, portable, mobile agents
> -Execution of untrusted code without worries
> -Migrating running code from one machine to another with a single click
> -Cloning running programs with equally little effort
> -Orthogonal or semi-orthogonal persistence
> -Logging of each and all activity, including full replayability - live
> or post-portem
> -Self-unpacking data with arbitrary algorithms (procedural compression)
> -A complete "Safe Lua OS" could be developed, providing perfect
> portability and much better and easier to handle security than
> traditional OSes
>
> So... well well. As I said before: Contributions, questions or ideas
> will be very appreciated. (Don't flame me though... I might flame
> back! *grins broadly*)
>
> Best regards to you all,
>
> Stefan Reich
> Software enthusiast / Activist of the German revolution
>
>

I'm not super sure what exactly you're going for here, Lua already is
very easy to sandbox completely the issue is providing "safe" APIs
that provide enough power to be useful. And that becomes more than
just safety it turns into the job of providing fine grained
permissions to individual applications.

If these kind of things are your goals then existing projects already
provide similar but more powerful control over these issues, things
like Linux cgroups and Chrome's NaCL.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Mateusz Czaplinski
On Wed, Aug 3, 2011 at 2:53 AM, Josh Simmons <[hidden email]> wrote:
> On Wed, Aug 3, 2011 at 4:30 AM, Stefan Reich
> <[hidden email]> wrote:
>> Here's the project homepage: http://safelua.sf.net
>
> I'm not super sure what exactly you're going for here, Lua already is
> very easy to sandbox completely

Umm... I'm not so sure about it... when I look at
http://lua-users.org/wiki/SandBoxes and the amount of question marks
and "not guaranteed" disclaimers on the page.
Among others, bytecode is a known non-obvious vector of attack.

> the issue is providing "safe" APIs
> that provide enough power to be useful.

What about infinite loops and eating all memory?

> If these kind of things are your goals then existing projects already
> provide similar but more powerful control over these issues, things
> like Linux cgroups and Chrome's NaCL.

Yeah, initially I thought the OP was doing some interesting scientific
research and wanted to publish it, but then I realized it's a project
claiming a lot ("super-duper extra safe!") but I totally don't trust
it. Zero proofs, zero references, no protection against malicious
bytecode if I see correctly (thus, absolutely not safe), and author
seems to skim over the "making minimal sandbox" step as if it was
trivial. <shudders>

/M.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Josh Simmons
On Wed, Aug 3, 2011 at 6:10 PM, Mateusz Czaplinski <[hidden email]> wrote:

> On Wed, Aug 3, 2011 at 2:53 AM, Josh Simmons <[hidden email]> wrote:
>> On Wed, Aug 3, 2011 at 4:30 AM, Stefan Reich
>> <[hidden email]> wrote:
>>> Here's the project homepage: http://safelua.sf.net
>>
>> I'm not super sure what exactly you're going for here, Lua already is
>> very easy to sandbox completely
>
> Umm... I'm not so sure about it... when I look at
> http://lua-users.org/wiki/SandBoxes and the amount of question marks
> and "not guaranteed" disclaimers on the page.
> Among others, bytecode is a known non-obvious vector of attack.
>
>> the issue is providing "safe" APIs
>> that provide enough power to be useful.
>
> What about infinite loops and eating all memory?
>
>> If these kind of things are your goals then existing projects already
>> provide similar but more powerful control over these issues, things
>> like Linux cgroups and Chrome's NaCL.
>
> Yeah, initially I thought the OP was doing some interesting scientific
> research and wanted to publish it, but then I realized it's a project
> claiming a lot ("super-duper extra safe!") but I totally don't trust
> it. Zero proofs, zero references, no protection against malicious
> bytecode if I see correctly (thus, absolutely not safe), and author
> seems to skim over the "making minimal sandbox" step as if it was
> trivial. <shudders>
>
> /M.
>
>

Lua is easy to sandbox since you can remove all the unsafe functions
and/or replace them with ones of your own design. Memory and CPU are
also reasonably easy to control with help from the OS, or by removing
unfriendly functions. The easy way to protect against malicious
byte-code is to disallow it altogether.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Stefan Reich
>>> I'm not super sure what exactly you're going for here, Lua already is
>>> very easy to sandbox completely

Yes. That's why I use Lua. The point is in running EVERYTHING in an
array of communicating sandboxes, which enables new kinds of
applications. I guess you'll understand the idea a lot better when
there are some non-trivial examples.

>> Umm... I'm not so sure about it... when I look at
>> http://lua-users.org/wiki/SandBoxes and the amount of question marks
>> and "not guaranteed" disclaimers on the page.
>> Among others, bytecode is a known non-obvious vector of attack.

Huh? Bytecode will be disallowed in my system as there is no bytecode
verifier for Lua yet. In fact, I just implemented that check (a simple
test for the first byte of the file) and it will be in the next
release.

>> What about infinite loops and eating all memory?

Will all be protected against. Infinite loops I have taken care of
already through a debug hook. It's also gonna be in the next release.
Memory can be handled similarly. In the worst case, we'll patch the
interpreter and add a check in the allocator. Any more questions? :)

>>
>>> If these kind of things are your goals then existing projects already
>>> provide similar but more powerful control over these issues, things
>>> like Linux cgroups and Chrome's NaCL.

Ah. OK! I'll look into those technologies and see if they bear any
relation to my vision. I kinda doubt it, but... well, you never know.
:)

>> Yeah, initially I thought the OP was doing some interesting scientific
>> research and wanted to publish it, but then I realized it's a project
>> claiming a lot ("super-duper extra safe!") but I totally don't trust
>> it. Zero proofs, zero references, no protection against malicious
>> bytecode if I see correctly (thus, absolutely not safe), and author
>> seems to skim over the "making minimal sandbox" step as if it was
>> trivial. <shudders>

Dude. It's the first release to get the word out. All the science you
want is gonna come. Protecting against bytecode is super simple. And
making a minimal sandbox is indeed easy with Lua. Lua is in use in
huge real-world projects and it seems it has proven that it works,
even with untrusted code, so it's a good basis to start with.

If there turn out to be any holes anyway, we'll fix them. This is a
research project, and it is obviously in the academic stage now.
Heavy-duty real-world application comes after that. You should know
that, given that you throw the word 'science' around a lot... :)

Having that out of the way: Have you even gotten a glimpse of the
vision of the whole thing? If not, please refrain from trashing the
project until you understand it. Because there is a vision behind it.
It's not just finger exercise. Thanks.

> Lua is easy to sandbox since you can remove all the unsafe functions
> and/or replace them with ones of your own design. Memory and CPU are
> also reasonably easy to control with help from the OS, or by removing
> unfriendly functions. The easy way to protect against malicious
> byte-code is to disallow it altogether.

Yes! That's what I do here and that's why I chose Lua.

-Stefan

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Josh Simmons
On Wed, Aug 3, 2011 at 7:37 PM, Stefan Reich
<[hidden email]> wrote:
>>> What about infinite loops and eating all memory?
>
> Will all be protected against. Infinite loops I have taken care of
> already through a debug hook. It's also gonna be in the next release.
> Memory can be handled similarly. In the worst case, we'll patch the
> interpreter and add a check in the allocator. Any more questions? :)
>

Debug hooks won't solve all your problems, it's easy to lock up in a C
function call. You need OS help unless you want to restrict the
environment completely.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Rena
In reply to this post by Stefan Reich
On Wed, Aug 3, 2011 at 03:37, Stefan Reich
<[hidden email]> wrote:
> Memory can be handled similarly. In the worst case, we'll patch the
> interpreter and add a check in the allocator. Any more questions? :)

That you suggest patching the interpreter instead of simply using a
custom allocator (which can be done without any patching) is a bit
worrisome.

I have thought about the idea of a Lua-based OS, and it could
certainly be interesting...

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Christian Thaeter
In reply to this post by Stefan Reich
Am Tue, 2 Aug 2011 20:30:25 +0200
schrieb Stefan Reich <[hidden email]>:

> Hello Lua people!
>
> I am currently realizing some ideas I have had for a long time
> regarding fine-grained customizable sandboxes.
>
> I had an earlier research project called "Imaginary Microcomputers"
> that explored this (parts of it still online) plus a few other
> projects. These projects yielded insights and prototypes; but what
> they lacked was the right programming language for the job.
> (Noteworthy language candidates included: Assembly, Java, Python, and
> "E".)
>
> There are very interesting applications for customizable, light-weight
> sandboxes that, to my knowledge, have not been realized anywhere yet.
>
> It all starts with a system that can run untrusted code safely. And
> then, optionally, you go on to connect running programs to each other
> in a well-defined, restricted way.
>
> I am happy to report that I think I have now finally found a language
> that is suitable for implementing this. As you may have guessed by
> now: It's Lua :)
>
> Lua seems to provide all the necessary means to create real sandboxes
> and extend/modify them the way I want. Even CPU and memory consumption
> can be limited which is an important feature that many other candidate
> languages I looked at did not provide.
>
> Here's the project homepage: http://safelua.sf.net
>
> I made a first release with a very simple script runner (safelua.lua)
> and two examples, downloadable from the project page.
>
> A general note: I don't intend to really "own" this project. I do want
> to maintain my own page about it. And maybe maintain some sort of
> steering oversight because I have a vision I want to see realized.
> Other than that, I really do welcome any and all collaboration here.
> And of course, you can always fork the thing if you feel that your
> vision is somehow cooler (hotter?) than mine :)
>
> In fact, if a better system exists that suits all my needs, I will be
> happy to throw mine away and use that system instead. However, I don't
> know of any such system yet.
>
> So, it does look like we're building something new here.
>
> Many components will want to be realized. A language definition for
> Safe Lua (quite simple really, it's just Lua with less globals and a
> bit of a new API). Safe Lua script runners, textual as well as
> graphical. Some simple means to combine scripts with each other.
> Standard components that take other scripts as input and/or output
> (this is where the real power of the approach begins).
>
> As for possible applications, here's a few:
>
> -Safe, portable, mobile agents
> -Execution of untrusted code without worries
> -Migrating running code from one machine to another with a single
> click -Cloning running programs with equally little effort
> -Orthogonal or semi-orthogonal persistence
> -Logging of each and all activity, including full replayability - live
> or post-portem
> -Self-unpacking data with arbitrary algorithms (procedural
> compression) -A complete "Safe Lua OS" could be developed, providing
> perfect portability and much better and easier to handle security than
> traditional OSes
>
> So... well well. As I said before: Contributions, questions or ideas
> will be very appreciated. (Don't flame me though... I might flame
> back! *grins broadly*)

I once made 'ulua' a suexec and chrooting lua interpreter:
 http://git.pipapo.org/?p=uwiki/cehteh;a=tree;f=ulua

It is in no means polished yet, but already somewhat useful.
If you like it you can integrate this in safe-lua.

        Christian


>
> Best regards to you all,
>
> Stefan Reich
> Software enthusiast / Activist of the German revolution
>

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Robert G. Jakabosky
In reply to this post by Stefan Reich
On Wednesday 03, Stefan Reich wrote:
> Memory can be handled similarly. In the worst case, we'll patch the
> interpreter and add a check in the allocator. Any more questions? :)

For limiting memory usage I would recommend my Emergency GC patch:
http://lua-users.org/wiki/EmergencyGarbageCollector

Using just a custom allocator (without EGC support) will not allows scripts to
run reliably when they get close to the limit.

--
Robert G. Jakabosky

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Stefan Reich
In reply to this post by Christian Thaeter
On Wed, Aug 3, 2011 at 2:23 PM, Christian Thaeter <[hidden email]> wrote:
> I once made 'ulua' a suexec and chrooting lua interpreter:
>  http://git.pipapo.org/?p=uwiki/cehteh;a=tree;f=ulua
>
> It is in no means polished yet, but already somewhat useful.
> If you like it you can integrate this in safe-lua.

Interesting. How would this add to Safe Lua though? I am not sure how
it would fit in and for exactly what purpose. I am concerned with
doing stuff within the Lua universe, not so much with what is on the
OS level. (Although we will look at that too when we make a
Lua-centric OS or build some realworld apps.)

Robert G. Jakabosky wrote:
> For limiting memory usage I would recommend my Emergency GC patch:
> http://lua-users.org/wiki/EmergencyGarbageCollector

> Using just a custom allocator (without EGC support) will not allows scripts to
> run reliably when they get close to the limit.

I see. I am currently not focused on the memory issue, but it's good
to know that solutions are available when we want to get airtight
there. Is your GC really stable? I see there is a stress test. Does it
pass? :D

Also, how does limiting memory for coroutines work with your GC? I
couldn't find an example. Is it done from within Lua?

I have always been interested in garbage collectors, I think they are
a fantastic addition to any programming language. Actually many things
just can't be done without them. I'm really glad the fight about
whether or not to have GC is mostly over. We still have crashing C++
programs, but hey, they're going out of fashion clearly.

Stefan

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] New project: Safe Lua

Robert G. Jakabosky
On Saturday 06, Stefan Reich wrote:

>
> Robert G. Jakabosky wrote:
> > For limiting memory usage I would recommend my Emergency GC patch:
> > http://lua-users.org/wiki/EmergencyGarbageCollector
> >
> > Using just a custom allocator (without EGC support) will not allows
> > scripts to run reliably when they get close to the limit.
>
> I see. I am currently not focused on the memory issue, but it's good
> to know that solutions are available when we want to get airtight
> there. Is your GC really stable? I see there is a stress test. Does it
> pass? :D

Yes it is stable, there are no outstanding bugs.  It is currently being used
by the eLua project [1].

> Also, how does limiting memory for coroutines work with your GC? I
> couldn't find an example. Is it done from within Lua?

With how the Lua vm was designed there is no way to have per-coroutine memory
limits in the same Lua State (I mean the main lua_State instance, not the
lua_State's you get from lua_newthread).

I would recommend using separate lua_State's (created from
lua_newstate/luaL_newstate) for each script.

To set the memory limit from C:
lua_gc(L, LUA_GCSETMEMLIMIT, 2 * 1024); /* 2Mbyte limit */

From Lua:
collectgarbage("setmemlimit", 2 * 1024) -- 2Mbyte limit

Set the limit to 0 (the default value) to disable the limit.

1. http://www.eluaproject.net/
--
Robert G. Jakabosky