embedded lua in web based REPL

[gary ng]

Hi,

Just wondering if anyone has done this and how one would handle DOS attack case of user entering :

for i = 1 to 10000000 do end


     

[Shaun McVey]

The Lua website itself has a live demo (http://www.lua.org/demo.html).
 Cases like this appear to be protected against.  Unfortunately, I
couldn't find any details on the website about how it's implemented.

On Fri, Mar 19, 2010 at 9:52 PM, gary ng <[hidden email]> wrote:
> Hi,
>
> Just wondering if anyone has done this and how one would handle DOS attack case of user entering :
>
> for i = 1 to 10000000 do end
>
>
>
>

[Sam Roberts]

On Fri, Mar 19, 2010 at 3:07 PM, Shaun McVey <[hidden email]> wrote:
> The Lua website itself has a live demo (http://www.lua.org/demo.html).
>  Cases like this appear to be protected against.  Unfortunately, I
> couldn't find any details on the website about how it's implemented.

Basically using this hook, if I recall correctly:

http://www.lua.org/manual/5.1/manual.html#pdf-debug.sethook

If you google a bit you'll find this discussed a bunch of times in the
list archive, and there might even be a page about it on the lua wiki

Cheers,
Sam

[Jonathan Castello-2]

On Fri, Mar 19, 2010 at 3:15 PM, Sam Roberts <[hidden email]> wrote:

> On Fri, Mar 19, 2010 at 3:07 PM, Shaun McVey <[hidden email]> wrote:
>> The Lua website itself has a live demo (http://www.lua.org/demo.html).
>>  Cases like this appear to be protected against.  Unfortunately, I
>> couldn't find any details on the website about how it's implemented.
>
> Basically using this hook, if I recall correctly:
>
> http://www.lua.org/manual/5.1/manual.html#pdf-debug.sethook
>
> If you google a bit you'll find this discussed a bunch of times in the
> list archive, and there might even be a page about it on the lua wiki
>
> Cheers,
> Sam

As a related question, is it legal in Lua 5.2 to use coroutine.yield
from within a hook callback? Lua 5.1 doesn't allow yielding across
C-call boundaries, but I recall reading that it was possible to some
extent in Lua 5.2. It could be moderately useful to do this when you
have a bunch of unrelated worker coroutines; you could assign
"policies" to them to regulate the amount of time each spends
computing by calling the hook after every X instructions.

~Jonathan Castello

[Luiz Henrique de Figueiredo]

> On Fri, Mar 19, 2010 at 3:07 PM, Shaun McVey <[hidden email]> wrote:
> > The Lua website itself has a live demo (http://www.lua.org/demo.html).
> >  Cases like this appear to be protected against.  Unfortunately, I
> > couldn't find any details on the website about how it's implemented.
>
> Basically using this hook, if I recall correctly:
>
> http://www.lua.org/manual/5.1/manual.html#pdf-debug.sethook

Actually, no: it's a simple "ulimit -t 1".
The source code for the demo is available at
        http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/#demo

I think the lua_bot in #lua used to use hooks but
now has moved to ulimit, but I'm not sure.

[Roberto Ierusalimschy]

> As a related question, is it legal in Lua 5.2 to use coroutine.yield
> from within a hook callback? [...]

This was already legal in 5.1.

-- Roberto

[Jonathan Castello-2]

On Sat, Mar 20, 2010 at 6:30 AM, Roberto Ierusalimschy
<[hidden email]> wrote:
>> As a related question, is it legal in Lua 5.2 to use coroutine.yield
>> from within a hook callback? [...]
>
> This was already legal in 5.1.
>
> -- Roberto

Odd; I tried using it (in both 5.1.4 and 5.2 work2) yesterday, and
coroutine.yield returned "attempt to yield across metamethod/C-call
boundary". Here's the code I used:

----
co = coroutine.create(function() for i=1,100 do print(i) end end)
debug.sethook(co, function() print("hooked") coroutine.yield() end, "", 1)
print(coroutine.resume())
----
Output:
hooked
false attempt to yield across metamethod/C-call boundary
----

~Jonathan

[Jonathan Castello-2]

On Sat, Mar 20, 2010 at 12:04 PM, Jonathan Castello <[hidden email]> wrote:
> print(coroutine.resume())

Sorry, I meant print(coroutine.resume(co)), mis-copied.

~Jonathan

[Roberto Ierusalimschy]

> On Sat, Mar 20, 2010 at 6:30 AM, Roberto Ierusalimschy
> <[hidden email]> wrote:
> >> As a related question, is it legal in Lua 5.2 to use coroutine.yield
> >> from within a hook callback? [...]
> >
> > This was already legal in 5.1.
> >
> > -- Roberto
>
> Odd; I tried using it (in both 5.1.4 and 5.2 work2) yesterday, and
> coroutine.yield returned "attempt to yield across metamethod/C-call
> boundary". Here's the code I used:
>
> ----
> co = coroutine.create(function() for i=1,100 do print(i) end end)
> debug.sethook(co, function() print("hooked") coroutine.yield() end, "", 1)
> print(coroutine.resume())
> ----
> Output:
> hooked
> false attempt to yield across metamethod/C-call boundary

I meant calling yield from the "real" hook, in C.

-- Roberto

[Jonathan Castello-2]

On Sat, Mar 20, 2010 at 12:25 PM, Roberto Ierusalimschy
<[hidden email]> wrote:
> I meant calling yield from the "real" hook, in C.
>
> -- Roberto

I'm not following. You're saying I can't use coroutine.yield() from a
Lua hook function, but I can use its equivalent from a C hook
function?

Well, okay. I think it would be really useful to be able to do it with
straight Lua, though. I've been thinking about it a lot lately; you
could almost build preemptive threads on top of it, just by having a
coroutine's hook check whatever constraints and yield when it's time
to let another thread do some work.

The specific situation I would want to use it for is in a web
application, where users can write plugins in Lua and run them on the
server, in a highly sandboxed environment. Such an approach to
coroutines would make it easy to prevent one rogue plugin from lagging
or DoSing everyone else, as the original email in this thread
mentioned. And it's important to my design that the plugins run in the
same Lua interpreter, so I don't think I could use OS-level threads
for this.

~Jonathan

[Roberto Ierusalimschy]

> I'm not following. You're saying I can't use coroutine.yield() from a
> Lua hook function, but I can use its equivalent from a C hook
> function?

Yes.

Note that the hook system does not use "Lua functions", but a special
kind of C function (lua_Hook). The Lua debug library uses a standard
lua_Hook function that calls your Lua function, and this call is not
yieldable.

-- Roberto

[Jonathan Castello-2]

On Sun, Mar 21, 2010 at 7:40 AM, Roberto Ierusalimschy
<[hidden email]> wrote:

>> I'm not following. You're saying I can't use coroutine.yield() from a
>> Lua hook function, but I can use its equivalent from a C hook
>> function?
>
> Yes.
>
> Note that the hook system does not use "Lua functions", but a special
> kind of C function (lua_Hook). The Lua debug library uses a standard
> lua_Hook function that calls your Lua function, and this call is not
> yieldable.
>
> -- Roberto

Okay, I understand now. Looking back through the list archives, I note
(happily) that others have had the same preemptive threading idea.
I've also gone into the Lua 5.2 source myself and tinkered with it,
changing the lua_call in the 'hookf' function to a lua_callk. It did
actually successfully yield, but once it left the lua hook function,
an access violation occurred. (Due to a typo, I noticed that the
access violation didn't happen if an error occurred within the hook
function after being resumed. Interesting!)

I'm getting more and more curious as I experiment further. From a
technical, low-level standpoint, what exactly prevents Lua hooks from
becoming yieldable? I'd just like to patch my own copy, if at all
possible.

~Jonathan

[Roberto Ierusalimschy]

> I'm getting more and more curious as I experiment further. From a
> technical, low-level standpoint, what exactly prevents Lua hooks from
> becoming yieldable? I'd just like to patch my own copy, if at all
> possible.

When a C hook yields, it is actually making the corresponding Lua
function (that triggered the hook) yield. The hook itself does not
need to be saved.

If a Lua hook yields, a resume should resume the Lua hook itself, not
the Lua function that hooked, so the C hook should be saved.

It is not difficult to allow the Lua hook to hook the original
Lua function (e.g., by returning a specific value).

-- Roberto

[Jonathan Castello-2]

On Mon, Mar 22, 2010 at 7:47 AM, Roberto Ierusalimschy
<[hidden email]> wrote:
> If a Lua hook yields, a resume should resume the Lua hook itself, not
> the Lua function that hooked, so the C hook should be saved.
>
> It is not difficult to allow the Lua hook to hook the original
> Lua function (e.g., by returning a specific value).

I've tried a few things along these lines but without much luck. I
think the main problem is just that I don't have a good understanding
of how yields and resumes operate.

The first thing I tried was changing the lua_call to lua_callk in
'hookf', as I mentioned before, and using coroutine.yield in the lua
hook. That caused an access violation when the Lua hook function
returned. The second thing I tried was adding lua_yield below lua_call
in 'hookf', as a trial run of the "yield based on hook return value"
idea. But every time I use coroutine.resume on the coroutine, it just
calls the Lua hook again. This might be obvious, since I called
debug.sethook with a count of 1, but the coroutine never ends. If I
have a print statement in the coroutine, it's never reached. I never
get "cannot resume dead coroutine".

Hence, I figure that yield/resume do something I'm not expecting, or
don't understand. I'd appreciate a simple example of how to use
lua_yield in a C hook function, if at all possible!

(And on a point of aesthetics, I find the idea of yielding based on a
hook's return value to feel rather un-Lua-y. I'd rather be able to
call coroutine.yield from the Lua hook, but I'll take what I can get.)

~Jonathan

[Roberto Ierusalimschy]

> I've tried a few things along these lines but without much luck. I
> think the main problem is just that I don't have a good understanding
> of how yields and resumes operate.

That really makes things difficult...


> Hence, I figure that yield/resume do something I'm not expecting, or
> don't understand. I'd appreciate a simple example of how to use
> lua_yield in a C hook function, if at all possible!

    return lua_yield(L, 0);

That is it.

As I told you, C hooks are not regular lua_Cfunctions. In particular, you
cannot use lua_*k functions inside them. This case of 'lua_yield' is
handled separately in the VM, as a special case.

-- Roberto

[Jonathan Castello-2]

>> I've tried a few things along these lines but without much luck. I
>> think the main problem is just that I don't have a good understanding
>> of how yields and resumes operate.
>
> That really makes things difficult...

Heh, yes, it does. I've read up on setjmp/longjmp, it's just how
they're used by Lua that's a bit beyond me.

>    return lua_yield(L, 0);

Well, I put exactly that after lua_call in the hook function; as I
said before, that was the second thing I tried. (Except that I can't
return the return value of lua_yield, the hook returns void, I just
left off the "return", but it's the last thing the function does.) And
it seemed to never go back to the coroutine, it just re-executed the
hook function every time I resumed.

> As I told you, C hooks are not regular lua_Cfunctions. In particular, you
> cannot use lua_*k functions inside them.

Ah, I didn't realize that. It's not really documented anywhere, is it?

~Jonathan

[Roberto Ierusalimschy]

> Ah, I didn't realize that. It's not really documented anywhere, is it?

Not yet. (This is a work version.)

-- Roberto

[Jonathan Castello-2]

> Not yet. (This is a work version.)

Right. Alright, I'll keep chipping away at it... Thanks for all the help so far!

~Jonathan