Using L in a function called via LuaJIT.FFI

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Using L in a function called via LuaJIT.FFI

Alexander Nasonov
Hi Mike,

I'm integrating readline-based interactive lua interpreter into my event
framework and I'm getting random crashes with -jon.

Lua interpreter code is based on src/lua.c from lua-5.1.4 but I made
some changes to accommodate to a different event model. Basically,
I do lua_cpcall in readline callback.

Now, to the tricky part. Plain cfunction inside a loop turns off JIT so
I pass lua_State* pointer from LuaJIT to C before I start a loop and
then I use the saved pointer when I call a callback function from inside
the loop via FFI interface.

In other words, I call read_char() from the event loop using FFI, which
eventually calls readline_callback(char *line) and at this point I use
the saved lua_State* pointer to call lua_cpcall(L, &cpreadline, line).

Is this valid at all?

(If I call read_char() via plain cfunction interface, I don't see crashes
but JIT is turned off in this case).

It's x86-64 Linux, in case it matters.

Thanks,
Alex

Reply | Threaded
Open this post in threaded view
|

Re: Using L in a function called via LuaJIT.FFI

Mike Pall-32
Alexander Nasonov wrote:
> Now, to the tricky part. Plain cfunction inside a loop turns off JIT so
> I pass lua_State* pointer from LuaJIT to C before I start a loop and
> then I use the saved pointer when I call a callback function from inside
> the loop via FFI interface.

Well, there's a reason the FFI doesn't provide for a way to access
the current lua_State pointer ... what you're doing is not safe.

> In other words, I call read_char() from the event loop using FFI, which
> eventually calls readline_callback(char *line) and at this point I use
> the saved lua_State* pointer to call lua_cpcall(L, &cpreadline, line).
>
> Is this valid at all?

Nope, the current lua_State is not in a safe state:

- When the FFI call is compiled, the global state is not in sync
  at all. Calling back into the same Lua state (or any coroutine
  of it) will lead to weird crashes, GC assertions and so on.

- When the FFI call is only interpreted, the global state is in
  sync, but the current Lua thread (coroutine) is not reentrant,
  i.e. you must not use lua_*call on it. It'll cause more subtle
  crashes if you violate this rule.

OTOH, if you ...

- Create an extra coroutine and anchor the Lua thread object
  somewhere

- *and* run your callback only on the associated lua_State pointer

- *and* make sure the FFI call to the outer event loop (or whatever)
  is not compiled,
 
... then this ought to be safe (famous last words).

To make sure the FFI call to the event loop is not JIT-compiled,
use jit.off(func) for the surrounding Lua function.

Another option is to call back into a separate Lua VM instance,
obtained with luaL_newstate() + luaL_openlib(). This is completely
safe, but you'll need to explicitly pass around code and data.
See: http://lua-users.org/lists/lua-l/2011-01/msg01579.html

--Mike

Reply | Threaded
Open this post in threaded view
|

Re: Using L in a function called via LuaJIT.FFI

Alexander Nasonov
Mike Pall wrote:
> Well, there's a reason the FFI doesn't provide for a way to access
> the current lua_State pointer ... what you're doing is not safe.

What's what I suspected.

> - When the FFI call is compiled, the global state is not in sync
>   at all. Calling back into the same Lua state (or any coroutine
>   of it) will lead to weird crashes, GC assertions and so on.
>
> - When the FFI call is only interpreted, the global state is in
>   sync, but the current Lua thread (coroutine) is not reentrant,
>   i.e. you must not use lua_*call on it. It'll cause more subtle
>   crashes if you violate this rule.

Very interesting. Can it be documented on ffi pages, please?

> OTOH, if you ...
>
> - Create an extra coroutine and anchor the Lua thread object
>   somewhere
>
> - *and* run your callback only on the associated lua_State pointer
>
> - *and* make sure the FFI call to the outer event loop (or whatever)
>   is not compiled,
>  
> ... then this ought to be safe (famous last words).
>
> To make sure the FFI call to the event loop is not JIT-compiled,
> use jit.off(func) for the surrounding Lua function.

Thanks, I'll try it.

> Another option is to call back into a separate Lua VM instance,
> obtained with luaL_newstate() + luaL_openlib(). This is completely
> safe, but you'll need to explicitly pass around code and data.
> See: http://lua-users.org/lists/lua-l/2011-01/msg01579.html

Yes, that should be completely safe. I'll give users a safe option but
for testing purposes I'd like to have access to the main state.

Many thanks for the help,
Alex

Reply | Threaded
Open this post in threaded view
|

Re: Using L in a function called via LuaJIT.FFI

Alek Paunov
In reply to this post by Mike Pall-32
Hi Mike,

I just read the "recipe" in:

http://lua-users.org/lists/lua-l/2011-02/msg00455.html

Which c coroutine library would you recommend for experimenting this
approach? Coco ?

Thank you!
Alek

Reply | Threaded
Open this post in threaded view
|

Re: Using L in a function called via LuaJIT.FFI

Mike Pall-32
Alek Paunov wrote:
> I just read the "recipe" in:
>
> http://lua-users.org/lists/lua-l/2011-02/msg00455.html
>
> Which c coroutine library would you recommend for experimenting this
> approach? Coco ?

Coco's context switching code isn't easily reusable outside of it.
For an initial experiment I'd recommend a C library that's
explicitly designed for lightweight context switching. I suggest
to get something running first and tune afterwards. Windows Fibers
or GNU Pth should be easy, but performance will definitely suck.
Once you are at that point, have a look at the GCC inline
assembler stuff in lcoco.c and try to adapt it to your needs.

--Mike