Tricky c#/LuaInterface Question

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Tricky c#/LuaInterface Question

Geoff Smith
Hello
 
I have been using LuaInterface for a little while, I have got on pretty well with it so far until I hit this difficult problem. The documentation for it doesn't really help me much on this issue, so hopefully someone here might be able to assist please.
 
I wanted to add a feature where I can create timers on the Lua side with something like
 
timerRefNum = myAPI.createTimer(1000, myCallbackHandler)     -- 1 sec timer
 
then at the bottom of my Lua script I might have something like
 
while true do
   myApi.sleep(100) -- Lua sleeps for 100 mS while nothing much to do
end
 
I am not the most experienced C# coders, but the outline for this code on the C# side seemed fairly easy.
 
I create a System.Timers.Timer for each Lua createTimer() call and add a c# ElapsedEvent handler which then calls the associated Lua function.
 
After having written the C# code I ran a Lua test script with 2 separate timers firing at 500 mS and 1000 mS intervals, with 2 different Lua handlers.  This appeared to work fine and ran for 15 minutes or so before crashing. :(
 
Of course my solution is fatally flawed, what I overlooked was that the main Lua thread will be executing in Windows thread "x", then timer 1 fires and that C# handler will run in thread "'y", the second timer fires and that runs in thread "z"
 
As Lua / LuaInterface isn't  threadsafe that is what I believe is causing the crash. I am struggling now to see how I would implement this timer functionality properly and threadsafe.
 
Any ideas or links to similar example code snippets would be much appreciated. Thanks
 
Regards Geoff
 
 
 
 
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Tricky c#/LuaInterface Question

Marco Mastropaolo

Sorry for the typos and missing links.. I'm on mobile.

There are several solutions :

1. Use another time class. .NET has a ton of timers most of which dispatch on a single thread in a non reentrant way. They are less precise but easier to work with.

2. Set the synchronization object of the timer to some UI element for automatic dispatching.

3. Use a Dispatcher or a SynchronizationContext to dispatch execution on the thread you want.

4. Set AutoReset to false - this should suffice to work (you have to manually restart the timer though). I would consider using a lock to handle concurrency in a safe way, though.

Note that as far as I know, Lua and LuaInterface will not complain if you use it from different threads as long as it's only one thread at the time.

I might be wrong though.

Shameless plug: you can also check other interface layers for C# and Lua like NLua, SharpLua, MoonSharp, UniLua, and KopiLua. As I'm the author of one of these I listed them in random order but all are good ;) None will help you much in this scenario but should be easier to handle on other aspects.


Il 18/nov/2014 11:25 "Geoff Smith" <[hidden email]> ha scritto:
Hello
 
I have been using LuaInterface for a little while, I have got on pretty well with it so far until I hit this difficult problem. The documentation for it doesn't really help me much on this issue, so hopefully someone here might be able to assist please.
 
I wanted to add a feature where I can create timers on the Lua side with something like
 
timerRefNum = myAPI.createTimer(1000, myCallbackHandler)     -- 1 sec timer
 
then at the bottom of my Lua script I might have something like
 
while true do
   myApi.sleep(100) -- Lua sleeps for 100 mS while nothing much to do
end
 
I am not the most experienced C# coders, but the outline for this code on the C# side seemed fairly easy.
 
I create a System.Timers.Timer for each Lua createTimer() call and add a c# ElapsedEvent handler which then calls the associated Lua function.
 
After having written the C# code I ran a Lua test script with 2 separate timers firing at 500 mS and 1000 mS intervals, with 2 different Lua handlers.  This appeared to work fine and ran for 15 minutes or so before crashing. :(
 
Of course my solution is fatally flawed, what I overlooked was that the main Lua thread will be executing in Windows thread "x", then timer 1 fires and that C# handler will run in thread "'y", the second timer fires and that runs in thread "z"
 
As Lua / LuaInterface isn't  threadsafe that is what I believe is causing the crash. I am struggling now to see how I would implement this timer functionality properly and threadsafe.
 
Any ideas or links to similar example code snippets would be much appreciated. Thanks
 
Regards Geoff
 
 
 
 
 
 
Reply | Threaded
Open this post in threaded view
|

RE: Tricky c#/LuaInterface Question

Geoff Smith
Hi
 
Thanks for the reply. I went with a simple solution of using a mutex, so that when I was executing in the Lua code I take the mutex and then when the script calls sleep and exits back out into the C# sleep function I release the mutex so that event handlers are safe to call into the Lua core again to execute a Lua callback function.
 
I have tested this pretty heavily it seems solid now. I have had 4 fast timers created and running from a Lua script without a crash.
 
As a follow up question, I was maybe hoping in the future I could expand on what I have now to spin up two Windows threads from C# and then execute two entirely separate Lua contexts  (both use LuaInterface of course)
 
I was thinking this should be threadsafe as the start of the task does something like this.  

 lua = new Lua();
lua.RegisterFunction("printToApp", this, GetType().GetMethod("printToApp"));

This idea seems to be flawed though from a quick experiment I did.  Having two threads each create a new Lua instance and register a few simple functions always crashes on execution. Emoji
 
Does anyone have some insight into is it possible to run 2 totally independent threads with LuaInterface ? Any help would be appreciated. Thanks
 
Geoff
 
 

Date: Tue, 18 Nov 2014 12:32:34 +0100
From: [hidden email]
To: [hidden email]
Subject: Re: Tricky c#/LuaInterface Question

Sorry for the typos and missing links.. I'm on mobile.

There are several solutions :

1. Use another time class. .NET has a ton of timers most of which dispatch on a single thread in a non reentrant way. They are less precise but easier to work with.

2. Set the synchronization object of the timer to some UI element for automatic dispatching.

3. Use a Dispatcher or a SynchronizationContext to dispatch execution on the thread you want.

4. Set AutoReset to false - this should suffice to work (you have to manually restart the timer though). I would consider using a lock to handle concurrency in a safe way, though.

Note that as far as I know, Lua and LuaInterface will not complain if you use it from different threads as long as it's only one thread at the time.

I might be wrong though.

Shameless plug: you can also check other interface layers for C# and Lua like NLua, SharpLua, MoonSharp, UniLua, and KopiLua. As I'm the author of one of these I listed them in random order but all are good ;) None will help you much in this scenario but should be easier to handle on other aspects.


Il 18/nov/2014 11:25 "Geoff Smith" <[hidden email]> ha scritto:
Hello
 
I have been using LuaInterface for a little while, I have got on pretty well with it so far until I hit this difficult problem. The documentation for it doesn't really help me much on this issue, so hopefully someone here might be able to assist please.
 
I wanted to add a feature where I can create timers on the Lua side with something like
 
timerRefNum = myAPI.createTimer(1000, myCallbackHandler)     -- 1 sec timer
 
then at the bottom of my Lua script I might have something like
 
while true do
   myApi.sleep(100) -- Lua sleeps for 100 mS while nothing much to do
end
 
I am not the most experienced C# coders, but the outline for this code on the C# side seemed fairly easy.
 
I create a System.Timers.Timer for each Lua createTimer() call and add a c# ElapsedEvent handler which then calls the associated Lua function.
 
After having written the C# code I ran a Lua test script with 2 separate timers firing at 500 mS and 1000 mS intervals, with 2 different Lua handlers.  This appeared to work fine and ran for 15 minutes or so before crashing. :(
 
Of course my solution is fatally flawed, what I overlooked was that the main Lua thread will be executing in Windows thread "x", then timer 1 fires and that C# handler will run in thread "'y", the second timer fires and that runs in thread "z"
 
As Lua / LuaInterface isn't  threadsafe that is what I believe is causing the crash. I am struggling now to see how I would implement this timer functionality properly and threadsafe.
 
Any ideas or links to similar example code snippets would be much appreciated. Thanks
 
Regards Geoff
 
 
 
 
 
 
Reply | Threaded
Open this post in threaded view
|

Re: Tricky c#/LuaInterface Question

Marco Mastropaolo
I'd say upgrading to a different library is the solution, as
LuaInterface seems to be abandoned.

I've detailed your best bets on the previous email, but to give you some
elements to choose from:

I'm the author of MoonSharp and with it, it should work with no problem
whatsoever. As it's a complete reimplementation you might find very
minor differences (and os/io is not there yet); but it should be very
easy to use and includes a debugger. It targets Lua 5.2.

I also think KopiLua and NLua will have no problems with that scenario,
and they are closer to LuaInterface as far as API goes, so you have a
lot less code to change, plus compatibility (w/Lua 5.1) is likely to be
higher.

In both cases beware of sharing objects between the two contexts without
some very careful attention on your part.

I don't know the others in depth and used them only sporadically, but I
don't rule them out to possibly be great choices.

Links:
MoonSharp : http://www.moonsharp.org/
NLua/KopiLua : http://nlua.org/

-- Marco


On 26/11/2014 18:26, Geoff Smith wrote:
> Does anyone have some insight into is it possible to run 2 totally
> independent threads with LuaInterface ? Any help would be appreciated.
> Thanks


Reply | Threaded
Open this post in threaded view
|

RE: Tricky c#/LuaInterface Question

Geoff Smith
Hi Marco
 
Thanks for the response. I have just found this quote from a post on Stackoverflow
 
"LuaInterface is not thread safe. From what I have read Lua itself appears to support multithreading (see Lua Lanes). However, LuaInterface(v2.0.3.7) still has some issues to work out before it is thread safe. Putting separate instances of the Lua interpreter in their own thread does not overcome these issues."
 
So as you say,  it looks like I am screwed if I want to get two scripts running from different threads using LuaInterface.
 
I did look initially at Nlua but documentation is severely lacking, and I couldn't get it to work at all if I remember correctly.
 
Of course the question still applies anyway ? Is Nlua threadsafe when running two Lua instances from two threads.
As it was originally forked from LuaInterface, there is a fair chance it would have the same exact problem ?
 
I have already put quite a bit of time and effort to get where I am using LuaInterface, so swapping horses now wasn't something I really wanted to do. Hmmmm have to ponder this some more.
 
Geoff
 

 
> Date: Wed, 26 Nov 2014 19:28:11 +0100

> From: [hidden email]
> To: [hidden email]
> Subject: Re: Tricky c#/LuaInterface Question
>
> I'd say upgrading to a different library is the solution, as
> LuaInterface seems to be abandoned.
>
> I've detailed your best bets on the previous email, but to give you some
> elements to choose from:
>
> I'm the author of MoonSharp and with it, it should work with no problem
> whatsoever. As it's a complete reimplementation you might find very
> minor differences (and os/io is not there yet); but it should be very
> easy to use and includes a debugger. It targets Lua 5.2.
>
> I also think KopiLua and NLua will have no problems with that scenario,
> and they are closer to LuaInterface as far as API goes, so you have a
> lot less code to change, plus compatibility (w/Lua 5.1) is likely to be
> higher.
>
> In both cases beware of sharing objects between the two contexts without
> some very careful attention on your part.
>
> I don't know the others in depth and used them only sporadically, but I
> don't rule them out to possibly be great choices.
>
> Links:
> MoonSharp : http://www.moonsharp.org/
> NLua/KopiLua : http://nlua.org/
>
> -- Marco
>
>
> On 26/11/2014 18:26, Geoff Smith wrote:
> > Does anyone have some insight into is it possible to run 2 totally
> > independent threads with LuaInterface ? Any help would be appreciated.
> > Thanks
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Tricky c#/LuaInterface Question

DeathByNukes
The linked issue seems to suggest that the problems is LuaBase finalizers being called from the finalizer thread but I don't see how that could be possible because it looks like the finalizer doesn't attempt to free the reference at all.

The biggest threading issue I see in LuaInterface is that the CodeGeneration class is a singleton. (CodeGeneration.Instance)
The fastest hack to fix that would be to make it use thread-local storage instead of global memory:

GenerateEventAssembly.cs line 99
--private static readonly  CodeGeneration instance = new CodeGeneration();
++[ThreadStatic] private static CodeGeneration instance;

GenerateEventAssembly.cs line 123
--return instance;
++return instance ?? (instance = new CodeGeneration());


A more memory-conservative approach would of course be to make all the CodeGeneration public members thread-safe.

On Wed, Nov 26, 2014 at 1:12 PM, Geoff Smith <[hidden email]> wrote:
Hi Marco
 
Thanks for the response. I have just found this quote from a post on Stackoverflow
 
"LuaInterface is not thread safe. From what I have read Lua itself appears to support multithreading (see Lua Lanes). However, LuaInterface(v2.0.3.7) still has some issues to work out before it is thread safe. Putting separate instances of the Lua interpreter in their own thread does not overcome these issues."
 
So as you say,  it looks like I am screwed if I want to get two scripts running from different threads using LuaInterface.
 
I did look initially at Nlua but documentation is severely lacking, and I couldn't get it to work at all if I remember correctly.
 
Of course the question still applies anyway ? Is Nlua threadsafe when running two Lua instances from two threads.
As it was originally forked from LuaInterface, there is a fair chance it would have the same exact problem ?
 
I have already put quite a bit of time and effort to get where I am using LuaInterface, so swapping horses now wasn't something I really wanted to do. Hmmmm have to ponder this some more.
 
Geoff
 

 
> Date: Wed, 26 Nov 2014 19:28:11 +0100
> From: [hidden email]
> To: [hidden email]
> Subject: Re: Tricky c#/LuaInterface Question
>
> I'd say upgrading to a different library is the solution, as

> LuaInterface seems to be abandoned.
>
> I've detailed your best bets on the previous email, but to give you some
> elements to choose from:
>
> I'm the author of MoonSharp and with it, it should work with no problem
> whatsoever. As it's a complete reimplementation you might find very
> minor differences (and os/io is not there yet); but it should be very
> easy to use and includes a debugger. It targets Lua 5.2.
>
> I also think KopiLua and NLua will have no problems with that scenario,
> and they are closer to LuaInterface as far as API goes, so you have a
> lot less code to change, plus compatibility (w/Lua 5.1) is likely to be
> higher.
>
> In both cases beware of sharing objects between the two contexts without
> some very careful attention on your part.
>
> I don't know the others in depth and used them only sporadically, but I
> don't rule them out to possibly be great choices.
>
> Links:
> MoonSharp : http://www.moonsharp.org/
> NLua/KopiLua : http://nlua.org/
>
> -- Marco
>
>
> On 26/11/2014 18:26, Geoff Smith wrote:
> > Does anyone have some insight into is it possible to run 2 totally
> > independent threads with LuaInterface ? Any help would be appreciated.
> > Thanks
>
>