Safely adding coroutine.yield() to Lua code string

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Safely adding coroutine.yield() to Lua code string

Abhijit Nandy
Hi,

I am developing a strategy game about a small village with some people in it :) 
I allow the behavior of people to be coded in Lua, making different facts about the world available to the code. The behaviors can get quite complex and take a lot of time to execute. I need to time-bound the execution.

So I want to run the behavior code in a coroutine and force a yield after some time. But users who write the behavior lua code may not add the coroutine.yield() lines at proper intervals. Therefore I want to add them.

Has anyone found a way to safely detect a place in a Lua code string, that's not inside of some literal string or in the middle of any other syntax, where coroutine.yield() will be valid, without breaking the original code?

Of course the yield() may still land up in a place just after a long running single line loop. I am not sure how to handle that yet.
Any other way to make the interpreter safely yield would be great too. Has anyone tried to use the debug library to make the interpreter yield after a specific amount of time, by using the line callbacks to track the time?

Thanks,
Abhijit
Reply | Threaded
Open this post in threaded view
|

Re: Safely adding coroutine.yield() to Lua code string

Philippe Verdy-2
You can't do that without using a true Lua parser (there are recipes on the wiki about how to parse Lua using LPeg).
anyway it's difficult to find places where you should yield to force a preemption: a basic but very short "for/while/repeat" loop with yield would turn very slow. And some functions calls can be also very long (notably those that perform network or database requests, if its code is outside the edited Lua code.
The only thing you can do is to have these Lua codes executed in a true preemptive thread (or a separate process but cost in memory is huge), or having your game engine create separate Lua instances in a native thread (which could be much faster and allows implementing limits on resource usage, such as memory, each instance having their quotas). And if one of these threads isolated in a separate Lua instance is taking too long, you can kill it with the Lua instance. Then have these Lua instances communicate their input or result with the rest of your game via an API.
Fur such app, your user-ediable Lua code should be a single function call where the game provides some parameters, and then returns the values. This function could still call several APIs of your game app and it is the implementation of this API that will enforce some rules for allowing other components to run.
Preemptive threads can also be considered like coroutines, execpt that these coroutines return a variant type: the exepted type, or nil and some "error" object where the error is a timing event indicating that the function has still not yielded its actual result: test this condition to know if the function can be reentered.
I wonder why coroutines in Lua do not standardize a way to "call" a thread with a flag indicating that that thread can be preempted and return early with such time-based condition (or other preemptive condition like maximum resource usage) returning variant types would be the way to go.


Le dim. 21 juin 2020 à 14:28, Abhijit Nandy <[hidden email]> a écrit :
Hi,

I am developing a strategy game about a small village with some people in it :) 
I allow the behavior of people to be coded in Lua, making different facts about the world available to the code. The behaviors can get quite complex and take a lot of time to execute. I need to time-bound the execution.

So I want to run the behavior code in a coroutine and force a yield after some time. But users who write the behavior lua code may not add the coroutine.yield() lines at proper intervals. Therefore I want to add them.

Has anyone found a way to safely detect a place in a Lua code string, that's not inside of some literal string or in the middle of any other syntax, where coroutine.yield() will be valid, without breaking the original code?

Of course the yield() may still land up in a place just after a long running single line loop. I am not sure how to handle that yet.
Any other way to make the interpreter safely yield would be great too. Has anyone tried to use the debug library to make the interpreter yield after a specific amount of time, by using the line callbacks to track the time?

Thanks,
Abhijit
Reply | Threaded
Open this post in threaded view
|

Re: Safely adding coroutine.yield() to Lua code string

Andrew Gierth
In reply to this post by Abhijit Nandy
>>>>> "Abhijit" == Abhijit Nandy <[hidden email]> writes:

 Abhijit> Any other way to make the interpreter safely yield would be
 Abhijit> great too. Has anyone tried to use the debug library to make
 Abhijit> the interpreter yield after a specific amount of time, by
 Abhijit> using the line callbacks to track the time?

Notice that using lua_yield() from a line hook or (better) a count hook
is explicitly documented, so this would be the first method I'd try.

--
Andrew.