Coroutines & scheduler query

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Coroutines & scheduler query

David Given
I have a project for which I'm going to want a large number of coroutines, all
communicating via a scheduler. Unfortunately, while I'm pretty much up to
speed with the rest of Lua, this isn't an area I've had much experience with.

I remember reading on the list that using a scheduler in this way is not
compatible with using coroutines to invert control flow, for example when
using a producer/consumer system. This seems to be because the scheduler
would have to resume a particular thread with coroutine.resume(), which means
that when that thread yields, expecting to go back to its caller, it'll
instead end up at the scheduler.

Are there any nifty ways round this?

In addition, is there any fundamental difference between coroutine.resume()
and coroutine.yield(), apart from the fact that yield knows which coroutine
to switch to automatically? Can I use resume() to keep switching between two
different coroutines without eating huge quantities of stack space?

--
+- David Given --McQ-+
|  [hidden email]    | "The README of fate is a complex program indeed."
| ([hidden email]) | --- Reboot
+- www.cowlark.com --+

attachment0 (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines & scheduler query

Javier Guerra Giraldez
On Friday 10 February 2006 7:19 am, David Given wrote:
> I remember reading on the list that using a scheduler in this way is not
> compatible with using coroutines to invert control flow, for example when
> using a producer/consumer system. This seems to be because the scheduler
> would have to resume a particular thread with coroutine.resume(), which
> means that when that thread yields, expecting to go back to its caller,
> it'll instead end up at the scheduler.

i think it can be used, but you have to use some caution.  in fact, some of
the core Xavante routines use contron inversion (to give a complex sequence
as an iterator), all under Copas' coroutine scheduler.

--
Javier

attachment0 (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Coroutines & scheduler query

Adrian Sietsma
In reply to this post by David Given
David Given wrote:
> I have a project for which I'm going to want a large number of coroutines, all
> communicating via a scheduler. Unfortunately, while I'm pretty much up to
> speed with the rest of Lua, this isn't an area I've had much experience with.
>
When are these coroutines created ? all at once, or in response to some event ?

> I remember reading on the list that using a scheduler in this way is not
> compatible with using coroutines to invert control flow, for example when
> using a producer/consumer system. This seems to be because the scheduler
> would have to resume a particular thread with coroutine.resume(), which means
> that when that thread yields, expecting to go back to its caller, it'll
> instead end up at the scheduler.
>
Yes, but the scheduler then resumes the "next" coroutine to proceed.

> Are there any nifty ways round this?
>
The following pseudocode is a simplified version of what i'm using for a
http proxy server.
(all the coroutine construction/scheduling is wrapped in the real thing)

-- note : threads is a collection of coroutines to be executed
active_thrd = nil

function startup()
   local thrd = coroutine.create(function() print("func1",active_thrd) end)
   -- schedule for immediate action (push onto threads collection)
   threads.push(thrd,0)

   thrd = coroutine.create(function() print("func2",active_thrd) end)
   threads.push(thrd,0)
   print("yielding")
   threads.push(active_thrd,0) -- re queue myself (vital !)
   coroutine.yield()
   print("i'm back")
   -- do more stuff
return true -- done with this thread


-- schedule the warmup
threads.push(coroutine.create(startup),0)

-- main scheduler
while (true)
    local thrd, args = threads.pop()
    if not thrd then break end
    active_thrd = thrd
    local status, ret, err = coroutine.resume(thrd, unpack(args))
    if not status then -- lua error or assert
       break
    end
end


note that this is co-operative multitasking- a thread will run until it yields.

(i'm converting to lua 5.1 - coroutine.running() removes the need to pass
around the current thread)

> In addition, is there any fundamental difference between coroutine.resume()
> and coroutine.yield(), apart from the fact that yield knows which coroutine
> to switch to automatically?
coroutine.yield() returns control to the caller, which may or may not be a
coroutine.

Can I use resume() to keep switching between two
> different coroutines without eating huge quantities of stack space?
>
I have run the proxy server for long periods without memory leakage or stack
growth.

Adrian