Is there an easier way to cancel threads???

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Is there an easier way to cancel threads???

Diego Nehab-3
Hi,

For this discussion, assume you have a pointer to a mutex
per lua_State, and that we are using pthreads (to
keep things simple I will cry about Windows later).

The shared lua_State mutex is created on a call to lua_open,
and all child lua_States (created with lua_newthread) point
to that same mutex.

This mutex is used to ensure exclusive access to the
lua_State. Accordingly, at a given moment, only one thread
will be running Lua code (or pretty much any Lua API
function, for that matter). Other threads will be blocked
attempting to acquire this mutex, at system calls, or at
other mutexes or condition variables that can be created
and manipulated from Lua code.

Now one of these threads ends up causing lua_close to be
called. Say the main thread decides to quit, for example.

Inside lua_close, the main thread will acquire the lua_State
lock. It will then proceed to destroy everything that was
allocated.

How do we prevent hell from being unleashed?

Child threads currently blocked on cancelation points (such
as those waiting on condition variables, or calling to
read()) can be cancelled one by one from inside lua_close,
as long as we keep a list of created threads.

But what about those blocked on the shared lua_State mutex
itself?  And what about those blocked on some other mutex?

We are not allowed to destroy locked mutexes. We would have
to somehow beg these other threads to unlock whatever they
had locked and politely quit. This can be done by placing
all threads in the assynchronous cancelation state at creation
time, pushing one cancelation handler for each mutex a
thread locks, then looping over all child threads canceling
them and joining them to wait until they terminate naturally.

One might think we would be able to get the __gc metamethod of
each mutex to cancel the thread that has it locked and wait
until that thread terminates, or directly unlock it if that
thread is the same as the one running the __gc metamethod.
But if any thread has more than one mutex locked, we would
it would exit without unlocking the others. We would need
the cancelation handlers anyways...

So, is there really no other way?

BTW, it looks like I can set the __gc metamethod for
threads. If this is really the case, I could use it instead
of keeping a list of threads myself. But still, this looks
like a complicated solution.

Regards,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Mike Pall-4-2
Hi,

Diego Nehab wrote:
> How do we prevent hell from being unleashed?

You can't. None of the pre-emptive threading libraries solve this
problem. Because there is no general solution given the degrees
of freedom for the programmer these libraries provide.

LuaThreads exposes enough of the underlying functionality to make
it suffer from the same problem.

So turn the argument around: exiting the main thread without
cleaning up all other threads is a programmer error. Document it
as such and be done.

[
  Optionally detect this case and call a thread.panic function.
  The default action would be an error message and _exit(1).

  Well, this would need two macros then: 1. luai_userstateclose()
  where it currently is (before all other threads are freed) and
  2. a new luai_userstatecleanup() [name?] after luaC_freeall().

  Pseudo-code:

  luai_userstateclose()
  {
    if (#threads > 0) {  // The main thread is not counted.
      do full GC
      do full GC
      if (#threads > 0) {
        call panic function
      }
    }
  }

  luai_userstatecleanup()  // Not to be called for lua_newstate failure.
  {
    unlock mutex
    destroy mutex
  }
]

[Now you know why I'm so fond about non-pre-emptive approaches.]

> BTW, it looks like I can set the __gc metamethod for
> threads. If this is really the case, I could use it instead
> of keeping a list of threads myself. But still, this looks
> like a complicated solution.

You can set __gc for every object. But it will only be called for
userdata objects. The code above only needs a thread counter.

Bye,
     Mike
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
Hi,

Safely canceling threads Windows is completely impossible.
So we can't hope to cancel child threads in a portable way
when one of them chooses to call lua_close. Using pthreads,
it seems this could be accomplished. But if it doesn't work
on Windows, I am not going to bother doing it. We need a
compromise, as you said.

> So turn the argument around: exiting the main thread
> without cleaning up all other threads is a programmer
> error. Document it as such and be done.

The problem is that Lua libraries should shield Lua code
from causing crashes or disrupting the system in any way.

Perhaps what we should do is wait for all threads to exit
naturaly, whenever lua_close is called. This can be
accomplished both on pthreads and Windows.

That way, the user can safely call lua_close and watch his
application wait until all threads choose to leave, or hang
on a dead-lock.

Dead-locks are fine. :) All you need is a couple mutexes to write
one. Adding this possibility to lua_close seems to be an
acceptable comprise.

> [Now you know why I'm so fond about non-pre-emptive approaches.]

Dude, it's not like LuaThread is my favourite child. :) I
don't know why people like threads, but there seems to be a
demand for thread support.

> You can set __gc for every object. But it will only be called for
> userdata objects. The code above only needs a thread counter.

We can use a condition variable and a counter to make sure
we only exit lua_close when all threads are done. Is this a
good solution? I think it is ok.

Regards,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Doug Rogers-4
Diego Nehab wrote:

>> [Now you know why I'm so fond about non-pre-emptive approaches.]
> Dude, it's not like LuaThread is my favourite child. :) I
> don't know why people like threads...

I have come to the conclusion that preemptive threading was created not
to support blocking calls but instead to support computer science
master's degrees!

> We can use a condition variable and a counter to make sure
> we only exit lua_close when all threads are done. Is this a
> good solution? I think it is ok.

That's what I did in my own threading library. My 'tasks' use post() and
read() in a way similar to resume() and yield(). Each task has a pthread
condition variable + mutex in the C backend. When it dies it signals the
main counting semaphore for the tasking library. I never linked it in
with lua_close(), though. That would have been smart. I'm not terribly
satisfied with my approach, but it worked for a few fun cases.

The next time I think I need preemptive threading in Lua I will use
someone else's better design.

Doug

--
--__-__-____------_--_-_-_-___-___-____-_--_-___--____
Doug Rogers - ICI - V:703.893.2007x220 www.innocon.com
-_-_--_------____-_-_-___-_--___-_-___-_-_---_--_-__-_
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
Hi,

> I have come to the conclusion that preemptive threading was created not
> to support blocking calls but instead to support computer science
> master's degrees!

I have the exact same opinion. I envision LuaThread being
used so kids can learn how to program multithreaded
applications. Then, when they grow up, they can go
assynchronous instead. :)

>> We can use a condition variable and a counter to make sure
>> we only exit lua_close when all threads are done. Is this a
>> good solution? I think it is ok.
>
> That's what I did in my own threading library.

That's good. I am about to settle for that.

> The next time I think I need preemptive threading in Lua I will use
> someone else's better design.

I wonder when that will happen... :)

[]s,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Mark Hamburg-4
In reply to this post by Diego Nehab-3
As a plug for the new Programming in Lua, I'll note that it has a nice
little multiple-universe threading system without any of these problems. One
would probably need more in a production system, but there's enough to get
one started.

Mark

Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Daniel Quintela
In reply to this post by Diego Nehab-3
Extracted from ftp://sources.redhat.com/pub/pthreads-win32/README:

"...
QueueUserAPCEx by Panagiotis E. Hadjidoukas
    For true async cancelation of threads (including blocked threads).
    This is a DLL and Windows driver that provides pre-emptive APC by
forcing threads into an alertable state when the APC is queued.
    Both the DLL and driver are provided with the pthreads-win32.exe
self-unpacking ZIP, and on the pthreads-win32 FTP site (in source and
pre-built forms). Currently this is a separate LGPL package to
pthreads-win32. See the README in the QueueUserAPCEx folder for
installation instructions.

    Pthreads-win32 will automatically detect if the QueueUserAPCEx DLL
QuserEx.DLL is available and whether the driver AlertDrv.sys is loaded.
If it is not available, pthreads-win32 will simulate async cancelation,
which means that it can async cancel only threads that are runnable. The
simulated async cancellation cannot cancel blocked threads.
..."

Regards,
Daniel


Diego Nehab escribió:

> Hi,
>
> Safely canceling threads Windows is completely impossible. So we can't
> hope to cancel child threads in a portable way
> when one of them chooses to call lua_close. Using pthreads,
> it seems this could be accomplished. But if it doesn't work
> on Windows, I am not going to bother doing it. We need a
> compromise, as you said.
>
>> So turn the argument around: exiting the main thread
>> without cleaning up all other threads is a programmer
>> error. Document it as such and be done.
>
>
> The problem is that Lua libraries should shield Lua code
> from causing crashes or disrupting the system in any way.
>
> Perhaps what we should do is wait for all threads to exit
> naturaly, whenever lua_close is called. This can be
> accomplished both on pthreads and Windows.
>
> That way, the user can safely call lua_close and watch his
> application wait until all threads choose to leave, or hang
> on a dead-lock.
>
> Dead-locks are fine. :) All you need is a couple mutexes to write
> one. Adding this possibility to lua_close seems to be an
> acceptable comprise.
>
>> [Now you know why I'm so fond about non-pre-emptive approaches.]
>
>
> Dude, it's not like LuaThread is my favourite child. :) I
> don't know why people like threads, but there seems to be a
> demand for thread support.
>
>> You can set __gc for every object. But it will only be called for
>> userdata objects. The code above only needs a thread counter.
>
>
> We can use a condition variable and a counter to make sure
> we only exit lua_close when all threads are done. Is this a
> good solution? I think it is ok.
>
> Regards,
> Diego.


Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
Hi,

> Pthreads-win32 will automatically detect if the
> QueueUserAPCEx DLL QuserEx.DLL is available and whether
> the driver AlertDrv.sys is loaded. If it is not
> available, pthreads-win32 will simulate async
> cancelation, which means that it can async cancel only
> threads that are runnable. The simulated async
> cancellation cannot cancel blocked threads.  ..."

<flame>
>From your message, I can't tell if you sent this to agree
with me that it is impossible to cancel a thread on Windows,
or to disagree. I believe this paragraph agrees with my
point.
</flame>

Regards,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Daniel Quintela
I'm sorry...  :-(  I only wanted to point you about the availability of
pthreads with real cancellation on Windows.

Is Windows capable to cancel threads? : no.
Did somebody make a workaround for that?: yes, pthreads-win32+QueueUserAPCEx

Regards,
Daniel


Diego Nehab escribió:

> Hi,
>
>> Pthreads-win32 will automatically detect if the
>> QueueUserAPCEx DLL QuserEx.DLL is available and whether
>> the driver AlertDrv.sys is loaded. If it is not
>> available, pthreads-win32 will simulate async
>> cancelation, which means that it can async cancel only
>> threads that are runnable. The simulated async
>> cancellation cannot cancel blocked threads.  ..."
>
>
> <flame>
>
>> From your message, I can't tell if you sent this to agree
>
> with me that it is impossible to cancel a thread on Windows,
> or to disagree. I believe this paragraph agrees with my
> point.
> </flame>
>
> Regards,
> Diego.


Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
Hi,

> I'm sorry...  :-(  I only wanted to point you about the availability of
> pthreads with real cancellation on Windows.
>
> Is Windows capable to cancel threads? : no.
> Did somebody make a workaround for that?: yes, pthreads-win32+QueueUserAPCEx

I am not mad or anything! It was supposed to be a joke. :) I
am sorry if I sounded arrogant. I had already looked into
pthread-win32.  It falls in the category of the cygwin
select kludge.

I try to rephrase this politely: there is no way I am using
this disgusting hack! Oops. :)

These things only make sense when you are porting some
monstruous software from unix to win32 and must absolutely
have the same behavior on both systems.

Since there is no large LuaThread codebase, we can simply
change the semantics of the library so it is possible to
implement without sacrificing simplicity.

[]s,
Diego.

Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

David Burgess
In reply to this post by Daniel Quintela
I have used QueueUserAPC() in C++ thread classes,
it works well in combination with Overlapped IO and
SleepEx(). This does not completely solve the
problem, but it goes close.

DB
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Daniel Quintela
In reply to this post by Diego Nehab-3
Diego Nehab escribió:

> Hi,
>
> I am not mad or anything! It was supposed to be a joke.

:-)

> These things only make sense when you are porting some
> monstruous software from unix to win32 and must absolutely
> have the same behavior on both systems.

Are you considering LuaTask a "monstruous software"?  (my joke)

>
> Since there is no large LuaThread codebase, we can simply
> change the semantics of the library so it is possible to
> implement without sacrificing simplicity.
>
I see your point.

Regards,
Daniel

Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

David Burgess
In reply to this post by David Burgess
BTB, the Java thread class model (I cant remember what it is called)
and the Borland C++ classlibs thread.h (which are strikingly similar)
have a rather good object model for thread usage. Both Win32 and
pthreads would fit into this structure.

DB
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
In reply to this post by David Burgess
Hi,

> I have used QueueUserAPC() in C++ thread classes,
> it works well in combination with Overlapped IO and
> SleepEx(). This does not completely solve the
> problem, but it goes close.

I agree. We would still have a problem with threads that are
blocked on system calls, such as recv/send. Those would not
awake with QueueUserAPC. I am not sure what we would gain
with it.

Regards,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

David Burgess
Diego Nehab wrote:

>
> > I have used QueueUserAPC() in C++ thread classes,
> > it works well in combination with Overlapped IO and
> > SleepEx(). This does not completely solve the
> > problem, but it goes close.
>
> I agree. We would still have a problem with threads that are
> blocked on system calls, such as recv/send. Those would not
> awake with QueueUserAPC.

Unless they are overlapped.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
Hi,

>> > I have used QueueUserAPC() in C++ thread classes,
>> > it works well in combination with Overlapped IO and
>> > SleepEx(). This does not completely solve the
>> > problem, but it goes close.
>>
>> I agree. We would still have a problem with threads that are
>> blocked on system calls, such as recv/send. Those would not
>> awake with QueueUserAPC.
>
> Unless they are overlapped.

:) Right, but then they are not blocking... :)

[]s,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

David Burgess
In reply to this post by Doug Rogers-4
A little off topic. My apologies in advance.

Doug Rogers wrote:

> Diego Nehab wrote:
>
> >> [Now you know why I'm so fond about non-pre-emptive approaches.]
> > Dude, it's not like LuaThread is my favourite child. :) I
> > don't know why people like threads...
>
> I have come to the conclusion that preemptive threading was created not
> to support blocking calls but instead to support computer science
> master's degrees!
>

I am sure you will be requoted many times. IMHO the way around the
thread problem is have the OS handle the threading for you. IO completion
ports and overlapped IO do this on Win32. For once MS almost got it right.
There is, I believe, a few *nix implementations of IOCPs.
Anyways, the IOCP model should not be tough to emulate on *nix and
it solves a *lot* of thread/mutex nightmares (tremble/shake).

DB
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
Hi,

> I am sure you will be requoted many times. IMHO the way around the
> thread problem is have the OS handle the threading for you. IO completion
> ports and overlapped IO do this on Win32. For once MS almost got it right.
> There is, I believe, a few *nix implementations of IOCPs.
> Anyways, the IOCP model should not be tough to emulate on *nix and
> it solves a *lot* of thread/mutex nightmares (tremble/shake).

I agree with that, and this is the direction we will take in
writing LuaSocket 3.0. We will try to come up with a nice
common interface to AIO what will work both on *nix and
Win32.

However, there will always be someone asking: "Why doesn't
Lua have multi-threading support?"

Something tells me that trying to convince that person not
to use threads will cause him/her to use Python. :/

[]s,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

Diego Nehab-3
In reply to this post by Daniel Quintela
Hi,

> Are you considering LuaTask a "monstruous software"?  (my joke)

Well... if you force your users to install a kernel mode
driver, then yes!!! :)

[]s,
Diego.
Reply | Threaded
Open this post in threaded view
|

Re: Is there an easier way to cancel threads???

David Burgess
In reply to this post by Diego Nehab-3
Diego Nehab wrote:
> However, there will always be someone asking: "Why doesn't
> Lua have multi-threading support?"
>
> Something tells me that trying to convince that person not
> to use threads will cause him/her to use Python. :/
>
For Lua, I think the right answer belongs to David Given(if I remember
correctly). This means a lua state per OS thread and YAN thread
for communication between the states. The overhead is the xtra
memory for each state (but who cares it will be less than python)
and the data transfer between states (which has to happen anyway).
This model has one mutex for transferring data between states.
Above all this model is simple.

DB
12