Asp Forum - Re: Thread#raise, Thread#kill, and timeout.rb are unsafe

Yukihiro Matsumoto

3/14/2008 2:43:00 PM

Hi,

In message "Re: Thread#raise, Thread#kill, and timeout.rb are unsafe"
on Fri, 14 Mar 2008 22:32:33 +0900, Paul Brannan <pbrannan@atdesk.com> writes:

|I wonder if there should be an equivalent trap mechanism for other
|asynchronous exceptions?

Probably. Does anyone have any idea?

matz.

11 Answers

Tanaka Akira

3/14/2008 5:49:00 PM

In article <E1JaB7P-0000Zc-QP@x61.netlab.jp>,
Yukihiro Matsumoto <matz@ruby-lang.org> writes:

> Probably. Does anyone have any idea?

The problem is, where is safe point.

You said "just turning on flags to reserve exception, then
raise exception at the safe place, as MRI does."

Your "safe" is for interpreter. Ruby shouldn't SEGV, etc.

But Charles's "safe" is for application. acquired lock
should be released later, etc.

Your "safe" is not enough for application.

We need a mechanism to delay asynchronous exceptions until a
safe point.

trap is bad way to do it. trap is similar to Unix signal
handler. Unix signal handler makes it difficult to avoid
race conditions around blocking operations such as I/O.
With trap, applications need to re-implement the race
condition handling in Ruby level. It is very difficult if
it is possible.

It is preferable to have a way to declare safe points
directly.

The safe points vary according to applications. Even in a
application, different kinds of exceptions may have
different safe points.

For example, there are applications KeyboardInterrupt is
safe as you said. I think a filter, such as grep, is a kind
of the applications. It means that any points in the
applications are safe points.

Another example: movemail is a program to move mbox in a
mail spool. It needs to lock a mbox. Assume we implement
movemail in Ruby and the lock scheme is file lock. The lock
file is removed in ensure clause. But asynchronous
exception in the ensure block may cause to fail removing the
lock file. So the ensure block is not safe points.

Yet another example: Apache can be stopped with apachectl
"stop" and "graceful-stop". "stop" aborts open connections
but "graceful-stop" doesn't. Assume we implement a
threading http server with keep-alive and similar stopping
feature in Ruby. "stop" can be implemented by killing
threads for each connections by (dangerous) Thread.kill. It
means that any points in the threads are safe points.
"graceful-stop" need to wait the threads until they waits
requests. Since keep-alive is assumed, the threads
doesn't terminate after the first request on a connection.
This means the safe points are the request waiting points of
the threads.

I think the safe points declaration mechanism should able to
define safe points for each exception and easy to handle
blocking operations without race conditions.
--
Tanaka Akira

MenTaLguY

3/14/2008 6:54:00 PM

On Fri, 14 Mar 2008 23:42:37 +0900, Yukihiro Matsumoto <matz@ruby-lang.org> wrote:
> Hi,
>
> In message "Re: Thread#raise, Thread#kill, and timeout.rb are unsafe"
> on Fri, 14 Mar 2008 22:32:33 +0900, Paul Brannan <pbrannan@atdesk.com>
> writes:
>
> |I wonder if there should be an equivalent trap mechanism for other
> |asynchronous exceptions?
>
> Probably. Does anyone have any idea?

The simplest way to "tame" asynchronous exceptions would be to enqueue such
exceptions until the recieving thread makes a blocking call, or else it calls
a special method like Thread.check_interrupts.

The main difficulty with the idea is defining clearly what a "blocking call"
is. There probably also needs to be some mechanism (which does not "stop
the world" like Thread.critical) to mask asynchronous exceptions even during
blocking calls, for particularly critical code.

-mental

ara.t.howard

3/18/2008 3:21:00 PM

On Mar 14, 2008, at 8:42 AM, Yukihiro Matsumoto wrote:

> Probably. Does anyone have any idea?

Thread.current.raises do |exception|
queue.push exception
end

....

even_loop do
stuff
if exception = queue.pop
...
end
end

seems like it would address a ton of issued without overly
complicating ruby's internals - just provide an 'on raise' handler.

cheers.

a @ http://codeforp...
--
share your knowledge. it's a way to achieve immortality.
h.h. the 14th dalai lama

Joel VanderWerf

3/18/2008 5:42:00 PM

ara howard wrote:
>
> On Mar 14, 2008, at 8:42 AM, Yukihiro Matsumoto wrote:
>
>> Probably. Does anyone have any idea?
>
>
> Thread.current.raises do |exception|
> queue.push exception
> end
>
> ....
>
> even_loop do
> stuff
> if exception = queue.pop
> ...
> end
> end
>
> seems like it would address a ton of issued without overly complicating
> ruby's internals - just provide an 'on raise' handler.

Is it possible to integrate this with IO#select, so a thread can handle
an exception sent to it while waiting on io?

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

MenTaLguY

3/18/2008 5:52:00 PM

On Wed, 19 Mar 2008 00:20:54 +0900, ara howard <ara.t.howard@gmail.com> wrote:
>> Probably. Does anyone have any idea?
>
>
> Thread.current.raises do |exception|
> queue.push exception
> end
>
> ....
>
> even_loop do
> stuff
> if exception = queue.pop
> ...
> end
> end
>
> seems like it would address a ton of issued without overly
> complicating ruby's internals - just provide an 'on raise' handler.

How do you plan on protecting against non-determinism introduced
by the "on raise" handler? For example, if queue were a
user-implemented data structure, and the handler fired in the middle
of queue.pop? If queue.pop is not protected by a lock, there is
potential to corrupt the queue object. If it is protected by a lock,
there is potential for deadlock.

(This is basically the same issue as asynchronous signal safety.)

I think the only way around this is for the receiving thread to
explicitly choose moments when it wants to receive any pending
asynchronous exceptions. Can you think of another way?

-mental

Roger Pack

3/18/2008 6:16:00 PM

> Thread.current.raises do |exception|
> queue.push exception
> end
>
> ....
>
> event_loop do
> long_running_stuff
> if exception = queue.pop
> ...
> end
> end
>
> seems like it would address a ton of issued without overly
> complicating ruby's internals - just provide an 'on raise' handler.

This looks nice!

I wish that a handler like this could be "unhandled," as well, as you
have the flexibility to handle exceptions only in parts of the code.
Say we call your first block of code as queue_future_exceptions

Then you could have code like

Thread.current.queue_future_exceptions
# code in here is uninterruptible
Thread.current.no_longer_queue_future_exceptions # so I don't have to
worry about them EVERYWHERE in my code--only at the end of certain
blocks.
Thread.current.raises do |exception| ; end

So basically it would allow for uninterruptible blocks. That's what I
wish we had.

In response to Mental's questions of concurrency problems for raises
injected whilst you are in the middle of handling raises, some options
that might help:

1) protect the queue with mutexes--add them to the end of the queue
atomically.
2) dare I say it, require the exception handling to be either within or
without of an "uninterruptible block" --that is, allow the exception
handling to be interrupted (itself), or prevent it from being
interrupted, always, to provide for sanity.

The first option (require handling outside of an uninterruptible block)
would allow for us to "not have to worry" about concurrency of adding to
the exception queue, since we no longer add to the queue but just raise
on the thread. It would allow it to be interrupted, which might be
annoying, however, and isn't our purpose to "avoid" those pesky
asynchronous interrupts?

The second option (requiring handling within an uninterruptible block)
avoids handling's being interrupted, however, it could "miss" an
interrupt or two, should they be raised between when we handle them and
when an "uninterruptible block" ends.

So I guess the best way to deal with it would be to surround queue
addition with mutexes, and handle them both right before an
uninterruptible block ends, and again right after it ends.

Ok so not such a great idea.
Thoughts?
--
Posted via http://www.ruby-....

ara.t.howard

3/18/2008 7:13:00 PM

On Mar 18, 2008, at 11:52 AM, MenTaLguY wrote:

> I think the only way around this is for the receiving thread to
> explicitly choose moments when it wants to receive any pending
> asynchronous exceptions. Can you think of another way?

perhaps i wasn't being clear. that's exactly what i was showing - a
handler that gets called (critical section of course) which simply
enques the exception. then the thread itself must check for these at
the 'right' moment.

a @ http://codeforp...
--
it is not enough to be compassionate. you must act.
h.h. the 14th dalai lama

MenTaLguY

3/18/2008 9:23:00 PM

On Wed, 19 Mar 2008 04:13:19 +0900, ara howard <ara.t.howard@gmail.com> wrote:
> On Mar 18, 2008, at 11:52 AM, MenTaLguY wrote:
>
>> I think the only way around this is for the receiving thread to
>> explicitly choose moments when it wants to receive any pending
>> asynchronous exceptions. Can you think of another way?
>
> perhaps i wasn't being clear. that's exactly what i was showing - a
> handler that gets called (critical section of course) which simply
> enques the exception. then the thread itself must check for these at
> the 'right' moment.

Okay. I'd thought you were suggesting an API rather than illustrating
an idea.

-mental

MenTaLguY

3/18/2008 9:37:00 PM

On Wed, 19 Mar 2008 03:16:05 +0900, Roger Pack <rogerpack2005@gmail.com> wrote:
> In response to Mental's questions of concurrency problems for raises
> injected whilst you are in the middle of handling raises, some options
> that might help:
>
> 1) protect the queue with mutexes--add them to the end of the queue
> atomically.

This would deadlock if the handler happened to fire while the queue's
lock was held. Generally, locking doesn't mix with anything that can
interrupt or suspend the execution of a thread at an arbitrary time.

> 2) dare I say it, require the exception handling to be either within or
> without of an "uninterruptible block" --that is, allow the exception
> handling to be interrupted (itself), or prevent it from being
> interrupted, always, to provide for sanity.

That's a good point -- besides conflicts between the handler and whatever
the recipient thread is running at the time, you do also need to be
concerned with conflicts between multiple handler instances (if that
is allowed).

> So I guess the best way to deal with it would be to surround queue
> addition with mutexes, and handle them both right before an
> uninterruptible block ends, and again right after it ends.
>
> Ok so not such a great idea.

In terms of an API for signal queuing, I think Tanaka Akira's idea
is elegant and reliable.

-mental

Charles Oliver Nutter

3/22/2008 6:04:00 AM

ara howard wrote:
>
> On Mar 18, 2008, at 11:52 AM, MenTaLguY wrote:
>
>> I think the only way around this is for the receiving thread to
>> explicitly choose moments when it wants to receive any pending
>> asynchronous exceptions. Can you think of another way?
>
>
>
> perhaps i wasn't being clear. that's exactly what i was showing - a
> handler that gets called (critical section of course) which simply
> enques the exception. then the thread itself must check for these at
> the 'right' moment.

Presumably by "critical section" you mean "locking on as narrow a lock
as possible so as not to stop the whole freaking world because one
thread wants to send an event to another". critical= is the devil.

- Charlie

comp.lang.ruby

Re: Thread#raise, Thread#kill, and timeout.rb are unsafe

Yukihiro Matsumoto

Tanaka Akira

MenTaLguY

ara.t.howard

Joel VanderWerf

MenTaLguY

Roger Pack

ara.t.howard

MenTaLguY

MenTaLguY

Charles Oliver Nutter

x Login to ForumsZone