Asp Forum - How to detect blocking?

David Masover

9/11/2008 4:48:00 AM

I've been (barely, occasionally) writing an Actor system for Ruby.

It was actually pretty easy, when I was doing one thread per actor, but that
has some serious drawbacks -- among them that threads can't be garbage
collected (can they?), whereas actors really should be collectable.

So now I'm considering running a smaller number of threads -- at least one per
usable CPU (so at least one on mainline Ruby, more on JRuby) -- but more,
since operations might block.

The problem is, how do I detect a blocked thread, and execute some code when
that happens?

The best I can think of is to poll, checking for #stop?, but that's hackish,
and performance would be awful. Poll too often, and I burn CPU -- not often
enough, and I end up with large windows of my app waiting for the poll thread
to wake up.

15 Answers

Robert Klemme

9/11/2008 6:39:00 AM

On 11.09.2008 06:48, David Masover wrote:
> I've been (barely, occasionally) writing an Actor system for Ruby.
>
> It was actually pretty easy, when I was doing one thread per actor, but that
> has some serious drawbacks -- among them that threads can't be garbage
> collected (can they?), whereas actors really should be collectable.
>
> So now I'm considering running a smaller number of threads -- at least one per
> usable CPU (so at least one on mainline Ruby, more on JRuby) -- but more,
> since operations might block.
>
> The problem is, how do I detect a blocked thread, and execute some code when
> that happens?

Block on what? IO? In that case you probably want to use more threads
as Ruby does handle this nicely.

> The best I can think of is to poll, checking for #stop?, but that's hackish,
> and performance would be awful. Poll too often, and I burn CPU -- not often
> enough, and I end up with large windows of my app waiting for the poll thread
> to wake up.

I do not have a clear idea of your app but usually with multiple threads
you use some form of synchronization (blocking queues, condition
variables etc.). HTH

Kind regards

robert

David Masover

9/12/2008 3:36:00 AM

On Thursday 11 September 2008 01:33:25 Robert Klemme wrote:
> On 11.09.2008 06:48, David Masover wrote:

> > The problem is, how do I detect a blocked thread, and execute some code
when
> > that happens?
>
> Block on what? IO?

Mostly, yes, but really any blocking operation. Could be a mutex. Could be a
sleep call. Could be it was deliberately suspended.

In other words, anything that would cause Thread#stop? to return true.

> In that case you probably want to use more threads
> as Ruby does handle this nicely.

Maybe not as nicely as I'd like, though -- I imagine 1.9's OS threads are
heavier. And it presents problems with garbage-collection, with my current
design.

> > The best I can think of is to poll, checking for #stop?, but that's
hackish,
> > and performance would be awful. Poll too often, and I burn CPU -- not
often
> > enough, and I end up with large windows of my app waiting for the poll
thread
> > to wake up.
>
> I do not have a clear idea of your app

It's an actor library, which means apps will be written for it.
(Maybe. I might still drop it and just use Dramatis.)

> but usually with multiple threads
> you use some form of synchronization (blocking queues, condition
> variables etc.).

Right. An actor library is designed to abstract all of that away.

Let me phrase it differently -- it's like having a hundred thousand fibers
that I want to execute at least once. I don't need to spawn a hundred
thousand threads -- and I can't, I remember the limit being around three
thousand in 1.9, and I really only need one or two.

But if one of those fibers blocks somewhere, I want to spawn another thread.
Ideally, I need N + S threads, where N is the number of CPUs my program can
use concurrently, and S is the number of other threads which are stopped.

That's oversimplifying a bit -- but if you want to know more, read about
Erlang's process model. I'm trying to do something like that. The technical
term for that is the "actor model".

David Masover

9/13/2008 8:59:00 PM

Suppose I start a worker thread, like so:

require 'thread'
queue = Queue.new

Thread.new do
loop do
# Suppose this does something useful...
puts queue.pop
end
end

And I start sending things to that queue:

queue.push 'Hello, world!'
queue.push '(from inside the thread)'

...and so on.

Now, at some point, the queue will fall out of scope -- at which point, I'm
obviously not going to push anything onto it. It's also the only way to talk
to the thread, so the thread isn't going to be doing anything useful at this
point.

I'd like to collect that thread, but, of course, that's really impossible.
After all, the loop will never end, so the thread will never exit normally.
Without solving the Halting Problem, Ruby can't know that the thread won't do
anything useful, even if nothing else refers to it -- maybe it will push
something onto that queue, after all.

Now, I know the logical solution to this particular problem:

Thread.new do
until (item = queue.pop).nil?
puts item
end
end

But then, I have to remember to send that message just before the queue goes
out of scope:

queue.push nil

Is there any way to handle this gracefully? For example: Any way to tell that
the queue has gone out of scope, there? Or any way to fire any code right
before an object is collected?

Or do I always have to remember to kill that thread explicitly, somehow -- by
sending that nil, for example?

Robert Klemme

9/14/2008 8:28:00 AM

On 13.09.2008 22:59, David Masover wrote:
> Suppose I start a worker thread, like so:
>
> require 'thread'
> queue = Queue.new
>
> Thread.new do
> loop do
> # Suppose this does something useful...
> puts queue.pop
> end
> end
>
> And I start sending things to that queue:
>
> queue.push 'Hello, world!'
> queue.push '(from inside the thread)'
>
> ..and so on.
>
> Now, at some point, the queue will fall out of scope -- at which point, I'm
> obviously not going to push anything onto it. It's also the only way to talk
> to the thread, so the thread isn't going to be doing anything useful at this
> point.
>
> I'd like to collect that thread, but, of course, that's really impossible.
> After all, the loop will never end, so the thread will never exit normally.
> Without solving the Halting Problem, Ruby can't know that the thread won't do
> anything useful, even if nothing else refers to it -- maybe it will push
> something onto that queue, after all.
>
> Now, I know the logical solution to this particular problem:
>
> Thread.new do
> until (item = queue.pop).nil?
> puts item
> end
> end
>
> But then, I have to remember to send that message just before the queue goes
> out of scope:
>
> queue.push nil
>
> Is there any way to handle this gracefully? For example: Any way to tell that
> the queue has gone out of scope, there? Or any way to fire any code right
> before an object is collected?
>
> Or do I always have to remember to kill that thread explicitly, somehow -- by
> sending that nil, for example?

The usual solution is that the queue will not "go out of scope" before
the thread dies. In a farmer worker scenario (or however you call it)
the farmer distributes work and is also often responsible for collecting
results. If there is no work left he needs to notify workers of this
fact so they can react appropriately (die).

If I implement such a scenario I define a terminator (you used nil for
that) that is sent n times down the queue (n = # threads). Then the
main thread joins all threads and that way you can be sure that all work
is done when main thread exits.

Kind regards

robert

David Masover

9/14/2008 5:27:00 PM

On Sunday 14 September 2008 03:24:53 Robert Klemme wrote:

> The usual solution is that the queue will not "go out of scope" before
> the thread dies.

Right.

> In a farmer worker scenario (or however you call it)
> the farmer distributes work and is also often responsible for collecting
> results. If there is no work left he needs to notify workers of this
> fact so they can react appropriately (die).

I'd like to detect "no work left" as a case of the "farmer" going out of
scope, if possible.

So, it sounds like you're telling me all the "usual" ways of doing this.
Is there no way to make my suggestion work?

Robert Klemme

9/14/2008 8:29:00 PM

On 14.09.2008 19:27, David Masover wrote:
> On Sunday 14 September 2008 03:24:53 Robert Klemme wrote:
>> In a farmer worker scenario (or however you call it)
>> the farmer distributes work and is also often responsible for collecting
>> results. If there is no work left he needs to notify workers of this
>> fact so they can react appropriately (die).
>
> I'd like to detect "no work left" as a case of the "farmer" going out of
> scope, if possible.

There is no hook in Ruby that is called when the scope of a variable
ends. An object without any references can be garbage collected. But a
finalizer won't work because then the object is gone already. Also, the
queue that connects farmer and workers will still have references even
if the farmer is gone, so the finalizer will never be called...

> So, it sounds like you're telling me all the "usual" ways of doing this.

What's wrong with using the usual ways?

> Is there no way to make my suggestion work?

You can use begin ensure end. This will guarantee that action will be
taken even in case of an exception. If that's what you're after.

Cheers

robert

David Masover

9/14/2008 8:34:00 PM

On Sunday 14 September 2008 15:22:53 Robert Klemme wrote:

> There is no hook in Ruby that is called when the scope of a variable
> ends. An object without any references can be garbage collected. But a
> finalizer won't work because then the object is gone already.

What is a "finalizer" in this context?

> Also, the
> queue that connects farmer and workers will still have references even
> if the farmer is gone, so the finalizer will never be called...

Is it possible to have weak references in Ruby?

> > So, it sounds like you're telling me all the "usual" ways of doing this.
>
> What's wrong with using the usual ways?

The usual ways still force me to explicitly clean up after my worker threads.
This is irritating, philosophically -- if I wanted to do manual resource
management, I'd write C++, not Ruby.

It also means that if I'm implementing any kind of library on top of that, I
have to force all my users to do the same manual cleanup.

Robert Klemme

9/15/2008 8:55:00 AM

2008/9/14 David Masover <ninja@slaphack.com>:
> On Sunday 14 September 2008 15:22:53 Robert Klemme wrote:
>
>> There is no hook in Ruby that is called when the scope of a variable
>> ends. An object without any references can be garbage collected. But a
>> finalizer won't work because then the object is gone already.
>
> What is a "finalizer" in this context?

http://www.ruby-doc.org/core/classes/ObjectSpace.ht...

>> Also, the
>> queue that connects farmer and workers will still have references even
>> if the farmer is gone, so the finalizer will never be called...
>
> Is it possible to have weak references in Ruby?

http://www.ruby-doc.org/core/classes/We...

>> > So, it sounds like you're telling me all the "usual" ways of doing this.
>>
>> What's wrong with using the usual ways?
>
> The usual ways still force me to explicitly clean up after my worker threads.
> This is irritating, philosophically -- if I wanted to do manual resource
> management, I'd write C++, not Ruby.

You have to close file handles as well (even if you use the block form
of File.open you take care of this in a way).

> It also means that if I'm implementing any kind of library on top of that, I
> have to force all my users to do the same manual cleanup.

No, because you can make that part of the library. At least you can
built your library in a way that it will invoke a cleanup hook which
you define as empty method for application specific cleanup so users
of the lib do not have to but can add cleanup functionality. Cleanup
of the framework is done automatically of course. Note that there are
already libraries that abstract such a scenario.

Cheers

robert

--
use.inject do |as, often| as.you_can - without end

David Masover

9/16/2008 3:22:00 AM

On Monday 15 September 2008 03:55:06 Robert Klemme wrote:
> 2008/9/14 David Masover <ninja@slaphack.com>:

> > What is a "finalizer" in this context?
>
> http://www.ruby-doc.org/core/classes/ObjectSpace.ht...

Cool.

Can they be lambdas/closures? If so, I think I've got what I need -- it's just
going to be very tricky to get right.

> > Is it possible to have weak references in Ruby?
>
> http://www.ruby-doc.org/core/classes/We...

Hmm. It mentions WeakRef::RefError. While I'm at it -- is it safe to poke at
WeakRefs to see if they're still alive?

(That is: If the object is gone, will I always get a RefError, or is it ever
possible to get something undefined or bad/wrong?)

> You have to close file handles as well (even if you use the block form
> of File.open you take care of this in a way).

The block form of File.open is a good example, though, and it's also an API
which is tolerable, in the common case.

But even here, it might be useful to wrap it in some sort of
garbage-collectable object -- assuming collection run on every single object
when the program exits.

To be usable, of course, you'd have to force collection to occur when running
out of file descriptors, in case one is being held open by an object that
hasn't been collected... right?

> > It also means that if I'm implementing any kind of library on top of that,
I
> > have to force all my users to do the same manual cleanup.
>
> No, because you can make that part of the library. At least you can
> built your library in a way that it will invoke a cleanup hook which
> you define as empty method for application specific cleanup so users
> of the lib do not have to but can add cleanup functionality. Cleanup
> of the framework is done automatically of course.

The problem is, with the API that it exposes, I don't know when to have the
framework cleanup, other than when a particular object has no more strong
references to it. I'm not worried about the app-specific cleanup.

The idea is: Apps can be built as they always were, just some objects are
really actors (object + thread). So, where an app would let an object fall
out of scope (and thus be collected), it can also let an actor fall out of
scope (and thus be collected, and have the thread closed, too).

Thanks for your help. I'm probably going to have to spend a few solid hours
(days?) playing with this to be sure, but it looks like I can build this.

Brian Candler

9/16/2008 8:55:00 AM

> I'd like to detect "no work left" as a case of the "farmer" going out of
> scope, if possible.

In principle, if the Farmer is an object, then it can have a finalizer
which sends the terminate message to all the workers. The finalizer
won't be called immediately when the Farmer goes out of scope, but at
some time later.

In practice, it isn't easy to do. The finalizer is only called *after*
the farmer has been destroyed, so you can't use any methods or instance
variables on it. And I find it very hard to demonstrate in practice. The
following attempt is broken and I can't see what's wrong:

#---- 8< --------------
require 'thread'

class Farmer
attr_reader :queue, :workers

def self.cleanup(workers)
lambda { |id|
workers.each { |t| p t; Thread.kill(t) }
}
end

def self.create
farmer = new
workers = farmer.workers
ObjectSpace.define_finalizer(farmer, cleanup(workers))
farmer
end

def self.make_worker(queue)
Thread.new do
loop do
puts queue.pop
end
end
end

def initialize
@workers = []
@queue = Queue.new
end

def add_worker
@workers << self.class.make_worker(queue)
end

def push(x)
@queue.push(x)
end
end

def testit
f = Farmer.create
3.times { f.add_worker }
f.push("hello")
f.push("world")
end

10.times { testit }
GC.start
# For some reason the farmers still exist
ObjectSpace.each_object(Farmer) { |f| p f }
puts "End of the world"

sleep 1
#---- 8< --------------

When I run this, I see that the finalizers are indeed called when ruby
terminates - but not when garbage-collection is requested. Indeed, the
Farmers remain in ObjectSpace even after garbage collection. I can't see
why that should be.

I've tried to ensure that neither the finalizer lambda nor the worker
threads have the farmer bound to their environment, and I can't see what
else is holding a reference. This is with ruby 1.8.6 (2008-03-03
patchlevel 114) [i686-linux]

Regards,

Brian.
--
Posted via http://www.ruby-....

comp.lang.ruby

How to detect blocking?

David Masover

Robert Klemme

David Masover

David Masover

Robert Klemme

David Masover

Robert Klemme

David Masover

Robert Klemme

David Masover

Brian Candler

x Login to ForumsZone