Asp Forum - Newly created thread blocks all others in Ruby 1.8.7 patchlevel 72 on Cygwin

Daniel Merk

8/19/2008 3:23:00 PM

[Note: parts of this message were removed to make it a legal post.]

Hello together,

I am using the following Ruby version:

ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]

I have problems with threads now that did not occur in Ruby 1.8.6. I execute
the following code (derived from an
example in the Ruby core documentation):

require 'thread'

count1 = count2 = 0
puts "starting thread a"
a = Thread.new do
loop { count1 += 1 }
end
a.priority = -1

puts "starting thread b"
b = Thread.new do
loop { count2 += 1 }
end
b.priority = -2
sleep 1 #=> 1
Thread.critical = 1

puts "count1: #{count1}"
puts "count2: #{count2}"

This code never starts thread b. It seems that thread a blocks all
other threads once it runs (though its priority is lowered), even the
main thread.
If I put a sleep command into the thread a loop, thread b starts, but
the problem remains, since then thread b blocks all others from
running.

The Ruby 1.8.7 release notes speak of a change in the mutex
implementation in C. Could that be a problem?

Thanks for any help
Daniel

5 Answers

Pit Capitain

8/19/2008 8:06:00 PM

2008/8/19 Daniel Merk <daniel.merk.rubytalk@googlemail.com>:
> I am using the following Ruby version:
>
> ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]

Hello Daniel, I don't have 1.8.7 yet, I don't have cygwin, and...

> I have problems with threads now that did not occur in Ruby 1.8.6.

...and I don't think that what you see is a problem. It may be that on
1.8.6 you saw a different behavior, but I still don't think that the
new behavior is a problem, because AFAIK Ruby doesn't guarantee that
threads with equal priority do get equal time to run. If you create a
thread with

> a = Thread.new do
> loop { count1 += 1 }
> end

then Ruby starts the thread as documented. I think it's perfectly
legal that Ruby runs only the new thread without scheduling other
threads, including the main thread, as long as there's no other thread
with higher priority.

> a.priority = -1

If you finally get there, then it should be the main thread again who
is blocking thread "a".

> b = Thread.new do
> loop { count2 += 1 }
> end

Here the same applies as above. Thread "b" could run all the time.

> b.priority = -2

Now it should be: main thread blocks "a" blocks "b".

> sleep 1 #=> 1

Give "a" some more time to run.

> Thread.critical = 1
>
> puts "count1: #{count1}"
> puts "count2: #{count2}"
>
> This code never starts thread b. It seems that thread a blocks all
> other threads once it runs (though its priority is lowered), even the
> main thread.

See above. I think that's ok. What behavior do you expect? Or better:
what do you want to achieve?

Regards,
Pit

Daniel Merk

8/20/2008 7:33:00 AM

[Note: parts of this message were removed to make it a legal post.]

Hi Pit,

thanks for your response.

The behaviour I expect from threads in general is that they virtually run in
parallel. So I expect each active thread to be at least scheduled, but not
blocked, even if it has a low priority. If there is a total of two threads
that have the same priority, I expect the thread scheduler to assign 50% of
processing resources to each of them. In earlier Ruby versions (e.g. 1.8.6),
this is exactly what has happened.

I could reach that behavior in the example before (which is as I stated an
"official" one, since you can also find it in the only ruby core
documentation of the thread class:
http://www.ruby-doc.org/core/classes/Thread.ht...) by inserting sleep
statements into the loop-blocks. I could then control the amount of
processing time for each thread by balancing the sleep times, therefore
emulating priority. But then I would mimic the behavior that usually rubys
thread scheduler should perform for me.

Best regards
Daniel

Pit Capitain

8/21/2008 10:25:00 AM

Hi Daniel.

> The behaviour I expect from threads in general is that they virtually run in
> parallel. So I expect each active thread to be at least scheduled, but not
> blocked, even if it has a low priority.

From looking at the source code of 1.8.6 and from running some tests
this doesn't seem to be the case. Threads with lower priority are
blocked as long as there are runnable threads with higher priority.

> If there is a total of two threads
> that have the same priority, I expect the thread scheduler to assign 50% of
> processing resources to each of them. In earlier Ruby versions (e.g. 1.8.6),
> this is exactly what has happened.

Indeed this seems to be the case in 1.8.6, again from looking at the
source and running some tests. If this doesn't work in Ruby 1.8.7, you
should perhaps report it on ruby-core.

> I could reach that behavior in the example before (which is as I stated an
> "official" one, since you can also find it in the only ruby core
> documentation of the thread class:
> http://www.ruby-doc.org/core/classes/Thread.ht...) by inserting sleep
> statements into the loop-blocks.

AFAIK there's no official description of Ruby yet. There are projects
trying to fill this gap with executable specifications.

> I could then control the amount of
> processing time for each thread by balancing the sleep times, therefore
> emulating priority. But then I would mimic the behavior that usually rubys
> thread scheduler should perform for me.

I don't buy this "should". The documentation you've cited clearly
says, that higher-priority threads will run before lower-priority
threads.

Regards,
Pit

Daniel Merk

8/22/2008 11:41:00 AM

Hi Pit,

thanks again for the response. I backgraded to Ruby 1.8.6 to make some
tests too, the behavior is as you have described it. Here the test
code I have used:

Thread.main.priority = 0
count1 = count2 = 0

a = Thread.new do
puts "starting thread a"
Thread.current.priority = -1
loop { count1 += 1 }
end

b = Thread.new do
puts "starting thread b"
Thread.current.priority = -1
loop { count2 += 1}
end

puts "stalling main thread for 1 second"
sleep 1

a.kill; b.kill
puts "count1: #{count1}"
puts "count2: #{count2}"

On Ruby 1.8.6 I get the following output:

starting thread a
starting thread b
stalling main thread for 1 second
count1: 953760
count2: 953753

So this is fine and what I expected (if I lower thread b priority to
-2, count2 drop to absolute 0. This is quite awful, but ok, I'll stop
complaining ;-). On Ruby 1.8.7 however, I just see:

starting thread a
starting thread b
stalling main thread for 1 second

No more output from here on. It seems that the sleep statement does not return.

Thanks however for your help.

Regards
Daniel

On 8/21/08, Pit Capitain <pit.capitain@gmail.com> wrote:
> Hi Daniel.
>
>> The behaviour I expect from threads in general is that they virtually run
>> in
>> parallel. So I expect each active thread to be at least scheduled, but not
>> blocked, even if it has a low priority.
>
> From looking at the source code of 1.8.6 and from running some tests
> this doesn't seem to be the case. Threads with lower priority are
> blocked as long as there are runnable threads with higher priority.
>
>> If there is a total of two threads
>> that have the same priority, I expect the thread scheduler to assign 50%
>> of
>> processing resources to each of them. In earlier Ruby versions (e.g.
>> 1.8.6),
>> this is exactly what has happened.
>
> Indeed this seems to be the case in 1.8.6, again from looking at the
> source and running some tests. If this doesn't work in Ruby 1.8.7, you
> should perhaps report it on ruby-core.
>
>> I could reach that behavior in the example before (which is as I stated an
>> "official" one, since you can also find it in the only ruby core
>> documentation of the thread class:
>> http://www.ruby-doc.org/core/classes/Thread.ht...) by inserting
>> sleep
>> statements into the loop-blocks.
>
> AFAIK there's no official description of Ruby yet. There are projects
> trying to fill this gap with executable specifications.
>
>> I could then control the amount of
>> processing time for each thread by balancing the sleep times, therefore
>> emulating priority. But then I would mimic the behavior that usually rubys
>> thread scheduler should perform for me.
>
> I don't buy this "should". The documentation you've cited clearly
> says, that higher-priority threads will run before lower-priority
> threads.
>
> Regards,
> Pit
>
>

ara.t.howard

8/22/2008 2:27:00 PM

On Aug 19, 2008, at 9:22 AM, Daniel Merk wrote:

> require 'thread'
>
> count1 = count2 = 0
> puts "starting thread a"
> a = Thread.new do
> loop { count1 += 1 }
> end
> a.priority = -1
>
> puts "starting thread b"
> b = Thread.new do
> loop { count2 += 1 }
> end
> b.priority = -2
> sleep 1 #=> 1
> Thread.critical = 1
>
> puts "count1: #{count1}"
> puts "count2: #{count2}"
>
>
> This code never starts thread b. It seems that thread a blocks all
> other threads once it runs (though its priority is lowered), even the
> main thread.
> If I put a sleep command into the thread a loop, thread b starts, but
> the problem remains, since then thread b blocks all others from
> running.

that code looks completely non-deterministic to me and i've had
similar issues many times in the past. i think the fundamental error
is mistaking the creation of a thread for scheduling it. i've used
the following pattern many times to force my code to know that a
thread has started processing before continuing:

cfp:~ > ruby186 a.rb
starting thread a
starting thread b
count1: 4580083
count2: 44732

cfp:~ > ruby187 a.rb
starting thread a
starting thread b
count1: 4394267
count2: 37032

cfp:~ > cat a.rb

require 'thread'

q = Queue.new
count1 = count2 = 0

a =
Thread.new do
puts "starting thread a"
q.push Thread.current
loop { count1 += 1 }
end

a = q.pop # wait for the thread to start
a.priority = -1

b =
Thread.new do
puts "starting thread b"
q.push Thread.current
loop { count2 += 1 }
end

b = q.pop # wait for the thread to start
b.priority = -2

sleep 1 #=> 1
Thread.critical = 1

puts "count1: #{count1}"
puts "count2: #{count2}"

i'm sorry that i cannot test this on cygwin so i'm not *sure* it's the
issue. but to me it simply looks like non-deterministic code. in
fact, even the code above has a race condition, i only force the
program to ensure each thread calls puts - there is nothing to make
sure that stdout has been flushed or that the increments have run in
the loop. to do that i think you'd need something like this

cfp:~ > cat a.rb
require 'thread'

q = Queue.new
count1 = count2 = 0

a =
Thread.new do
STDERR.puts "starting thread a"
loop do
count1 += 1
q.push Thread.current if count1 == 1
end
end

a = q.pop # wait for the thread to start
a.priority = -1

b =
Thread.new do
puts "starting thread b"
q.push Thread.current
loop do
count2 += 1
q.push Thread.current if count2 == 1
end
end

b = q.pop # wait for the thread to start
b.priority = -2

Thread.critical = 1

puts "count1: #{count1}"
puts "count2: #{count2}"

cfp:~ > ruby186 a.rb
starting thread a
starting thread b
count1: 19297
count2: 28490

cfp:~ > ruby187 a.rb
starting thread a
starting thread b
count1: 17435
count2: 24603

i'd be curious how that fairs on cygwin. notice that the sleep is not
required with the queues.

regards.

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

comp.lang.ruby

Newly created thread blocks all others in Ruby 1.8.7 patchlevel 72 on Cygwin

Daniel Merk

Pit Capitain

Daniel Merk

Pit Capitain

Daniel Merk

ara.t.howard

x Login to ForumsZone