[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

A tricky problem about Process.wait and popen

uncutstone

5/12/2006 7:38:00 PM

My OS is windows 98, and ruby version is ruby 1.8.2 (2004-12-25)
[i386-mswin32] .

The following is the code:

main.rb:

pipe = IO.popen("ruby saygoodbye.rb", "w+")
sleep 3
cpid = Process.wait
lines = pipe.readlines
puts lines

saygoodbye.rb:

34.times do
puts "Please take a look at what will happen, it's really
tricky"
end
$stderr.puts "child process #{Process.pid} exits"

The above code cannot execute as expected: the parent process will wait
for ever. When I use ctrl-c to break it , I got

main.rb:3:in `wait': Interrupt
from main.rb:3

But if I change 34.times in saygoodbye.rb to 33.times, it works well.
So , it seems there is a limit of pipe's capacity.

Somebody can give me a clear explanation, Very thanks.
13 Answers

uncutstone

5/12/2006 7:50:00 PM

0

It seems that when child process write too many things to the pipe, the
signal of child process exit sent to parent process will lost.

Is it a bug?

Gonald

5/13/2006 7:10:00 AM

0

uncutstone wrote:
> It seems that when child process write too many things to the pipe, the
> signal of child process exit sent to parent process will lost.
>
> Is it a bug?
>
It's not a bug. The child process just never exits. This is because it
blocks while the pipe is full. Since the parent waits for the child to
exit before it reads from the pipe, a deadlock occurs after a certain
amount of data.

Try this, instead:

pipe = IO.popen("ruby saygoodbye.rb", "w+")
sleep 3
lines = pipe.readlines
cpid = Process.wait
puts lines

uncutstone

5/13/2006 9:37:00 AM

0

Thanks.
But it seems that child process has exited.
Acctually the result is :

child process 577605 exits

And main.rb is waiting. After I use ctrl-c to break it , i got :

main.rb:2:in `wait': Interrupt
from main.rb:2

Robert Klemme

5/13/2006 9:46:00 AM

0

Gonald wrote:
> uncutstone wrote:
>> It seems that when child process write too many things to the pipe, the
>> signal of child process exit sent to parent process will lost.
>>
>> Is it a bug?
>>
> It's not a bug. The child process just never exits. This is because it
> blocks while the pipe is full. Since the parent waits for the child to
> exit before it reads from the pipe, a deadlock occurs after a certain
> amount of data.
>
> Try this, instead:
>
> pipe = IO.popen("ruby saygoodbye.rb", "w+")
> sleep 3
> lines = pipe.readlines
> cpid = Process.wait
> puts lines
>
The pipe is not closed properly. I'd use the block form

IO.popen("ruby saygoodbye.rb", "w+") do |pipe|
....
end

Also, I'm not sure I get the point of the "sleep 3" in there. IMHO it's
superfluous since readlines won't return before the process has died
anyway. In the worst case it slows things down otherwise it just does
merely nothing.

Also I think Process.wait won't work because readlines won't return
before the child terminated.

If you want more direct output then it's better to do the iteration and
printing directly, like

IO.popen("ruby saygoodbye.rb", "w+") do |pipe|
pipe.each {|line| puts line}
puts "Exit status: #{$?.exitstatus}"
end

Kind regards

robert

uncutstone

5/13/2006 10:43:00 AM

0

>Also, I'm not sure I get the point of the "sleep 3" in there.

It is nothing to do with "sleep 3", I just forgot to remove it. Sorry
for that it may blur the real problem.

>The pipe is not closed properly. I'd use the block form

>IO.popen("ruby saygoodbye.rb", "w+") do |pipe|
>...
>end

Yes, it's better. But my real problem is the lost signal that child
process send to parent when it exit.

>Also I think Process.wait won't work because readlines won't >return before the child terminated.
Sorry , I think I don't get your point. Process.wait is executed before
realines, so why readlines affects Process.wait??

uncutstone

5/13/2006 11:38:00 AM

0

I made more test of the code and eventually found you are right and got
a clear understanding of it. Yes, the child process never exits and
sends no signal .

It is really tricky and interesting to see the way how the child
process get blocked.

When the code is "34.times do" , the result is:

"child process 577605 exits " was displayed on the screen
and then deadlock happened.

And after I use ctrl -c to break it, I got:

main.rb:2:in `wait': Interrupt
from main.rb:2

It shows that child process get blocked before it exits. It wait the
parent process to read pipe before it exits.

But when I change the code to "68.times do", the result is:

Nothing displayed on the screen and deadlock happens.

And after I use ctrl -c to break it, I got:

main.rb:2:in `wait': Interrupt
from main.rb:2
saygoodbye.rb:3:in `write': Bad file descriptor (Errno::EBADF)
from saygoodbye.rb:3:in `puts'
from saygoodbye.rb:3
from saygoodbye.rb:1:in `times'
from saygoodbye.rb:1

It shows that child process get blocked immediatly when it is doing
pipe writing.

Let me make a simple summary for it:
1. If the child process write too much to pipe, it will get blocked and
doesn't exits , doesn't send exit signal.
2.The way the child process get blocked is depending on how much it
want to write to the pipe.
2.1 If the child process make the pipe full but not too full, it will
get blocked before it exits.
2.1 If the child process make the pipe full and even too full, it will
get blocked immediatly at pipe writing.

Do you agree with it .Very Thanks .

uncutstone

5/13/2006 11:42:00 AM

0

I made more test of the code and eventually found you are right and got

a clear understanding of it. Yes, the child process never exits and
sends no signal .

It is really tricky and interesting to see the way how the child
process get blocked.


When the code is "34.times do" , the result is:


"child process 577605 exits " was displayed on the screen
and then deadlock happened.


And after I use ctrl -c to break it, I got:


main.rb:2:in `wait': Interrupt
from main.rb:2


It shows that child process get blocked before it exits. It wait the
parent process to read pipe before it exits.


But when I change the code to "68.times do", the result is:


Nothing displayed on the screen and deadlock happens.


And after I use ctrl -c to break it, I got:


main.rb:2:in `wait': Interrupt
from main.rb:2
saygoodbye.rb:3:in `write': Bad file descriptor (Errno::EBADF)
from saygoodbye.rb:3:in `puts'
from saygoodbye.rb:3
from saygoodbye.rb:1:in `times'
from saygoodbye.rb:1


It shows that child process get blocked immediatly when it is doing
pipe writing.


Let me make a simple summary for it:
1. If the child process write too much to pipe, it will get blocked and

doesn't exits , doesn't send exit signal.
2.The way the child process get blocked is depending on how much it
want to write to the pipe.
2.1 If the child process make the pipe full but not too full, it will
get blocked before it exits.
2.2 If the child process make the pipe full and even too full, it will
get blocked immediatly at pipe writing.


Do you agree with it .Very Thanks .

uncutstone

5/13/2006 12:26:00 PM

0

I think I already have understanded all of these. But a understanding
doesn't mean a solution.

Let me give the problem I want to solve.

There is a parent process and a fixed number of child processes. Parent
process is responsible for forking a fixed number of child processes
using IO.popen and then wait for any child process exits. Whenever a
child process complete its task , it will write the result in pipe and
signal the parent process "mission complete" and then exits. When
parent get the signal, it will read correspondant pipe to get result
and then creating a new child process .

If the result returned by the child process is small, pipe doesn't get
full, it works well.

But if the result is big and pipe gets full, the child get blocked and
signal cannot send to parent process.

Let me make it simple.
1.Multiple child process want to return result to parent process via
pipe.
2. Parent need a mechanism similar to unix select() to get notification
from child process when it completes its task.

The code is something like this:

main.rb
processNum = 10
pipes = Hash.new
numOfProcessRunning = 0
loop do
apipe = IO.popen("ruby child.rb","w+")
cpid = apipe.gets.strip.to_i
pipes[cpid] = apipe
numOfProcessRunning += 1
if numOfProcessRunning >= processNum
cpid = Process.wait
apipe = pipes[cpid]
apipe.each { |line|
#.... , get the result
}
apipe.close
pipes.delete(cpid)
numOfProcessRunning -= 1
end
end

child.rb:
puts Process.pid
$stdout.flush
#..... , do something, then
puts result

uncutstone

5/13/2006 12:52:00 PM

0

Is there some ways to increase the capacity of pipe?

I am using windows 98.

uncutstone

5/13/2006 12:52:00 PM

0

Is there some ways to increase the capacity of pipe?

I am using windows 98.