Asp Forum - Timeout lengthy external executable

Ema Fuma

6/15/2008 4:33:00 PM

Hi all,
I'm new to Ruby, I'm trying to use it to create a script where I launch
some executables.
I'd like to timeout if one of them takes too long, I'm using IO.popen to
launch the exe.
The problem is that the timeout doesn't seem to work, so if my
executable hangs for some reason I have no way to stop it and go on.

Is there something I can do?
If I use "system" instead of IO.popen it works but I need popen to parse
stdout

I include a simplified code of what I'm doing (no parsing)

Thanks in advance for any tips on that
Bye

begin
timeout(5) {
IO.popen(command_media)
}
puts "Timeout didn't occur"
rescue Timeout::Error
puts "Timed out!"
end
--
Posted via http://www.ruby-....

11 Answers

ara.t.howard

6/15/2008 4:49:00 PM

On Jun 15, 2008, at 10:32 AM, Ema Fuma wrote:

> Hi all,
> I'm new to Ruby, I'm trying to use it to create a script where I
> launch
> some executables.
> I'd like to timeout if one of them takes too long, I'm using
> IO.popen to
> launch the exe.
> The problem is that the timeout doesn't seem to work, so if my
> executable hangs for some reason I have no way to stop it and go on.
>
> Is there something I can do?
> If I use "system" instead of IO.popen it works but I need popen to
> parse
> stdout
>
> I include a simplified code of what I'm doing (no parsing)
>
> Thanks in advance for any tips on that
> Bye
>
> begin
> timeout(5) {
> IO.popen(command_media)
> }
> puts "Timeout didn't occur"
> rescue Timeout::Error
> puts "Timed out!"
> end

if you're on windows try using 'systemu', which will allow you to
capture stdout/stderr and should also should not block your app.

gem install systemu

http://codeforp...lib/ruby/systemu/systemu-1....

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Ema Fuma

6/15/2008 5:23:00 PM

> gem install systemu
>
> http://codeforp...lib/ruby/systemu/systemu-1....
>
> a @ http://codeforp...

Hi thanks for the quick answer,

I'm new to ruby and RubyGems, it seemed that the installation was
succesful
but I get an error when requiring the file. I'll try to figure it how,
what I would like to ask is that if with systemu I will be able to parse
the stdout and stderr line by line at real time (not waiting the process
to finish to parse stdout). From the doc it's not too clear to me

Thanks again

error when launching:
encode7.rb:92:in `require': no such file to load -- systemu (LoadError)
from encode7.rb:92
--
Posted via http://www.ruby-....

ara.t.howard

6/15/2008 5:44:00 PM

On Jun 15, 2008, at 11:22 AM, Ema Fuma wrote:

> I'm new to ruby and RubyGems, it seemed that the installation was
> succesful
> but I get an error when requiring the file. I'll try to figure it how,

require 'rubygems'
require 'systemu'

>
> what I would like to ask is that if with systemu I will be able to
> parse
> the stdout and stderr line by line at real time (not waiting the
> process
> to finish to parse stdout). From the doc it's not too clear to me

you cannot do this on windows safely. if you are on *nix you can
easily parse in realtime using the open4 lib, which is much more
powerful than systemu. however, only systemu is cross platform.

cheers.

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Ema Fuma

6/15/2008 6:00:00 PM

> require 'rubygems'
> require 'systemu'

Thanks, with that it worked perfectly!

> you cannot do this on windows safely. if you are on *nix you can
> easily parse in realtime using the open4 lib, which is much more
> powerful than systemu. however, only systemu is cross platform.

Yes, I tried systemu and work exactly as you explained.
Yes, for me it would be better to have my script cross-platform
(providing for each OS the compiled executables to launch).
With IO.popen I was able to parse at run-time line by line the stdout
like this:

IO.popen(command) do |pipe|
pipe.each("\n") do |line|
puts line
end

And it's working (Windows and OSX), the only problem I had was that
timeout(10) wasn't ignored.
Now what I want to achieve by the run-time parsing are two things:
- display a progess of what is happening
- check that the process keeps writing on stdout, so I know it's still
alive. If it doesn't for 10 sec. exit the entire program.

Is that possible?

Thanks a lot again for your kindness!

--
Posted via http://www.ruby-....

ara.t.howard

6/15/2008 11:03:00 PM

On Jun 15, 2008, at 12:00 PM, Ema Fuma wrote:

> Yes, I tried systemu and work exactly as you explained.
> Yes, for me it would be better to have my script cross-platform
> (providing for each OS the compiled executables to launch).
> With IO.popen I was able to parse at run-time line by line the stdout
> like this:
>
> IO.popen(command) do |pipe|
> pipe.each("\n") do |line|
> puts line
> end
>
> And it's working (Windows and OSX), the only problem I had was that
> timeout(10) wasn't ignored.
> Now what I want to achieve by the run-time parsing are two things:
> - display a progess of what is happening
> - check that the process keeps writing on stdout, so I know it's still
> alive. If it doesn't for 10 sec. exit the entire program.
>
> Is that possible?

i'm 99% positive that trying to read from the pipe will conflict with
the timeout method - because of the way ruby uses green threads it's
easy to lock up threads where io is concerned. sometimes select can
be used but it's a case by case basis for mixing threads, io, and non-
blocking behaviour on windows. YMMV.

cheers.

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Ema Fuma

6/16/2008 7:26:00 AM

> i'm 99% positive that trying to read from the pipe will conflict with
> the timeout method - because of the way ruby uses green threads it's
> easy to lock up threads where io is concerned. sometimes select can
> be used but it's a case by case basis for mixing threads, io, and non-
> blocking behaviour on windows. YMMV.

Ok, I thought I was doing something wrong, or not using the right
method.
Maybe I should use some customized timeout?
To me it looked strange that in a scripting language like Ruby it was
impossible to timeout certain operations and I thought someone else
found a solution.
Thanks

bye
--
Posted via http://www.ruby-....

ara.t.howard

6/16/2008 4:07:00 PM

On Jun 16, 2008, at 1:25 AM, Ema Fuma wrote:

> Ok, I thought I was doing something wrong, or not using the right
> method.
> Maybe I should use some customized timeout?
> To me it looked strange that in a scripting language like Ruby it was
> impossible to timeout certain operations and I thought someone else
> found a solution.
> Thanks
>
> bye

it's not ruby per-se, but the interaction of select on windows. you
haven't yet said - what platform are you on?

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Ema Fuma

6/16/2008 4:41:00 PM

> it's not ruby per-se, but the interaction of select on windows. you
> haven't yet said - what platform are you on?

I'm under Windows but I would like to use the same script on Linux.
Basically it's just
-launch an external executable that write it's progress on stdout
-parse the stdout line by line
-if there's no new line for more than 10 sec close the external exe

Thanks
--
Posted via http://www.ruby-....

ara.t.howard

6/16/2008 5:13:00 PM

On Jun 16, 2008, at 10:41 AM, Ema Fuma wrote:

> I'm under Windows but I would like to use the same script on Linux.
> Basically it's just
> -launch an external executable that write it's progress on stdout
> -parse the stdout line by line
> -if there's no new line for more than 10 sec close the external exe

it's nearly impossible to do this on windows reliably. this are just
too many failure conditions, for instance say the child program does

gb = 2 ** 30

huge = '*' * gb

STDOUT.write huge

and the parent does

buf = child.gets

both processes will hang. the child because it fills the pipe, the
parent because it never sees a newline. you'd think this could be
solved by using timeout, but it cannot (on windows) because ruby's
threads schedule with select and therefore many/most io operations
will block the scheduler - causing a hang.

there is some hope for using socketpair and a tcp server in a cross
platoform manner, rough idea here:

http://pastie....

but this doesn't address issues like stderr or zombie processes.

anyhow, i'm not saying it *can't* be done on windows, but it's very
non-trival to combine pipes, timeout, and process management in a
robust fashion.

one idea might be to launch and external process to signal yourself
before reading, something like this:

signaler = Thread.new do
system "ruby -e' sleep 10; Process.kill(:HUP.to_s,
#{ Process.pid })"
end

begin
child.gets
rescue SignalException
...
end

however signals are very limited on windows and i've no idea if even
the signal handlers would fire once a process is blocked - worth
trying though... it's definitely going to take something creative on
your end. one other thought is to spawn and external reaper before
reading, one which would kill the external exe for you

reaper = Thread.new
system "ruby -e' sleep 10; Process.kill -9, #{ child.pid }'"
end

and then, in the parent arrange to schedule this before each read, and
kill it afterwards...

i'd be interested to hear other's thoughts on the issue (dan berger in
the house?) as i'm no windows expert.

regards.

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Ema Fuma

6/17/2008 9:38:00 AM

Thanks for the very detailed explanation.
This is my first project using Ruby, I didn't expect that doing this
quite simple taks was so difficult, if I have to resort to signals and
threads maybe it would be better for me to do that in C++ where I'm more
experienced.
Thanks again
--
Posted via http://www.ruby-....

comp.lang.ruby

Timeout lengthy external executable

Ema Fuma

ara.t.howard

Ema Fuma

ara.t.howard

Ema Fuma

ara.t.howard

Ema Fuma

ara.t.howard

Ema Fuma

ara.t.howard

Ema Fuma

x Login to ForumsZone