Asp Forum - IO redirecting; waitpid

TPReal

8/16/2008 1:14:00 PM

Hello.

I have two problems.

First: is there an easy way to redirect data from an incoming IO stream
to an outgoing IO stream, in a nonblocking way? For example, can you run
a console program from Ruby program and redirect the program's output to
STDOUT in realtime? I know one solution that works, but is there
anything better?

Thread::new(IO::popen(COMMAND)){ |srv|
until srv.eof?
print srv.readpartial(1024)
end
}

Replacing readpartial with sysread also works, and with read_nonblock
works partially.

Second problem: when I use the code above, and at the same time the main
thread is waiting on Process.waitpid for some other program (not the one
of which I redirect the output, but some else), then the waiting doesn't
terminate correctly. It waits until the process finishes, and then,
until there is some data available from the process I'm streaming in the
thread above. And only then the main thread goes on.

I use Windows XP, Ruby 1.8.6.

Here's a blog entry describing the same problem, with no solution:
http://al2o3-cr.blogspot.com/2008/08/io... .
--
Posted via http://www.ruby-....

5 Answers

ara.t.howard

8/16/2008 3:07:00 PM

On Aug 16, 2008, at 7:13 AM, Thomas Bl. wrote:

> Hello.
>
> I have two problems.
>
> First: is there an easy way to redirect data from an incoming IO
> stream
> to an outgoing IO stream, in a nonblocking way? For example, can you
> run
> a console program from Ruby program and redirect the program's
> output to
> STDOUT in realtime? I know one solution that works, but is there
> anything better?
>
> Thread::new(IO::popen(COMMAND))> { |srv|
> until srv.eof?
> print srv.readpartial(1024)
> end
> }
>
> Replacing readpartial with sysread also works, and with read_nonblock
> works partially.
>
> Second problem: when I use the code above, and at the same time the
> main
> thread is waiting on Process.waitpid for some other program (not the
> one
> of which I redirect the output, but some else), then the waiting
> doesn't
> terminate correctly. It waits until the process finishes, and then,
> until there is some data available from the process I'm streaming in
> the
> thread above. And only then the main thread goes on.
>
> I use Windows XP, Ruby 1.8.6.
>
> Here's a blog entry describing the same problem, with no solution:
> http://al2o3-cr.blogspot.com/2008/08/io... .

you can do it, but not on windows.

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Robert Klemme

8/17/2008 11:00:00 AM

On 16.08.2008 17:07, ara.t.howard wrote:
> On Aug 16, 2008, at 7:13 AM, Thomas Bl. wrote:
>
>> Hello.
>>
>> I have two problems.
>>
>> First: is there an easy way to redirect data from an incoming IO
>> stream
>> to an outgoing IO stream, in a nonblocking way? For example, can you
>> run
>> a console program from Ruby program and redirect the program's
>> output to
>> STDOUT in realtime? I know one solution that works, but is there
>> anything better?
>>
>> Thread::new(IO::popen(COMMAND))>> { |srv|
>> until srv.eof?
>> print srv.readpartial(1024)
>> end
>> }

Yes, use the block form - much safer because streams are properly closed:

Thread.new do
IO.popen COMMAND do |srv|
buffer = "" # more efficient
# while srv.readpartial(512, buffer)
while srv.read(512, buffer)
STDOUT.write(buffer)
end
end
end

>> Second problem: when I use the code above, and at the same time the
>> main
>> thread is waiting on Process.waitpid for some other program (not the
>> one
>> of which I redirect the output, but some else), then the waiting
>> doesn't
>> terminate correctly. It waits until the process finishes, and then,
>> until there is some data available from the process I'm streaming in
>> the
>> thread above. And only then the main thread goes on.
>>
>> I use Windows XP, Ruby 1.8.6.
>>
>> Here's a blog entry describing the same problem, with no solution:
>> http://al2o3-cr.blogspot.com/2008/08/io... .
>
> you can do it, but not on windows.

Well, maybe with cygwin.

Kind regards

robert

Michal Suchanek

8/17/2008 2:36:00 PM

On 16/08/2008, ara.t.howard <ara.t.howard@gmail.com> wrote:
>
> On Aug 16, 2008, at 7:13 AM, Thomas Bl. wrote:
>
>
> > Hello.
> >
> > I have two problems.
> >
> > First: is there an easy way to redirect data from an incoming IO stream
> > to an outgoing IO stream, in a nonblocking way? For example, can you run
> > a console program from Ruby program and redirect the program's output to
> > STDOUT in realtime? I know one solution that works, but is there
> > anything better?
> >
> > Thread::new(IO::popen(COMMAND))> > { |srv|
> > until srv.eof?
> > print srv.readpartial(1024)
> > end
> > }
> >
> > Replacing readpartial with sysread also works, and with read_nonblock
> > works partially.
> >
> > Second problem: when I use the code above, and at the same time the main
> > thread is waiting on Process.waitpid for some other program (not the one
> > of which I redirect the output, but some else), then the waiting doesn't
> > terminate correctly. It waits until the process finishes, and then,
> > until there is some data available from the process I'm streaming in the
> > thread above. And only then the main thread goes on.
> >
> > I use Windows XP, Ruby 1.8.6.
> >
> > Here's a blog entry describing the same problem, with no solution:
> > http://al2o3-cr.blogspot.com/2008/08/io... .
> >
>
>
> you can do it, but not on windows.
>

If Ruby on Windows emulates fork (and there is nothing in the docs
saying fork would not work on some platforms) you can use fork and
execute the program normally without popen. The program then inherits
the standard output fd and you do not have to copy the data to it.

For copying data I haven't found a working solution on Linux either.

Thanks

Michal

Michael Guterl

8/17/2008 2:48:00 PM

On Sat, Aug 16, 2008 at 11:07 AM, ara.t.howard <ara.t.howard@gmail.com> wrote:
>
> On Aug 16, 2008, at 7:13 AM, Thomas Bl. wrote:
>
>> Hello.
>>
>> I have two problems.
>>
>> First: is there an easy way to redirect data from an incoming IO stream
>> to an outgoing IO stream, in a nonblocking way? For example, can you run
>> a console program from Ruby program and redirect the program's output to
>> STDOUT in realtime? I know one solution that works, but is there
>> anything better?
>>
>> Thread::new(IO::popen(COMMAND))>> { |srv|
>> until srv.eof?
>> print srv.readpartial(1024)
>> end
>> }
>>
>> Replacing readpartial with sysread also works, and with read_nonblock
>> works partially.
>>
>> Second problem: when I use the code above, and at the same time the main
>> thread is waiting on Process.waitpid for some other program (not the one
>> of which I redirect the output, but some else), then the waiting doesn't
>> terminate correctly. It waits until the process finishes, and then,
>> until there is some data available from the process I'm streaming in the
>> thread above. And only then the main thread goes on.
>>
>> I use Windows XP, Ruby 1.8.6.
>>
>> Here's a blog entry describing the same problem, with no solution:
>> http://al2o3-cr.blogspot.com/2008/08/io... .
>
>
> you can do it, but not on windows.
>

I seem to see this response a lot with regard to ruby's threading and windows.

I'm typically lucky enough to avoid developing for windows. However,
I am curious if JRuby or 1.9 works around these issues? What about
IronRuby?

I don't mean to hijack, but I feel the original poster could benefit
from answers to these questions also...

Thanks,
Michael Guterl

Nobuyoshi Nakada

8/18/2008 1:29:00 AM

Hi,

At Sat, 16 Aug 2008 22:13:30 +0900,
Thomas Bl. wrote in [ruby-talk:311482]:
> Second problem: when I use the code above, and at the same time the main
> thread is waiting on Process.waitpid for some other program (not the one
> of which I redirect the output, but some else), then the waiting doesn't
> terminate correctly. It waits until the process finishes, and then,
> until there is some data available from the process I'm streaming in the
> thread above. And only then the main thread goes on.
>
> I use Windows XP, Ruby 1.8.6.

What's the exact revision?

It seems not to reproduce with latest 1.8.6 and 1.8.7, but
could with latest 1.8 branch.

Could you report it to the ITS?
<http://redmine.ruby-lang.org/projects/ruby/issu...

--
Nobu Nakada

comp.lang.ruby