cs5b
9/30/2006 4:00:00 AM
Hi there,
I have a little program that mines the web. It consists of two ruby
programs. The first is responsible to get the data from the web; the
second is responsible of mining that data. I connect the two via
command line redirection/ pipe (the first executes stdin.puts data; the
second calls data = stdin.gets)
This works just fine when I access html pages. However, when I access
binary data, at times, the pipe misbehaves. I encountered one jpg file
that causes a crash of the consuming data mining program on linux; and
a infinite loop (stdin.gets just returns nil after a while without
having read the entire data) on windows.
I suppose what I have to do is encode the data before I place it on the
pipe and decode it after I consume it from the pipe. Any suggestions?
Cheers
Christian