[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Parallel for loop

Fredrik

4/16/2008 3:34:00 AM

There doesn't seem to be any EASY way of doing a parallel computation
in Ruby.
I would like to do something like this :

array.map do |i|
fork do
i + 1
end
end
Process.waitall

wich would give back the array with one added to each element in an
array, and it would perform this "calculation" in parallel. However,
this doesn't work since fork runs a subprocess which is another Ruby
interpreter and I can't get anything back from that black hole, except
some exit status.

Actually, it would be really nice if there was a 'forkmap' method that
could do this:

array.forkmap do |i|
i + 1
end

But there isn't, right?
23 Answers

Kyle Hunter

4/16/2008 3:42:00 AM

0

http://peach.rubyforge....
--
Posted via http://www.ruby-....

Phillip Gawlowski

4/16/2008 3:47:00 AM

0

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Fredrik wrote:
| There doesn't seem to be any EASY way of doing a parallel computation
| in Ruby.
| I would like to do something like this :
|
| array.map do |i|
| fork do
| i + 1
| end
| end
| Process.waitall
|
| wich would give back the array with one added to each element in an
| array, and it would perform this "calculation" in parallel. However,
| this doesn't work since fork runs a subprocess which is another Ruby
| interpreter and I can't get anything back from that black hole, except
| some exit status.
|
| Actually, it would be really nice if there was a 'forkmap' method that
| could do this:
|
| array.forkmap do |i|
| i + 1
| end
|
| But there isn't, right?
|
|

Ruby uses green threads. All your threads would run within the Ruby
process, and aren't running parallel in the sense you seem to imply. If
you want to use threads, maybe JRuby and Java are what you seek.

JRuby uses Java threads, which are OS threads.

If I'm on the wrong tangent, just ignore this reply. :)

- --
Phillip Gawlowski
Twitter: twitter.com/cynicalryan

You thought I was taking your woman away from you. You're jealous.
You tried to kill me with your bare hands. Would a Kelvan do that?
Would he have to? You're reacting with the emotions of a human.
You are human.
~ -- Kirk, "By Any Other Name," stardate 4657.5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail....

iEYEARECAAYFAkgFdsUACgkQbtAgaoJTgL/lnwCfad0hWxWIiJPFyTSdCAkpVIT5
UaQAmwSDyRWArC1bKgnlnYkydZWWHfAN
=6rBM
-----END PGP SIGNATURE-----

ara.t.howard

4/16/2008 4:11:00 AM

0


On Apr 15, 2008, at 9:35 PM, Fredrik wrote:
> But there isn't, right?


cfp:~ > cat a.rb
module Enumerable
def forkify &b
map do |*a|
r, w = IO.pipe
fork do
r.close
w.write( Marshal.dump( b.call(*a) ) )
end
[ w.close, r ].last
end.map{|r| Marshal.load [ r.read, r.close ].first}
end
end


result =
[0, 1, 2, 3].forkify do |i|
p [ Process.ppid, Process.pid ]
i ** 2
end

p result





cfp:~ > ruby a.rb
[80870, 80871]
[80870, 80872]
[80870, 80873]
[80870, 80874]
[0, 1, 4, 9]


a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Michael Guterl

4/16/2008 4:26:00 AM

0

On Tue, Apr 15, 2008 at 11:35 PM, Fredrik <fredjoha@gmail.com> wrote:
> There doesn't seem to be any EASY way of doing a parallel computation
> in Ruby.
> I would like to do something like this :
>
> array.map do |i|
> fork do
> i + 1
> end
> end
> Process.waitall
>
> wich would give back the array with one added to each element in an
> array, and it would perform this "calculation" in parallel. However,
> this doesn't work since fork runs a subprocess which is another Ruby
> interpreter and I can't get anything back from that black hole, except
> some exit status.
>
> Actually, it would be really nice if there was a 'forkmap' method that
> could do this:
>
> array.forkmap do |i|
> i + 1
> end
>
> But there isn't, right?
>

http://skynet.ruby...

HTH,
Michael Guterl

Fredrik

4/16/2008 4:29:00 AM

0

On Apr 16, 1:10 pm, "ara.t.howard" <ara.t.how...@gmail.com> wrote:
> On Apr 15, 2008, at 9:35 PM, Fredrik wrote:
>
> > But there isn't, right?
>
> cfp:~ > cat a.rb
> module Enumerable
> def forkify &b
> map do |*a|
> r, w = IO.pipe
> fork do
> r.close
> w.write( Marshal.dump( b.call(*a) ) )
> end
> [ w.close, r ].last
> end.map{|r| Marshal.load [ r.read, r.close ].first}
> end
> end
>
> result =
> [0, 1, 2, 3].forkify do |i|
> p [ Process.ppid, Process.pid ]
> i ** 2
> end
>
> p result
>
> cfp:~ > ruby a.rb
> [80870, 80871]
> [80870, 80872]
> [80870, 80873]
> [80870, 80874]
> [0, 1, 4, 9]
>
> a @http://codeforp...
> --
> we can deny everything, except that we have the possibility of being
> better. simply reflect on that.
> h.h. the 14th dalai lama

Thanks! This code is just what I am looking for!
Peach for JRuby seems nice too, but I don't have JRuby :)

Phillip Gawlowski

4/16/2008 4:35:00 AM

0

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Fredrik wrote:

|
| Thanks! This code is just what I am looking for!
| Peach for JRuby seems nice too, but I don't have JRuby :)

It's just a download away. ;)
jruby.codehouse.org

However, you'll need a JVM that is compatible (IIRC, JRE 1.4.2 and newer).

- --
Phillip Gawlowski
Twitter: twitter.com/cynicalryan

~ "But the important thing is persistence." -Calvin trying to juggle eggs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail....

iEYEARECAAYFAkgFgfsACgkQbtAgaoJTgL+V6wCfUXlPOkq/DXhXQgz/MPLOLWYi
T44An0NLjW0yBzJFEMaGPD//SKcOtGdq
=ODYR
-----END PGP SIGNATURE-----

ara.t.howard

4/16/2008 4:45:00 AM

0


On Apr 15, 2008, at 10:30 PM, Fredrik wrote:
> Thanks! This code is just what I am looking for!

probably some errors there to catch - but the concept is solid
enough. you might also be interested in slave.rb

http://codeforp...lib/ruby/slave/slave-1....

regards.

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Fredrik

4/16/2008 4:51:00 AM

0

Actually, I'll change it a bit. I added Process.waitall since there
are otherwise some dead(?) processes left.


module Enumerable
def fmap &b
result = map do |*a|
r, w = IO.pipe
fork do
r.close
w.write( Marshal.dump( b.call(*a) ) )
end
[ w.close, r ].last
end
Process.waitall
result.map{|r| Marshal.load [ r.read, r.close ].first}
end
end

Charles Oliver Nutter

4/16/2008 5:04:00 AM

0

Phillip Gawlowski wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Fredrik wrote:
>
> |
> | Thanks! This code is just what I am looking for!
> | Peach for JRuby seems nice too, but I don't have JRuby :)
>
> It's just a download away. ;)
> jruby.codehouse.org
>
> However, you'll need a JVM that is compatible (IIRC, JRE 1.4.2 and newer).

www.jruby.org will get you there, and JRuby 1.1 requires Java 1.5 or higher.

- Charlie

Phillip Gawlowski

4/16/2008 5:11:00 AM

0

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Charles Oliver Nutter wrote:
| Phillip Gawlowski wrote:
|> -----BEGIN PGP SIGNED MESSAGE-----
|> Hash: SHA1
|>
|> Fredrik wrote:
|>
|> |
|> | Thanks! This code is just what I am looking for!
|> | Peach for JRuby seems nice too, but I don't have JRuby :)
|>
|> It's just a download away. ;)
|> jruby.codehouse.org
|>
|> However, you'll need a JVM that is compatible (IIRC, JRE 1.4.2 and
|> newer).
|
| www.jruby.org will get you there,

Right, codehaus, not house. *facepalm*

| and JRuby 1.1 requires Java 1.5 or
| higher.

Oops, my mistake (didn't the 1.0 series require only JRE 1.4.2, or so?).

- --
Phillip Gawlowski
Twitter: twitter.com/cynicalryan

~ - You know you've been hacking too long when...
...you see a flock of birds and try to figure out the algorithms that
determine their movement.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail....

iEYEARECAAYFAkgFimEACgkQbtAgaoJTgL9AzwCgpV+uKtcKsXrDUHnydsTzpUzP
X0cAn1cJhepq29B51Fp5f7oHytMyk21m
=KUqL
-----END PGP SIGNATURE-----