Asp Forum - [ANN] threadify-0.0.1

ara.t.howard

7/1/2008 8:04:00 PM

this one's for you charlie ;-)

NAME
threadify.rb

SYNOPSIS
enumerable = %w( a b c d )
enumerable.threadify(2){ 'process this block using two worker
threads' }

DESCRIPTION
threadify.rb makes it stupid easy to process a bunch of data using
'n'
worker threads

INSTALL
gem install threadify

URI
http://rub.../projects/cod...

SAMPLES

<========< sample/a.rb >========>

~ > cat sample/a.rb

require 'open-uri'
require 'yaml'

require 'rubygems'
require 'threadify'

uris =
%w(
http://...
http:/...
http://rub...
http://rub...
http:...
http://dra...
http://codefor...
)

time 'without threadify' do
uris.each do |uri|
body = open(uri){|pipe| pipe.read}
end
end

time 'with threadify' do
uris.threadify do |uri|
body = open(uri){|pipe| pipe.read}
end
end

BEGIN {
def time label
a = Time.now.to_f
yield
ensure
b = Time.now.to_f
y label => (b - a)
end
}

~ > ruby sample/a.rb

---
without threadify: 7.41900205612183
---
with threadify: 3.69886112213135

a @ http://codefor.../
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

18 Answers

fedzor

7/1/2008 8:20:00 PM

Thank you! This gem pretty much makes my life simpler, and will
continue to make it simpler!

(stdlib please?)

~ ari

Martin DeMello

7/1/2008 9:00:00 PM

On Tue, Jul 1, 2008 at 1:04 PM, ara howard <ara.t.howard@gmail.com> wrote:
>
> URI
> http://rubyforge.org/projects/cod...

I only see a tgz link which redirects me to

http://rubyforge.rubyuser.de/codeforpeople/threadify...

which in turn 404s

martin

Charles Oliver Nutter

7/1/2008 9:03:00 PM

Martin DeMello wrote:
> On Tue, Jul 1, 2008 at 1:04 PM, ara howard <ara.t.howard@gmail.com> wrote:
>> URI
>> http://rubyforge.org/projects/cod...
>
> I only see a tgz link which redirects me to
>
> http://rubyforge.rubyuser.de/codeforpeople/threadify...
>
> which in turn 404s

mirror delay. check codeforpeople svn, it's only one file.

- Charlie

Charles Oliver Nutter

7/1/2008 9:05:00 PM

ara howard wrote:
>
>
> this one's for you charlie ;-)

Appears to work just dandy under JRuby:

â?? time jruby --server -rthreadify -e "nums = *(1..35); def fib(n); if n
< 2; return n; else; return fib(n - 1) + fib(n - 2); end; end; nums.each
{|i| p fib(i)}"
...
real 0m11.889s
user 0m11.733s
sys 0m0.188s
~/NetBeansProjects/jruby â?? time jruby --server -rthreadify -e "nums =
*(1..35); def fib(n); if n < 2; return n; else; return fib(n - 1) +
fib(n - 2); end; end; nums.threadify {|i| p fib(i)}"
...
real 0m8.213s
user 0m12.722s
sys 0m0.178s

(One thread on my system consumes roughly 65-70% CPU, which explains why
full CPU on both cores doesn't double performance here)

I also found some weird bug where Thread#kill/exit from within the
thread interacts weirdly with join happening outside, and never
terminates. Fixing that now.

- Cahrlie

Martin DeMello

7/1/2008 9:11:00 PM

On Tue, Jul 1, 2008 at 2:02 PM, Charles Oliver Nutter
<charles.nutter@sun.com> wrote:
> mirror delay. check codeforpeople svn, it's only one file.

thanks, gotit. will also install the gem when it propagates, just to
keep my system informed :)

m.

ara.t.howard

7/1/2008 9:35:00 PM

On Jul 1, 2008, at 3:04 PM, Charles Oliver Nutter wrote:

> Appears to work just dandy under JRuby:
>
> =E2=9E=94 time jruby --server -rthreadify -e "nums =3D *(1..35); def =
fib(n); =20
> if n < 2; return n; else; return fib(n - 1) + fib(n - 2); end; end; =20=

> nums.each {|i| p fib(i)}"

wow that's cool - now that's a a seriously easy way to parallelize ;-)

> I also found some weird bug where Thread#kill/exit from within the =20
> thread interacts weirdly with join happening outside, and never =20
> terminates. Fixing that now.

glad to have helped ;-)

i just pushed out 0.0.2 and it just lets the thread die rather that =20
self-destructing. see how that works...

cheers.

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being =20
better. simply reflect on that.
h.h. the 14th dalai lama

ara.t.howard

7/1/2008 9:35:00 PM

On Jul 1, 2008, at 3:11 PM, Martin DeMello wrote:

> thanks, gotit. will also install the gem when it propagates, just to
> keep my system informed :)

0.0.2 and gem should be up

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

ara.t.howard

7/1/2008 9:35:00 PM

Charles Oliver Nutter

7/1/2008 9:47:00 PM

ara.t.howard wrote:
> i just pushed out 0.0.2 and it just lets the thread die rather that
> self-destructing. see how that works...

I fixed in JRuby just now (Thread#kill does an implicit join in JRuby to
make sure the thread dies...but if target == caller it was still trying
to join itself in a weird way) but basically breaking out of the loop
instead of Thread#exit solved it. Your 0.0.2 change is probably equivalent.

- Charlie

Michael Guterl

7/8/2008 11:44:00 PM

On Tue, Jul 1, 2008 at 5:04 PM, Charles Oliver Nutter
<charles.nutter@sun.com> wrote:
> ara howard wrote:
>>
>>
>> this one's for you charlie ;-)
>
> Appears to work just dandy under JRuby:
>

I was doing some comparison between threadify and peach with JRuby,
when I noticed some interesting behavior with using
Enumerator#to_enum.

Sample code and three different results are posted here:
http://pastie..... Each result randomly occurs and sometimes
the code produces no error whatsoever. MRI does not seem to exhibit
the same behavior.

I am not sure that what I am doing in the code is even reasonable,
however, I thought it might be worth pointing out.

threadify-0.0.2
jruby 1.1.3-dev (ruby 1.8.6 patchlevel 114) (2008-07-08 rev 7130) [i386-java]
java version "1.5.0_13"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05-237)
Java HotSpot(TM) Client VM (build 1.5.0_13-119, mixed mode, sharing)

OS X 10.5.4

Thanks,
Michael Guterl

comp.lang.ruby

[ANN] threadify-0.0.1

ara.t.howard

fedzor

Martin DeMello

Charles Oliver Nutter

Charles Oliver Nutter

Martin DeMello

ara.t.howard

ara.t.howard

ara.t.howard

Charles Oliver Nutter

Michael Guterl

x Login to ForumsZone