[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

How to control thread number

Li Chen

9/15/2008 1:03:00 PM

Hi all,

I copy the following script from Ruby programming language(bundling
with Ruby). If the size for pages (array size) is small,thread works
very well. If the array size is big (such 2000) the thread doesn't work
at or very slow. I wonder what is the best means to control thread
number based on this script, read a small trunk of array at a time using
each_slice method?

Thanks,

Li

##################### script#####################
pages = %w( www.rubycentral.com
www.awl.com
www.pragmaticprogrammer.com
)

threads = []

for page in pages
threads << Thread.new(page) { |myPage|

h = Net::HTTP.new(myPage, 80)
puts "Fetching: #{myPage}"
resp, data = h.get('/', nil )
puts "Got #{myPage}: #{resp.message}"
}
end
--
Posted via http://www.ruby-....
3 Answers

ara.t.howard

9/15/2008 2:57:00 PM

0


On Sep 15, 2008, at 7:02 AM, Li Chen wrote:

> Hi all,
>
> I copy the following script from Ruby programming language(bundling
> with Ruby). If the size for pages (array size) is small,thread works
> very well. If the array size is big (such 2000) the thread doesn't
> work
> at or very slow. I wonder what is the best means to control thread
> number based on this script, read a small trunk of array at a time
> using
> each_slice method?
>
> Thanks,
>
> Li
>
> ##################### script#####################
> pages = %w( www.rubycentral.com
> www.awl.com
> www.pragmaticprogrammer.com
> )
>
> threads = []
>
> for page in pages
> threads << Thread.new(page) { |myPage|
>
> h = Net::HTTP.new(myPage, 80)
> puts "Fetching: #{myPage}"
> resp, data = h.get('/', nil )
> puts "Got #{myPage}: #{resp.message}"
> }
> end
> --
> Posted via http://www.ruby-....
>



this is exactly what threadify does

cfp:~ > cat a.rb
require 'open-uri'

require 'rubygems'
require 'threadify'

# gem install threadify
#
# http://codeforp...lib/ruby/threadify/threadify-0....

uris =
%w(
http://www.rub...
http://w...
http://www.pragmaticprog...
)

uris.threadify(2) do |uri|
tid = Thread.current.object_id
response = open(uri){|fd| fd.read}
puts "#{ tid } : #{ uri } => #{ response[0,42].inspect }"
end


cfp:~ > ruby a.rb
1867720 : http://www.rub... => "<html>\n<head>\n<title>Ruby
Land</title>\n<me"
1864060 : http://w... => "<!DOCTYPE html PUBLIC \"-//W3C//DTD
XHTML 1"
1867720 : http://www.pragmaticprog... => "<!DOCTYPE html PUBLIC
\"-//W3C//DTD XHTML 1"




this implementation is very short, you can read it so see one
technique - which uses sized queues with a fixed number of producers
and consumers


http://codeforp...lib/ruby/threadify/threadify-0.0.3/lib/th...


cheers.


a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

Lex Williams

9/15/2008 3:49:00 PM

0

are you calling join on the threads?
--
Posted via http://www.ruby-....

Erik Veenstra

9/15/2008 4:32:00 PM

0

That's exactly what ThreadLimiter is designed for:

http://www.erikveen.dds.nl/threadlimiter/doc/...

gegroet,
Erik V.