Asp Forum - Multitasking and collecting results

Greg Willits

10/7/2007 5:58:00 AM

(new to ruby)

I have a routine that makes a web call then does some work. I need to do
many of these, and because of latency and all that, I want to go ahead
and spawn several of these in async "tasks" (trying to avoid key words
like processes and threads). I need to collect the results of each task.

So, generically, let's say I have a TaskManager and a Task. TaskManager
is initialized with an array of elements to process. Each element is
passed to a Task for processing. I want several Tasks processing in
parallel, and the result from each Task must get reported back to
TaskManager.

I see Ruby threads is one way, and despite the disparaging commens I see
about them, they're probably fine for me to get started with to figure
out the "feedback" mechanisms, but eventually I do need a method which
is capable of leveraging multi-core machines.

Using the pickaxe book Multithreading chapter, I've put together this
little outline, but the results always come back in order when I believe
useing sleep should randomize them, so I'm suspicioius that I'm not
really getting async processing out of my code.

#! /usr/local/bin/ruby

class TaskManager

attr_reader :results

def manageTasks
taskList = ['one', 'two', 'three']
threads = []
@results = []
indx = 0

taskList.each do |taskParam|
threads[indx] = Thread.new do
thisTask = Task.new
@results[indx] = thisTask.doStuff(taskParam)
end
indx += 1
end

threads.each {|thred| thred.join}
end
end

class Task
def doStuff(taskParam)
pause = ((rand(0.1))*4)
sleep pause
return ('handing back ' + taskParam + ', ' + pause.to_s)
end
end

taskMngr = TaskManager.new
taskMngr.manageTasks
puts taskMngr.results

Looking for hints as to what I am doing wrong.

Can anyone point me to a tutorial using a technique that will use
multi-core CPUs?

Many thanks.

-- gw
--
Posted via http://www.ruby-....

4 Answers

Greg Willits

10/7/2007 6:25:00 AM

Greg Willits wrote:
> I've put together this
> little outline, but the results always come back in order when I believe
> useing sleep should randomize them, so I'm suspicioius that I'm not
> really getting async processing out of my code.

Oh wait, they're in order because that's the order I have them inserted
into @results. Duh.

I confirmed async by putting a timer wrapper around the whole thing, and
indeed it completes in less time than the sum of each task's pause time.

-- gw
--
Posted via http://www.ruby-....

7stud 7stud

10/7/2007 11:32:00 AM

Greg Willits wrote:
> Greg Willits wrote:
>> I've put together this
>> little outline, but the results always come back in order when I believe
>> useing sleep should randomize them, so I'm suspicioius that I'm not
>> really getting async processing out of my code.
>
> Oh wait, they're in order because that's the order I have them inserted
> into @results. Duh.
>
> I confirmed async by putting a timer wrapper around the whole thing, and
> indeed it completes in less time than the sum of each task's pause time.
>

Try the following to see some random order in the results. The threads
in the code enter their data in a
Queue (located in the 'thread' module in the Standard Library). A Queue
enables you to pass data between threads in a "thread-safe manner". A
"thread safe manner" means that only one thread can access the Queue at
the same time. A Queue guarantees that two threads can't access the
Queue at the same time, so the data cannot get scrambled.

require 'thread'

class TaskManager

attr_reader :results

def initialize(*task_params)
@task_params = task_params
@results = Queue.new
end

def start_producers
threads = []

@task_params.each_with_index do |param, indx|
threads[indx] = Thread.new do
@results << Task.new.do_stuff(param)
end
end

threads.each {|thred| thred.join}
end
end

class Task
def do_stuff(x)
pause = rand() * 4
sleep pause

"handing back #{x}, #{pause}"
end
end

task_mngr = TaskManager.new('one', 'two', 'three')
task_mngr.start_producers
num_results = task_mngr.results.length
num_results.times {puts task_mngr.results.pop}

Note that rand(.1) is the same as rand(), so I don't know why pickaxe2
uses rand(.1).

--
Posted via http://www.ruby-....

Clifford Heath

10/7/2007 11:50:00 AM

Greg Willits wrote:
> I see Ruby threads is one way, and despite the disparaging commens I see

Ruby threads are a good thing - I wouldn't disparage them - but beware
that some Windows versions have broken synchronization primitives.

> Can anyone point me to a tutorial using a technique that will use
> multi-core CPUs?

Yes, but not using the current Ruby interpreter, which uses a single
operating-system thread, so cannot parallelize across a multi-core
system. If you need this with Ruby, you'll have to look at one of the
new interpreters.

Clifford Heath.

Tony Owens

11/4/2011 2:52:00 PM

People, that noticeable brown spot on Heinie's nose is directly attributable
to his goy rectum sucking compliments of Revd Bendover.

"Heinrich" <Heinrich@Ruhrgasnet.de> wrote in message
news:4eb3e175$0$12563$2e0edba0@news.tweakdsl.nl...
>
>
> "The Revd" <peeling@degenerate.Grik> schreef in bericht
> news:60o7b750srom2gucc8jsdgitsnsn4ava9v@4ax.com...
> > The jew shabbos starts soon! Brace yourselves for 24 hours of nonstop
> > jew rectum sucking!
> >
> > Enjoy already! Mazel tov! L'chaim! LOL
>
> viel Spass Jude
>

comp.lang.ruby

Multitasking and collecting results

Greg Willits

Greg Willits

7stud 7stud

Clifford Heath

Tony Owens

x Login to ForumsZone