[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Forking job scheduler

Krishna Dole

9/29/2006 2:10:00 PM

Hi all,
If anyone is willing, I'd be grateful for some advice on the forking
job scheduler I've written. It works fine in simple tests, but does
not feel elegant. On IRC kbrooks recommended an asynchronous main
loop, but i don't understand how to implement that in this situation.
The first version I wrote used threads, but several sources
recommended fork instead. I have also considered just using the shell
command 'ps' to see how many jobs are running, launching more as
needed.

The basic requirements:
- Each job is a long-running external process (taking a day or more)
and all jobs require a different amount of time to run (so
asynchronous launching will be needed).
- I want to keep N jobs running at all times (N = 4 in the example below)

Thanks,
Krishna

##############################################

@jobs = (1..10).to_a # an imaginary list of jobs

# any time a job finishes, launch another
Signal.trap("CLD") { start_job unless @jobs.empty? }

def start_job
my_job = @jobs.pop
puts "starting job #{my_job}"
exec("sleep 2") if fork == nil # launch a job. in reality it would
run for a day or more
end

for num in 1..4 # i want to keep 4 jobs running at all times
start_job
end

# this doesn't wait for the last jobs to finish
while @jobs.size > 0
Process.wait
end

# this waits for the last jobs, but if i only had this line, it
wouldn't wait for all the jobs to start!
Process.wait
4 Answers

Krishna Dole

9/29/2006 2:46:00 PM

0

Hi Francis,

I actually had considered the cron approach, but wasn't sure if it was
the best way to do things. What you say makes a lot of sense (I was
already nervous about the watchdog running into trouble, and there are
no coordination requirements), so I will go with your suggestion.

Thanks!
Krishna

On 9/29/06, Francis Cianfrocca <garbagecat10@gmail.com> wrote:
> On 9/29/06, Krishna Dole <dontfall@gmail.com> wrote:
> >
> > Hi all,
> > If anyone is willing, I'd be grateful for some advice on the forking
> > job scheduler I've written. It works fine in simple tests, but does
> > not feel elegant. On IRC kbrooks recommended an asynchronous main
> > loop, but i don't understand how to implement that in this situation.
> > The first version I wrote used threads, but several sources
> > recommended fork instead. I have also considered just using the shell
> > command 'ps' to see how many jobs are running, launching more as
> > needed.
> >
> > The basic requirements:
> > - Each job is a long-running external process (taking a day or more)
> > and all jobs require a different amount of time to run (so
> > asynchronous launching will be needed).
> > - I want to keep N jobs running at all times (N = 4 in the example below)
>
>
>
> You say nothing about the coordination requirements of the external
> processes with the "watchdog" process. Is your requirement really just to
> ensure that four jobs are running at all times? If so, I would avoid using a
> long-running watchdog process, because you're making an assumption that it
> will never crash, catch a signal, etc. Why not run a cron job every five
> minutes or so that checks the running processes (via pgrep or ps as you
> suggested), starts more if necessary, writes status to syslog, and then
> quits? Much, much easier.
>
>

Ara.T.Howard

9/29/2006 2:56:00 PM

0

Ara.T.Howard

9/29/2006 2:57:00 PM

0

Ara.T.Howard

9/29/2006 3:45:00 PM

0