[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

[ANN] rq-3.0.0 : ruby queue gets gem'd

Ara.T.Howard

3/2/2007 2:03:00 AM

3 Answers

Ara.T.Howard

3/2/2007 3:56:00 PM

0

Jeremy Hinegardner

3/2/2007 4:20:00 PM

0

On Sat, Mar 03, 2007 at 12:55:32AM +0900, ara.t.howard@noaa.gov wrote:
>
> say you have 30 machines, like we do. if you want to use 'gem install rq'
> to
> actually populate an nfs installed location, then you first need ruby and
> sqlite installed on that location, then you need LD_RUN_PATH and
> LD_LIBRARY_PATH setup too, then you need to install the gem. so, intead,
> it's
> all in one. the tar ball with everything is also the gem so, now matter
> what
> you grab, you're good.
>
> the only reason i made a gem is so people could do
>
> gem install rq --no-wrappers
>
> and play around on the local machine but, in reality, rubygems is rather
> useless for managing a cluster - unless you're advanced, like you are.

Yeah right, I learn someting new every time you post :-). Especially
this time.

> now, having said that, i'm all ears as to how best manage rq's installation.
> here are the issues
>
> first some rubygems ones (arrgghh)
>
> - users cannot use rubygmes if ANY softlinks exist along the installation
> path - this is __very__ common when a shared nfs location is involved
>
> http://rubyforge.org/tracker/index.php?func=detail&aid=8811&group_id=126&am...
>
> - wrappers don't work
>
> http://rubyforge.org/tracker/index.php?func=detail&aid=8819&group_id=126&am...
> http://rubyforge.org/tracker/index.php?func=detail&aid=8985&group_id=126&am...
> http://rubyforge.org/tracker/index.php?func=detail&aid=8987&group_id=126&am...
>
> - finally, gem cannot be used easily to install a bunch of stuff into
> another location, even though, in theory, it's possible a.k.a
> LD_RUN_PATH/-R encoding. in otherwords, GEM_HOME isn't that useful
>
> http://rubyforge.org/tracker/index.php?func=detail&aid=8818&group_id=126&am...
>
> - depdencies don't work correctly when mixing local/remote installs
> (that's
> why i bundled them)
>
> second, some generic ones
>
> - people who actually use clusters (scientists) tend not to be too good at
> compiling things, especially into non-standard locations
>
> - people who use clusters tend to be on 'stable' (crappy) operating
> systems
> with auto-updates of crappy rubys. we're still on 1.6.8 with
> enterprise!!!!!!!! only our newest machines switched.
>
> - sqlite is a biatch to install. sqlite1,2,3, wtf? the ruby installer
> loses
> some information during install (i'm forgetting the issue attm). plus
> the
> ruby bindings have not remained backward comptible over time. same goes
> for the sql understood by the api. thus the choice to bundle it.
>
> so, my thinking was that an rq gem install is really just marketing:
> download it, play with it, then you have all the stuff locally to do a
> __real__ install onto your cluster without doing another thing.

Ah, I was thinking the gem would be what you would install on each
machine in the cluster. I'm flip flopped, thinking each machine in the
cluster would have local copies of all the libs/applications/etc that
it would need to run the cluters jobs, but instead each machine in the
cluster is more barebones and the common apps/libs/etc are all from the
NFS location.

> >And does rq need to do its own cleanup? There's a 'build' directory in
> >the gem which has the dross from building sqlite-2.8.15 and
> >sqlite-ruby-1.3.1.
>
> you mean it should blow it away after building? the only issue with that is
> that an error along the way means all the compilation is for nothing (you
> have
> to start over) but that's proably fine. i'll tweak it

Maybe a bad assumption on my part, after a successful build I would
think it would blow away the dross. Maybe if there are error/warning it
leaves it around.

This is probably the most complex gem install I've seen, I'll have to
dig through it and see how it all works :-).


> >I still want to create ActiveQueue based upon rq at some point.
>
> yeah, sure. i'm all ears. maybe we could get it out before mtnwest
> rubyconf!?

Probably not, I'm not going to make it to the conf.

> thanks a bunch for the feedback.

this was all just wondering how things worked :-).

enjoy,

-jeremy

--
========================================================================
Jeremy Hinegardner jeremy@hinegardner.org


Ara.T.Howard

3/2/2007 4:38:00 PM

0