[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Using fork to conserve memory

Daniel DeLorme

2/3/2007 7:42:00 AM

Lately I've been bothered by the large start-up time and memory consumption of
rails (although this could apply to any large framework). The solution I have in
mind is to load rails in one master process (slow, high memory) and then fork
child processes (fast, most mem shared with parent) to be used by apache. Are
there any projects out there that do something like this? Or am I gonna have to
make it myself?

Daniel
21 Answers

Jano Svitok

2/3/2007 8:40:00 AM

0

On 2/3/07, Daniel DeLorme <dan-ml@dan42.com> wrote:
> Lately I've been bothered by the large start-up time and memory consumption of
> rails (although this could apply to any large framework). The solution I have in
> mind is to load rails in one master process (slow, high memory) and then fork
> child processes (fast, most mem shared with parent) to be used by apache. Are
> there any projects out there that do something like this? Or am I gonna have to
> make it myself?

I'd say it should be possible to modify mongrel_cluster to do this
(maybe it even already does). I suppose that if you'll fork *after*
some processed requests, your memory gain will be better - as rails
uses lazy class loading - you want to do the fork after most of the
classes were loaded. OTOH, I guess that most of the memory is the
cache for objects, html snippets etc. and forking won't help you much
there - they cannot be shared.

Anyway, just try and see if it helps and what works best.

Ara.T.Howard

2/3/2007 3:50:00 PM

0

Ara.T.Howard

2/3/2007 3:52:00 PM

0

Daniel DeLorme

2/4/2007 3:56:00 AM

0

ara.t.howard@noaa.gov wrote:
> you realize that this is __exactly__ what running rails, or any cgi, under
> fastcgi does right?

No, fastcgi creates a bunch of worker processes and loads the full rails
environment in EACH of them. That means a lot of memory (30+ MB for each
process) and a long initialization time (2+ seconds for each process).

What I'm talking about is loading the large environment once and THEN forking
off worker processes that don't need to go through the expensive initialization
sequence. It seems like an obvious idea and rails is not the only framework with
a large footprint, so *someone* must have done something like this already.

Daniel

M. Edward (Ed) Borasky

2/4/2007 4:45:00 AM

0

Daniel DeLorme wrote:
> ara.t.howard@noaa.gov wrote:
>> you realize that this is __exactly__ what running rails, or any cgi,
>> under
>> fastcgi does right?
>
> No, fastcgi creates a bunch of worker processes and loads the full
> rails environment in EACH of them. That means a lot of memory (30+ MB
> for each process) and a long initialization time (2+ seconds for each
> process).
>
> What I'm talking about is loading the large environment once and THEN
> forking off worker processes that don't need to go through the
> expensive initialization sequence. It seems like an obvious idea and
> rails is not the only framework with a large footprint, so *someone*
> must have done something like this already.
>
> Daniel
>
>
What about Mongrel? Isn't that the "fastest" web server for Ruby? How
does Mongrel's memory footprint compare with the others?

Incidentally, speaking of fast web servers, how much can be gained (on a
Linux platform, of course) in a Rails server with Mongrel and Apache by
using Tux? Zed, any idea?

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blo...

If God had meant for carrots to be eaten cooked, He would have given rabbits fire.

Rob Sanheim

2/4/2007 5:25:00 AM

0

On 2/3/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> Daniel DeLorme wrote:
> > ara.t.howard@noaa.gov wrote:
> >> you realize that this is __exactly__ what running rails, or any cgi,
> >> under
> >> fastcgi does right?
> >
> > No, fastcgi creates a bunch of worker processes and loads the full
> > rails environment in EACH of them. That means a lot of memory (30+ MB
> > for each process) and a long initialization time (2+ seconds for each
> > process).
> >
> > What I'm talking about is loading the large environment once and THEN
> > forking off worker processes that don't need to go through the
> > expensive initialization sequence. It seems like an obvious idea and
> > rails is not the only framework with a large footprint, so *someone*
> > must have done something like this already.
> >
> > Daniel
> >
> >
> What about Mongrel? Isn't that the "fastest" web server for Ruby? How
> does Mongrel's memory footprint compare with the others?
>
> Incidentally, speaking of fast web servers, how much can be gained (on a
> Linux platform, of course) in a Rails server with Mongrel and Apache by
> using Tux? Zed, any idea?
>
> --

Mongrel will use up plenty of memory, generally around 30 megs per
mongrel to start. That will grow with your app, of course. Most
people who have limited memory go with a much leaner web server then
apache, but mongrel is still the preferred choice for serving Rails.

I'm not sure how the OP imagines these child processes would work -
Rails doesn't really have any sort of threading model where it could
hand out work to the child processes. A lot of Rails isn't even
threadsafe AFAIK. Thats why all the current deployment recipes have
one full rails environment per mongrel instance/fastcgi process.

Rob

Daniel DeLorme

2/4/2007 6:30:00 AM

0

Rob Sanheim wrote:
> I'm not sure how the OP imagines these child processes would work -
> Rails doesn't really have any sort of threading model where it could
> hand out work to the child processes. A lot of Rails isn't even
> threadsafe AFAIK. Thats why all the current deployment recipes have
> one full rails environment per mongrel instance/fastcgi process.

I was talking about processes, not threads. The point is to use the
copy-on-write capabilities of the OS to share the footprint of all the
code that is loaded upfront while maintaining the independance of each
process. i.e. http://en.wikipedia.org/wiki/Cop...

Daniel

khaines

2/4/2007 7:28:00 AM

0

Ara.T.Howard

2/4/2007 7:42:00 AM

0

Ara.T.Howard

2/4/2007 7:47:00 AM

0