Asp Forum - Re: Ruby Scales just fine !

Yukihiro Matsumoto

9/26/2007 12:39:00 AM

Hi,

In message "Re: Ruby Scales just fine !"
on Wed, 26 Sep 2007 09:28:36 +0900, Ari Brown <ari@aribrown.com> writes:

|Soo........ Ruby DOES use multiple cores? I read somewhere that it
|didn't (probably this thread).

Single Ruby process does not. But we can use as much cores we have by
forking processes, and usually it's the case for web applications.

9 Answers

Ruby Maniac

9/26/2007 12:51:00 AM

On Sep 25, 5:38 pm, Yukihiro Matsumoto <m...@ruby-lang.org> wrote:
> Hi,
>
> In message "Re: Ruby Scales just fine !"
> on Wed, 26 Sep 2007 09:28:36 +0900, Ari Brown <a...@aribrown.com> writes:
>
> |Soo........ Ruby DOES use multiple cores? I read somewhere that it
> |didn't (probably this thread).
>
> Single Ruby process does not. But we can use as much cores we have by
> forking processes, and usually it's the case for web applications.

Ruby would have to use real OS threads to take advantage of how
threads may be assigned to more than one core or more than one CPU,
etc, depending on how the parent process was spawned in terms of the
affinity for CPU's or cores.

Ari Brown

9/26/2007 1:25:00 AM

On Sep 25, 2007, at 8:38 PM, Yukihiro Matsumoto wrote:

> Hi,
>
> In message "Re: Ruby Scales just fine !"
> on Wed, 26 Sep 2007 09:28:36 +0900, Ari Brown
> <ari@aribrown.com> writes:
>
> |Soo........ Ruby DOES use multiple cores? I read somewhere that it
> |didn't (probably this thread).
>
> Single Ruby process does not. But we can use as much cores we have by
> forking processes, and usually it's the case for web applications.

So fork() moves to a new processor if there is one, else a new thread?

-------------------------------------------|
Nietzsche is my copilot

Marc Heiler

9/26/2007 1:54:00 AM

Ruby Maniac, you seem to like this recurrent topic, but why not give it
a
little rest before posting a new thread with the same old speed
arguments?
Its not as if your speed problems magically go away from
one day to the other, and I guess we know your opinion by now too ;)

You seem to continually start new and new threads about this.
--
Posted via http://www.ruby-....

James Gray

9/26/2007 2:58:00 AM

On Sep 25, 2007, at 8:24 PM, Ari Brown wrote:

>
> On Sep 25, 2007, at 8:38 PM, Yukihiro Matsumoto wrote:
>
>> Hi,
>>
>> In message "Re: Ruby Scales just fine !"
>> on Wed, 26 Sep 2007 09:28:36 +0900, Ari Brown
>> <ari@aribrown.com> writes:
>>
>> |Soo........ Ruby DOES use multiple cores? I read somewhere that it
>> |didn't (probably this thread).
>>
>> Single Ruby process does not. But we can use as much cores we
>> have by
>> forking processes, and usually it's the case for web applications.
>
> So fork() moves to a new processor if there is one, else a new thread?

fork() creates another process which your schedular is free to handle
as it sees best. This may mean running that process on a different
CPU, if the schedular decides that's the way to go.

James Edward Gray II

Alex Young

9/26/2007 8:16:00 AM

James Edward Gray II wrote:
> On Sep 25, 2007, at 8:24 PM, Ari Brown wrote:
>
>>
>> On Sep 25, 2007, at 8:38 PM, Yukihiro Matsumoto wrote:
>>
>>> Hi,
>>>
>>> In message "Re: Ruby Scales just fine !"
>>> on Wed, 26 Sep 2007 09:28:36 +0900, Ari Brown <ari@aribrown.com>
>>> writes:
>>>
>>> |Soo........ Ruby DOES use multiple cores? I read somewhere that it
>>> |didn't (probably this thread).
>>>
>>> Single Ruby process does not. But we can use as much cores we have by
>>> forking processes, and usually it's the case for web applications.
>>
>> So fork() moves to a new processor if there is one, else a new thread?
>
> fork() creates another process which your schedular is free to handle as
> it sees best. This may mean running that process on a different CPU, if
> the schedular decides that's the way to go.
OT, but is there a combination of libraries/OS/network-level
accoutrements that would allow a scheduler to migrate the process to
another host on fork()?

--
Alex

M. Edward (Ed) Borasky

9/26/2007 2:47:00 PM

Alex Young wrote:
> James Edward Gray II wrote:
>> On Sep 25, 2007, at 8:24 PM, Ari Brown wrote:
>>
>>>
>>> On Sep 25, 2007, at 8:38 PM, Yukihiro Matsumoto wrote:
>>>
>>>> Hi,
>>>>
>>>> In message "Re: Ruby Scales just fine !"
>>>> on Wed, 26 Sep 2007 09:28:36 +0900, Ari Brown <ari@aribrown.com>
>>>> writes:
>>>>
>>>> |Soo........ Ruby DOES use multiple cores? I read somewhere that it
>>>> |didn't (probably this thread).
>>>>
>>>> Single Ruby process does not. But we can use as much cores we have by
>>>> forking processes, and usually it's the case for web applications.
>>>
>>> So fork() moves to a new processor if there is one, else a new thread?
>>
>> fork() creates another process which your schedular is free to handle
>> as it sees best. This may mean running that process on a different
>> CPU, if the schedular decides that's the way to go.
> OT, but is there a combination of libraries/OS/network-level
> accoutrements that would allow a scheduler to migrate the process to
> another host on fork()?
>

If you mean the "native" scheduler that's built in to the OS, you or
someone would have to write a library extension in C to make that
happen, which is OS dependent, and in the case of Linux, kernel version
dependent as well.

In general, however, unless you *really* *really* know what you're doing
and *why* you're doing it, you're better off letting the OS scheduler do
its things without even trying to give it *hints* on how to treat your
code. It's not nice to fool the scheduler -- that way lies deadlocks,
race conditions, starving philosophers, etc.

Just throw hardware at it -- lots of cores, lots of RAM, lots of
light-weight processes communicating by message passage and immutable
objects a la Erlang. The next version of Event Machine will have the
underlying mechanisms to make this work.

Rick DeNatale

9/26/2007 5:40:00 PM

Here's a recent presentation from RailsConf Europe (actually Jason has
given a similar talk several places) which puts the leverage of the
language/application framework in perspective for web apps.

http://jxh.bingodisk.com/bingo/public/presentations/JHoffmanRailsConf-Berlin-Se...

Anyone who thinks that it's essential for Ruby to have better thread
support to survive, and has an open mind MIGHT think differently after
reading this.

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denh...

Jay Levitt

9/27/2007 5:12:00 PM

On Thu, 27 Sep 2007 02:40:06 +0900, Rick DeNatale wrote:

> Here's a recent presentation from RailsConf Europe (actually Jason has
> given a similar talk several places) which puts the leverage of the
> language/application framework in perspective for web apps.
>
> http://jxh.bingodisk.com/bingo/public/presentations/JHoffmanRailsConf-Berlin-Se...
>
> Anyone who thinks that it's essential for Ruby to have better thread
> support to survive, and has an open mind MIGHT think differently after
> reading this.

That is a truly great presentation.

The *only* thing I might quibble with is the "one vendor" suggestion. If
you're really going to be that big, you don't really want to tie your fate
to one vendor:

* If you're small potatoes, you're not important to them
* If you're the big cheese, you can overwhelm their capacity
* Either way, you have no leverage if they have no competition
* You'll get more data on relative failure rates, performance, bugs, etc.

One *configuration*, yes, but one vendor leads to heartache.

As AOL was scaling in the 1990s, we nearly always had two major vendors for
any one component, plus at least one experimental vendor. We discovered
very quickly which big-iron kernels had better TCP stack performance, which
hard drive brands failed more often, etc. And we got great prices, because
we could always to go The Other Guy.

In the end, the biggest "single vendor" we were tied to was the RBOCs,
which couldn't build local exchanges fast enough to provide enough dial
tones to local customers, since we were seriously messing with their
simultaneous-user projections, all of which assumed voice rather than data.
Probably not an issue anymore.

--
Jay Levitt |
Boston, MA | My character doesn't like it when they
Faster: jay at jay dot fm | cry or shout or hit.
http://... | - Kristoffer

brabuhr

9/27/2007 9:06:00 PM

On 9/26/07, Alex Young <alex@blackkettle.org> wrote:
> James Edward Gray II wrote:
> > On Sep 25, 2007, at 8:24 PM, Ari Brown wrote:
> > fork() creates another process which your schedular is free to handle as
> > it sees best. This may mean running that process on a different CPU, if
> > the schedular decides that's the way to go.
>
> OT, but is there a combination of libraries/OS/network-level
> accoutrements that would allow a scheduler to migrate the process to
> another host on fork()?

http://www....
http://openmosix.source...

http://en.wikipedia.org/...
"The latest version of MOSIX, called MOSIX2, is compatible
with Linux-2.6. MOSIX2 is implemented as an OS virtuallization
layer that provides to users and applications an SSI with
the Linux run-time environment. It allows applications to
run in remote nodes as if they run locally. Users run their
regular (sequential and parallel) applications while MOSIX
transparently and automatically seek resources and migrate
processes among nodes to improve the overall performance."

comp.lang.ruby

Re: Ruby Scales just fine !