[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Memory leak??? (top reporting high memory usage under Solaris

Mark Probert

2/9/2005 7:27:00 AM


Hi ..

Has anyone seen anything like this?

load averages: 0.63, 0.30, 0.16
02:07:51
134 processes: 118 sleeping, 14 zombie, 2 on cpu
CPU states: 77.0% idle, 16.8% user, 5.6% kernel, 0.6% iowait, 0.0% swap
Memory: 4096M real, 2984M free, 2952M swap in use, 2357M swap free

PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
6683 root 1 0 4 1744M 94M cpu0 1:18 16.93% healthcollect.r
28954 bwczkdj 1 58 0 2608K 1792K sleep 10:55 0.49% top
...

The 13 mins later:

load averages: 0.57, 0.65, 0.48
02:20:11
139 processes: 124 sleeping, 14 zombie, 1 on cpu
CPU states: 93.9% idle, 3.7% user, 2.4% kernel, 0.0% iowait, 0.0% swap
Memory: 4096M real, 2678M free, 5294M swap in use, 15M swap free

PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
6683 root 1 20 4 4082M 291M sleep 9:15 12.43% healthcollect.r
...

Notice that the reported memory size of the Ruby application is growing? All
the swap gets consumed and the process dies. Looks like a memory leak
somewhere, though I could well be mistaken. Certainly, the process reporting
that it is using 4Gb of memory raise my eye brows.

As background, this script runs data collection against about 220 network
elements. The script is multithreaded, with the current thread count at 15.
Each thread will have a dedicated telnet session to the network element. The
data collected is sent straight to disk, then post-processed.

I don't have access to upgrading the hardware, the swap size at all, or the
ruby version easily.

$ ruby -v
ruby 1.8.0 (2003-08-04) [sparc-solaris2.8]

Regards,

--
-mark. (probertm at acm dot org)
6 Answers

Bill Kelly

2/9/2005 8:02:00 AM

0

From: "Mark Probert" <probertm@acm.org>
>
> I don't have access to upgrading the hardware, the swap size at all, or the
> ruby version easily.
>
> $ ruby -v
> ruby 1.8.0 (2003-08-04) [sparc-solaris2.8]

Sorry if you already were aware of this; but you should
typically be able to compile and install a version of
ruby in your home directory without any superuser privileges.
( ./configure --prefix ~/somedir )

Just wondering if that would give you the opportunity to
try a different ruby version?


Regards,

Bill

Thomas Counsell

2/9/2005 10:29:00 AM

0

Hello

With my multithreaded web-rick apps an exception thrown somewhere in
the code sometimes causes the memory used by ruby to spiral out of
control. True for me on Mac OS X with various 1.8.x installs. Hard
for me to repeat though, and when I have cleared all the exception
throwing bugs it goes away.

Not a lot of use, but perhaps a clue. I discovered it was the
exceptions causing the spiral by running the code with the tracer
enabled (which requires a lot of patience for an app of any size)

Tom


On 9 Feb 2005, at 07:27, Mark Probert wrote:

>
> Hi ..
>
> Has anyone seen anything like this?
>
> load averages: 0.63, 0.30, 0.16
> 02:07:51
> 134 processes: 118 sleeping, 14 zombie, 2 on cpu
> CPU states: 77.0% idle, 16.8% user, 5.6% kernel, 0.6% iowait, 0.0%
> swap
> Memory: 4096M real, 2984M free, 2952M swap in use, 2357M swap free
>
> PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
> 6683 root 1 0 4 1744M 94M cpu0 1:18 16.93%
> healthcollect.r
> 28954 bwczkdj 1 58 0 2608K 1792K sleep 10:55 0.49% top
> ...
>
> The 13 mins later:
>
> load averages: 0.57, 0.65, 0.48
> 02:20:11
> 139 processes: 124 sleeping, 14 zombie, 1 on cpu
> CPU states: 93.9% idle, 3.7% user, 2.4% kernel, 0.0% iowait, 0.0%
> swap
> Memory: 4096M real, 2678M free, 5294M swap in use, 15M swap free
>
> PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
> 6683 root 1 20 4 4082M 291M sleep 9:15 12.43%
> healthcollect.r
> ...
>
> Notice that the reported memory size of the Ruby application is
> growing? All
> the swap gets consumed and the process dies. Looks like a memory leak
> somewhere, though I could well be mistaken. Certainly, the process
> reporting
> that it is using 4Gb of memory raise my eye brows.
>
> As background, this script runs data collection against about 220
> network
> elements. The script is multithreaded, with the current thread count
> at 15.
> Each thread will have a dedicated telnet session to the network
> element. The
> data collected is sent straight to disk, then post-processed.
>
> I don't have access to upgrading the hardware, the swap size at all,
> or the
> ruby version easily.
>
> $ ruby -v
> ruby 1.8.0 (2003-08-04) [sparc-solaris2.8]
>
> Regards,
>
> --
> -mark. (probertm at acm dot org)
>

llothar

2/9/2005 12:13:00 PM

0


Thomas Counsell wrote:
> Hello
>
> With my multithreaded web-rick apps an exception thrown somewhere in
> the code sometimes causes the memory used by ruby to spiral out of
> control. True for me on Mac OS X with various 1.8.x installs. Hard
> for me to repeat though, and when I have cleared all the exception
> throwing bugs it goes away.
>
> Not a lot of use, but perhaps a clue. I discovered it was the
> exceptions causing the spiral by running the code with the tracer
> enabled (which requires a lot of patience for an app of any size)

Correct. Exceptions and the heavy use of ccalloc can result in this
memory leak. I got this report also from 2 of my ArachnoRuby users, but
somebody must find a small reproduceable script for this.

Mark Probert

2/9/2005 6:11:00 PM

0

Hi ..

On Wednesday 09 February 2005 00:01, Bill Kelly wrote:
>
> Just wondering if that would give you the opportunity to
> try a different ruby version?
>
Unfortunately, the customer here is really strict about software on their
production servers. I even need someone to run the commands whilst I watch
over their shoulder :-). To get a different version of Ruby is a process of
many weeks. Though it will happen in the end ..

Regards,

-mark.

Mark Probert

2/9/2005 6:14:00 PM

0

Hi ..

On Wednesday 09 February 2005 04:15, llothar wrote:
> Thomas Counsell wrote:
> > Hello
> >
> > With my multithreaded web-rick apps an exception thrown somewhere in
> > the code sometimes causes the memory used by ruby to spiral out of
> > control.
>
> Correct. Exceptions and the heavy use of ccalloc can result in this
> memory leak. I got this report also from 2 of my ArachnoRuby users, but
> somebody must find a small reproduceable script for this.

Thanks. I'll see what I can do.

So exceptions, even caught ones, will trigger a memory leak? Won't the GC
kick in and clean them up? Or is it only uncaught exceptions?


--
-mark. (probertm at acm dot org)

Thomas Counsell

2/10/2005 9:59:00 AM

0

Not sure, because 1) it hasn't been that repeatable for me and 2) I'm
pretty ignorant about threading and 3) even more ignorant about how the
garbage collector works* .

My experience has been that exceptions created by deliberate 'raise'
messages have been ok, and exceptions created by poor coding (e.g.
NameError: undefined local variable or method ...) can cause the memory
to be consumed even if I've surrounded everything with a
begin...rescue.

Again, only clues I'm afraid. Next time it happens, I'll spend some
time trying to develop a repeatable case so that I can get the lists
combined brainpower on it.

Tom
* One of the reasons I haven't really raised this issue much is because
I've always assumed it was my poor coding

On 9 Feb 2005, at 18:13, Mark Probert wrote:
> So exceptions, even caught ones, will trigger a memory leak? Won't
> the GC
> kick in and clean them up? Or is it only uncaught exceptions?