Asp Forum - Testing for memory leaks

Berger, Daniel

9/17/2003 9:38:00 PM

Hi all,

Being as I write a lot of extensions, I've discovered that it's best to
check for memory leaks. The way I'm doing this now is something like this:

# pseudo code
while 1
call some_method
if Linux or BSD
check /proc/Process.pid/fd count
elsif Solaris
check pfiles Process.pid count
elsif Win32
start Task Manager and watch used file descriptors and memory for
Process.pid
end
sleep 1
end

# Ctrl-C to break out of loop only after at least 10 iterations

Is there a better and/or more streamlined way to do this? Any chance of
adding assert_no_memory_leak to TestUnit, for example?

Regards,

Dan

7 Answers

Mark J. Reed

9/17/2003 9:48:00 PM

On Thu, Sep 18, 2003 at 06:38:24AM +0900, Berger, Daniel wrote:
> Being as I write a lot of extensions, I''ve discovered that it''s best to
> check for memory leaks. The way I''m doing this now is something like this:

I''m confused - you talk about *memory* leaks, but your code checks for
*file descriptor* leaks. Which are you concerned about?

-Mark

Sean O'Dell

9/17/2003 10:39:00 PM

Berger, Daniel wrote:
> Hi all,
>
> Being as I write a lot of extensions, I''ve discovered that it''s best to
> check for memory leaks. The way I''m doing this now is something like this:
>
> # pseudo code
> while 1
> call some_method
> if Linux or BSD
> check /proc/Process.pid/fd count
> elsif Solaris
> check pfiles Process.pid count
> elsif Win32
> start Task Manager and watch used file descriptors and memory for
> Process.pid
> end
> sleep 1
> end
>
> # Ctrl-C to break out of loop only after at least 10 iterations
>
> Is there a better and/or more streamlined way to do this? Any chance of
> adding assert_no_memory_leak to TestUnit, for example?

The way I have been handling it is I''ve been launching an infinite loop
that creates instances of my extension and uses it, then calls GC.start
every iteration. I launch it as a background process and then run top
and watch its memory usage. If it hovers at a certain point and doesn''t
grow and grow, I assume its safe. To provide a positive, I first
deliberately put in code that should prevent the GC from collecting
something, then run top. Watch it grow and grow. Put the code back to
normal, watch it grow to a certain point and then hover.

Sean O''Dell

Lothar Scholz

9/17/2003 11:23:00 PM

Hello Daniel,

Wednesday, September 17, 2003, 10:38:24 PM, you wrote:

BD> Testing for memory leaksHi all,

BD> Being as I write a lot of extensions, I''ve discovered that
BD> it''s best to check for memory leaks.  The way I''m doing this now
BD> is something like this:

BD> Is there a better and/or more streamlined way to do this? 
BD> Any chance of adding assert_no_memory_leak to TestUnit, for
BD> example?

One of the best ways to detect memory leaks is to add the Boehm GC in
"leak report mode" to your app. This does not find everything but it
should be very easy to recompile with this enabled. Or search source
forge for the zillions of memory debugging libraries.

I''m not sure if ruby does free everything on normal exit. If not you
get some real problems with all this tools.

I would highly recommend to create a debugging ruby library and a
normal ruby library. There are much more things that could be added to
the debugging version at the C core level.

--
Best regards,
Lothar mailto:mailinglists@scriptolutions.com

nobu.nokada

9/18/2003 12:17:00 AM

Hi,

At Thu, 18 Sep 2003 06:38:24 +0900,
Berger, Daniel wrote:
> Being as I write a lot of extensions, I''ve discovered that it''s best to
> check for memory leaks. The way I''m doing this now is something like this:

This is the script "memusage.rb" I''m using on Linux. Requires
procfs mounted at /proc.

#! /usr/bin/ruby
module Memory
keys = []
vals = []
IO.foreach("/proc/self/status") do |l|
/^Vm(\w+):\s+(\d+)/ =~ l or next
keys << $1.downcase.intern
vals << $2.to_i
end
Status = Struct.new(*keys)
Status.module_eval do
const_set(:Header, " "+keys.collect {|k| k.to_s.upcase.rjust(6)}.join)
const_set(:Format, "%6d")
end
init = Status.new(*vals)
class Status
@@count = 0
def initialize
@count = @@count += 1
IO.foreach("/proc/self/status") do |l|
/^Vm(\w+):\s+(\d+)/ =~ l or next
self.__send__($1.downcase+"=", $2.to_i)
end
end
def to_s
"%4d: "%@count + collect {|n| Format % n}.join
end
end
puts Status::Header, init
end
if __FILE__ == $0
10.times do
puts Memory::Status.new
GC.start
end
end

> Is there a better and/or more streamlined way to do this? Any chance of
> adding assert_no_memory_leak to TestUnit, for example?

I don''t guess that assertion is easy. Generally, it''s
difficult to determin the memory is really leaked or not.

--
Nobu Nakada

ahoward

9/18/2003 4:22:00 AM

Simon Strandgaard

9/18/2003 1:01:00 PM

On Thu, 18 Sep 2003 09:22:30 +0900, Lothar Scholz wrote:

> I would highly recommend to create a debugging ruby library and a
> normal ruby library. There are much more things that could be added to
> the debugging version at the C core level.

Great Idea.. Would it be an idea to create a new target
in Ruby''s makefile which can build a extension-debug-version of Ruby ?

--
Simon Strandgaard

Brett H. Williams

9/18/2003 1:10:00 PM

On Sep 18, Berger, Daniel wrote:
> Hi all,
>
> Being as I write a lot of extensions, I''ve discovered that it''s best to
> check for memory leaks. The way I''m doing this now is something like this:
>
> # pseudo code
> while 1
> call some_method
> if Linux or BSD
> check /proc/Process.pid/fd count
> elsif Solaris
> check pfiles Process.pid count
> elsif Win32
> start Task Manager and watch used file descriptors and memory for
> Process.pid
> end
> sleep 1
> end
>
> # Ctrl-C to break out of loop only after at least 10 iterations
>
> Is there a better and/or more streamlined way to do this? Any chance of
> adding assert_no_memory_leak to TestUnit, for example?

If you only care about x86, you might out valgrind. It integrates nicely
with gdb and does a decent job without much effort to get going.

--
---------------------------------------------- | --------------------------
Brett Williams | (970) 288-0475
Agilent Technologies | brett_williams@agilent.com
---------------------------------------------- | --------------------------

comp.lang.ruby

Testing for memory leaks