Asp Forum - Memory Leak (again

Horacio Sanson

1/19/2006 10:56:00 AM

There is a little memory leak in my script I am not able to spot. I have tried
everything I found about detecting leaks but no luck.

I first tried all the recomendations found here

http://theexciter.com/articles/finding-leaking-ruby-objects?commented...

I can see the number of objects but the total number of objects does not
increment indefinitely but the script keeps consuming memory.

I tried this code:

##################################33
Entry = Struct.new( "MEntry", :c, :mem )
class MEntry; def to_s() "#{c} : #{mem}"; end; end

GroupEntry = Struct.new( "GroupEntry", :c, :mem, :total )
class GroupEntry; def to_s() "#{c} x#{total}(#{mem})"; end; end

def profile_mem
groups = {}
ObjectSpace.each_object { |x|
e = nil
begin
e = MEntry.new( x.class, Marshal::dump(x).size )
rescue TypeError # undumpable
e = MEntry.new( x.class, 0 )
end
if groups.has_key? e.c
groups[e.c].mem += e.mem
groups[e.c].total += 1
else
groups[e.c] = GroupEntry.new( e.c, e.mem, 1 )
end
}
File.open( "mem_log", "a+" ) { |file|
total = 0
groups.to_a.sort_by { |e| e[1].mem }.each { |e|
file << "#{e[1]} ";
total += e[1].mem
}
file << "TOTAL == #{total}"
file << "\n"
}
end

This code gives a list of all objects in ObjectSpace and their memory usage.
Also I log the total memory usage.

The log indicates that my scripts uses between 3 to 4 megs of memory
constantly, that is, it fluctuates between these values but the windows
manager indicates that the memory increase all the time and never goes down.

Running the script for a day consumes all 2G ram of my machine and causes the
script to bail out.

So are there any other tips on how to detect memory leaks?? Ruby Memory
Validator is only available to accepted beta testers so is not an option for
me.

I use Windows XP pro with the One-Click Ruby installer (1.8.2) and
ActiveRecord (1.13.2).

Any ideas are welcome...

regards,
Horacio

3 Answers

Stephen Kellett

1/19/2006 11:57:00 AM

In message <200601191952.21386.hsanson@moegi.waseda.jp>, Horacio Sanson
<hsanson@moegi.waseda.jp> writes
>So are there any other tips on how to detect memory leaks?? Ruby Memory
>Validator is only available to accepted beta testers so is not an option for
>me.

What makes you think you would not be accepted?

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/sof...
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting

Robert Klemme

1/19/2006 2:31:00 PM

Horacio Sanson wrote:
> There is a little memory leak in my script I am not able to spot. I
> have tried everything I found about detecting leaks but no luck.
>
> I first tried all the recomendations found here
>
>
http://theexciter.com/articles/finding-leaking-ruby-objects?commented...
>
> I can see the number of objects but the total number of objects does
> not increment indefinitely but the script keeps consuming memory.
>
> I tried this code:
>
> ##################################33
> Entry = Struct.new( "MEntry", :c, :mem )
> class MEntry; def to_s() "#{c} : #{mem}"; end; end
>
> GroupEntry = Struct.new( "GroupEntry", :c, :mem, :total )
> class GroupEntry; def to_s() "#{c} x#{total}(#{mem})"; end; end
>
> def profile_mem
> groups = {}
> ObjectSpace.each_object { |x|
> e = nil
> begin
> e = MEntry.new( x.class, Marshal::dump(x).size )
> rescue TypeError # undumpable
> e = MEntry.new( x.class, 0 )
> end
> if groups.has_key? e.c
> groups[e.c].mem += e.mem
> groups[e.c].total += 1
> else
> groups[e.c] = GroupEntry.new( e.c, e.mem, 1 )
> end
> }
> File.open( "mem_log", "a+" ) { |file|
> total = 0
> groups.to_a.sort_by { |e| e[1].mem }.each { |e|
> file << "#{e[1]} ";
> total += e[1].mem
> }
> file << "TOTAL == #{total}"
> file << "\n"
> }
> end
>
>
> This code gives a list of all objects in ObjectSpace and their memory
> usage. Also I log the total memory usage.

Your script does not determine the amount of individual objects. By doing
Marshal::dump(x).size you calculate the memory needed by a complete graph
of objects that is accessible from x. You'll count object size at least
twice. Also, you will see growing mem usage if you create a bunch of
objects and then stuff them in several collections.

> The log indicates that my scripts uses between 3 to 4 megs of memory
> constantly, that is, it fluctuates between these values but the
> windows manager indicates that the memory increase all the time and
> never goes down.

That's probably a better measure.

> Running the script for a day consumes all 2G ram of my machine and
> causes the script to bail out.

Difficult to tell what the problem is without seeing the script. A likely
cause is that you somehow store objects in some statically (i.e. from a
constant) accessible container.

Since the number of objects stays put another likely cause of memory
consumption is a String which gets appended to all the time. Do you maybe
store some kind of log info in mem?

Kind regards

robert

Eero Saynatkari

1/20/2006 1:26:00 AM

Horacio Sanson wrote:
> There is a little memory leak in my script I am not able to spot. I have
> tried
> everything I found about detecting leaks but no luck.
>
> I first tried all the recomendations found here
>
> http://theexciter.com/articles/finding-leaking-ruby-objects?commented...
>
> I can see the number of objects but the total number of objects does not
> increment indefinitely but the script keeps consuming memory.
>
> I tried this code:
>
> ##################################33
> Entry = Struct.new( "MEntry", :c, :mem )
> class MEntry; def to_s() "#{c} : #{mem}"; end; end
>
> GroupEntry = Struct.new( "GroupEntry", :c, :mem, :total )
> class GroupEntry; def to_s() "#{c} x#{total}(#{mem})"; end; end
>
> def profile_mem
> groups = {}
> ObjectSpace.each_object { |x|
> e = nil
> begin
> e = MEntry.new( x.class, Marshal::dump(x).size )
> rescue TypeError # undumpable
> e = MEntry.new( x.class, 0 )
> end
> if groups.has_key? e.c
> groups[e.c].mem += e.mem
> groups[e.c].total += 1
> else
> groups[e.c] = GroupEntry.new( e.c, e.mem, 1 )
> end
> }
> File.open( "mem_log", "a+" ) { |file|
> total = 0
> groups.to_a.sort_by { |e| e[1].mem }.each { |e|
> file << "#{e[1]} ";
> total += e[1].mem
> }
> file << "TOTAL == #{total}"
> file << "\n"
> }
> end
>
>
> This code gives a list of all objects in ObjectSpace and their memory
> usage.
> Also I log the total memory usage.
>
> The log indicates that my scripts uses between 3 to 4 megs of memory
> constantly, that is, it fluctuates between these values but the windows
> manager indicates that the memory increase all the time and never goes
> down.

The above logger seems to be reasonably good (although it
will not count Symbols and Fixnums) so there is a chance
this might be caused by your platform.

> Running the script for a day consumes all 2G ram of my machine and
> causes the
> script to bail out.
>
> So are there any other tips on how to detect memory leaks?? Ruby Memory
> Validator is only available to accepted beta testers so is not an option
> for
> me.
>
> I use Windows XP pro with the One-Click Ruby installer (1.8.2) and
> ActiveRecord (1.13.2).
>
> Any ideas are welcome...

If you can publish the script or a minimal version exhibiting
the same behaviour, we can try to troubleshoot it (or just run
it on different platforms in case the problem is caused by WinXP).

> regards,
> Horacio

E

--
Posted via http://www.ruby-....

comp.lang.ruby

Memory Leak (again

Horacio Sanson

Stephen Kellett

Robert Klemme

Eero Saynatkari

x Login to ForumsZone