[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

How to find out memory leak?

Chiyuan Zhang

2/25/2008 3:28:00 PM

Hi, all!

I'm having memory leak problem with my program, but can't find where's
the leak.
I have a program written in pure-Ruby. It implemented a Chinese word
segment
algorithm. I call `segment(text)' to get the result. But each time I
call `segment',
the memory usage of my program (Ruby) increased several handred K-
bytes -- the
number of bytes of increment is roughly equal during each call.

I can see from outside (using `ps' or `top') that the memory is
leaking. But I can't
find where the leaking goes. I tried to use ObjectSpace.each_object to
find out
what leaks. I followed this article

http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-...

and compare the objects after each call to `segment'. What surprise me
is that
there's no object leaked between each call. The number of objects
after each
call is roughly equal.

I also tried bleak_house:

http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/R...

I don't precisely how to use it. I just dump a snapshot after each
call to `segment'
and use `bleak' to analyze the result. However, it also shows no leak.
But if
there's no leak, where goes the memory?

Here's the result of using the `pmap' tool to inspect the running Ruby
process:

933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many
08048000 4K r-x-- /usr/bin/ruby1.8
08049000 4K rw--- /usr/bin/ruby1.8
0804a000 33412K rw--- [ anon ]
b74a8000 7016K rw--- [ anon ]
b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so
b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
ruby_prof.so
b7b8c000 1192K rw--- [ anon ]
b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so
b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so
b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so
b7e00000 12K rw--- [ anon ]
b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so
b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so
b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so
b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so
b7e33000 156K rw--- [ anon ]
b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so
b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so
b7e5e000 4K rw--- [ anon ]
b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so
b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so
b7e75000 8K rw--- [ anon ]
b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6
b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6
b7f36000 64K rw--- [ anon ]
b7f48000 8K rw--- [ anon ]
b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so
b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so
b7f51000 8K rw--- [ anon ]
b7f53000 104K r-x-- /lib/ld-2.7.so
b7f6d000 8K rw--- /lib/ld-2.7.so
bfdea000 116K rw--- [ stack ]
ffffe000 4K r-x-- [ anon ]
total 44568K

The section "0804a000 33412K rw--- [ anon ]" increased after each
call. I've no idea now. Do you have any suggestion for me?
2 Answers

Jano Svitok

2/25/2008 3:51:00 PM

0

On Mon, Feb 25, 2008 at 4:29 PM, pluskid <pluskid@gmail.com> wrote:
> Hi, all!
>
> I'm having memory leak problem with my program, but can't find where's
> the leak.
> I have a program written in pure-Ruby. It implemented a Chinese word
> segment
> algorithm. I call `segment(text)' to get the result. But each time I
> call `segment',
> the memory usage of my program (Ruby) increased several handred K-
> bytes -- the
> number of bytes of increment is roughly equal during each call.
>
> I can see from outside (using `ps' or `top') that the memory is
> leaking. But I can't
> find where the leaking goes. I tried to use ObjectSpace.each_object to
> find out
> what leaks. I followed this article
>
> http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-...
>
> and compare the objects after each call to `segment'. What surprise me
> is that
> there's no object leaked between each call. The number of objects
> after each
> call is roughly equal.
>
> I also tried bleak_house:
>
> http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/R...
>
> I don't precisely how to use it. I just dump a snapshot after each
> call to `segment'
> and use `bleak' to analyze the result. However, it also shows no leak.
> But if
> there's no leak, where goes the memory?
>
> Here's the result of using the `pmap' tool to inspect the running Ruby
> process:
>
> 933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many
> 08048000 4K r-x-- /usr/bin/ruby1.8
> 08049000 4K rw--- /usr/bin/ruby1.8
> 0804a000 33412K rw--- [ anon ]
> b74a8000 7016K rw--- [ anon ]
> b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so
> b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so
> b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
> ruby_prof.so
> b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
> ruby_prof.so
> b7b8c000 1192K rw--- [ anon ]
> b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so
> b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so
> b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so
> b7e00000 12K rw--- [ anon ]
> b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so
> b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so
> b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so
> b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so
> b7e33000 156K rw--- [ anon ]
> b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so
> b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so
> b7e5e000 4K rw--- [ anon ]
> b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so
> b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so
> b7e75000 8K rw--- [ anon ]
> b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6
> b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6
> b7f36000 64K rw--- [ anon ]
> b7f48000 8K rw--- [ anon ]
> b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so
> b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so
> b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so
> b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so
> b7f51000 8K rw--- [ anon ]
> b7f53000 104K r-x-- /lib/ld-2.7.so
> b7f6d000 8K rw--- /lib/ld-2.7.so
> bfdea000 116K rw--- [ stack ]
> ffffe000 4K r-x-- [ anon ]
> total 44568K
>
> The section "0804a000 33412K rw--- [ anon ]" increased after each
> call. I've no idea now. Do you have any suggestion for me?

This is what I have read here or somewhere else, and I an not
knowledgeable enough to confirm it, anyway:
Ruby allocs space from the OS in large chunks (initially 8 MB, and
each other chunk is bigger by a predefined coefficient).
Any chunk can be returned to the OS only if it is empty - so if your
memory is fragmented, ruby will have to alloc new chunks
in order to satisfy ruby script being run.

If this is true, it means: the order and size of your allocations is important.
Especially order of the allocations that you later release relative to
those that you don't.

Chiyuan Zhang

2/26/2008 10:01:00 AM

0

If that is the case, that might be quite random, istead
of seeing the increment of every call roughly equal. :(

2008/2/25, Jano Svitok <jan.svitok@gmail.com>:
> On Mon, Feb 25, 2008 at 4:29 PM, pluskid <pluskid@gmail.com> wrote:
> > Hi, all!
> >
> > I'm having memory leak problem with my program, but can't find where's
> > the leak.
> > I have a program written in pure-Ruby. It implemented a Chinese word
> > segment
> > algorithm. I call `segment(text)' to get the result. But each time I
> > call `segment',
> > the memory usage of my program (Ruby) increased several handred K-
> > bytes -- the
> > number of bytes of increment is roughly equal during each call.
> >
> > I can see from outside (using `ps' or `top') that the memory is
> > leaking. But I can't
> > find where the leaking goes. I tried to use ObjectSpace.each_object to
> > find out
> > what leaks. I followed this article
> >
> > http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-...
> >
> > and compare the objects after each call to `segment'. What surprise me
> > is that
> > there's no object leaked between each call. The number of objects
> > after each
> > call is roughly equal.
> >
> > I also tried bleak_house:
> >
> > http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/R...
> >
> > I don't precisely how to use it. I just dump a snapshot after each
> > call to `segment'
> > and use `bleak' to analyze the result. However, it also shows no leak.
> > But if
> > there's no leak, where goes the memory?
> >
> > Here's the result of using the `pmap' tool to inspect the running Ruby
> > process:
> >
> > 933: /usr/bin/ruby1.8 /usr/bin/rake PROFILE=false test:many
> > 08048000 4K r-x-- /usr/bin/ruby1.8
> > 08049000 4K rw--- /usr/bin/ruby1.8
> > 0804a000 33412K rw--- [ anon ]
> > b74a8000 7016K rw--- [ anon ]
> > b7b82000 16K r-x-- /usr/lib/ruby/1.8/i486-linux/strscan.so
> > b7b86000 4K rw--- /usr/lib/ruby/1.8/i486-linux/strscan.so
> > b7b87000 16K r-x-- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
> > ruby_prof.so
> > b7b8b000 4K rw--- /var/lib/gems/1.8/gems/ruby-prof-0.6.0/lib/
> > ruby_prof.so
> > b7b8c000 1192K rw--- [ anon ]
> > b7cb6000 1308K r-x-- /lib/i686/cmov/libc-2.7.so
> > b7dfd000 4K r---- /lib/i686/cmov/libc-2.7.so
> > b7dfe000 8K rw--- /lib/i686/cmov/libc-2.7.so
> > b7e00000 12K rw--- [ anon ]
> > b7e03000 140K r-x-- /lib/i686/cmov/libm-2.7.so
> > b7e26000 8K rw--- /lib/i686/cmov/libm-2.7.so
> > b7e28000 36K r-x-- /lib/i686/cmov/libcrypt-2.7.so
> > b7e31000 8K rw--- /lib/i686/cmov/libcrypt-2.7.so
> > b7e33000 156K rw--- [ anon ]
> > b7e5a000 8K r-x-- /lib/i686/cmov/libdl-2.7.so
> > b7e5c000 8K rw--- /lib/i686/cmov/libdl-2.7.so
> > b7e5e000 4K rw--- [ anon ]
> > b7e5f000 80K r-x-- /lib/i686/cmov/libpthread-2.7.so
> > b7e73000 8K rw--- /lib/i686/cmov/libpthread-2.7.so
> > b7e75000 8K rw--- [ anon ]
> > b7e77000 756K r-x-- /usr/lib/libruby1.8.so.1.8.6
> > b7f34000 8K rw--- /usr/lib/libruby1.8.so.1.8.6
> > b7f36000 64K rw--- [ anon ]
> > b7f48000 8K rw--- [ anon ]
> > b7f4a000 8K r-x-- /usr/lib/ruby/1.8/i486-linux/etc.so
> > b7f4c000 4K rw--- /usr/lib/ruby/1.8/i486-linux/etc.so
> > b7f4d000 12K r-x-- /usr/lib/ruby/1.8/i486-linux/thread.so
> > b7f50000 4K rw--- /usr/lib/ruby/1.8/i486-linux/thread.so
> > b7f51000 8K rw--- [ anon ]
> > b7f53000 104K r-x-- /lib/ld-2.7.so
> > b7f6d000 8K rw--- /lib/ld-2.7.so
> > bfdea000 116K rw--- [ stack ]
> > ffffe000 4K r-x-- [ anon ]
> > total 44568K
> >
> > The section "0804a000 33412K rw--- [ anon ]" increased after each
> > call. I've no idea now. Do you have any suggestion for me?
>
>
> This is what I have read here or somewhere else, and I an not
> knowledgeable enough to confirm it, anyway:
> Ruby allocs space from the OS in large chunks (initially 8 MB, and
> each other chunk is bigger by a predefined coefficient).
> Any chunk can be returned to the OS only if it is empty - so if your
> memory is fragmented, ruby will have to alloc new chunks
> in order to satisfy ruby script being run.
>
> If this is true, it means: the order and size of your allocations is important.
> Especially order of the allocations that you later release relative to
> those that you don't.
>
>