Asp Forum - Memory leak in ruby code

Pratik

3/27/2007 6:34:00 PM

Hi,

I'm trying to write a code that'd inspect my method calls for any
memory leaks. You can find my code at http://pastie.cabo...

In my method, I have

def leaker
"hello"
end

Here, as per my code, the string "hello" is caught as a leak. Could
someone tell me if I'm doing it the right way ?

Thanks,
Pratik

12 Answers

Rick DeNatale

3/27/2007 7:17:00 PM

On 3/27/07, Jason Roelofs <jameskilton@gmail.com> wrote:
> So my first question is this: why are you doing this? Ruby's garbage
> collector is perfectly capable of finding and preventing REAL memory leaks.
>
> What you have here is not a memory leak detector, but an "objects have been
> created" detector. "hello" is then obviously caught as a new object created,
> and as Ruby returns the result of the last statement of a method, it is not
> yet out of scope until the program quits, in your case.

There's also the problem that he's trying to detect surviving objects
by comparing object_ids before and after GC, there's no guarantee that
an object_id of a gc'd object won't be reused. Object_id's only need
to be unique among LIVE objects.

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denh...

Pratik

3/28/2007 10:47:00 AM

Hi,

Thanks for your replies. I've modified the code a little so that my
string "hello" is not the last statement. And also, it is used in
"puts". Even when it's out of scope, it's not being free.

http://pastie.cabo...

Jason : Why I'm doin this, frankly speaking, the concept of memory
leaks in ruby code is still not very clear to me. I came across
http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-...
- which checks for common objects within 10 seconds interval. So I
wrote a lil piece of code myself to see if any objects created in a
method call, are getting removed or not when it's out of scope.

I'd like to see if someone can point me to an example of memory leak
in ruby code. Or lemme know how to detect proper memory leaks.

Thanks!
Pratik

On 3/27/07, Rick DeNatale <rick.denatale@gmail.com> wrote:
> On 3/27/07, Jason Roelofs <jameskilton@gmail.com> wrote:
> > So my first question is this: why are you doing this? Ruby's garbage
> > collector is perfectly capable of finding and preventing REAL memory leaks.
> >
> > What you have here is not a memory leak detector, but an "objects have been
> > created" detector. "hello" is then obviously caught as a new object created,
> > and as Ruby returns the result of the last statement of a method, it is not
> > yet out of scope until the program quits, in your case.
>
> There's also the problem that he's trying to detect surviving objects
> by comparing object_ids before and after GC, there's no guarantee that
> an object_id of a gc'd object won't be reused. Object_id's only need
> to be unique among LIVE objects.
>
> --
> Rick DeNatale
>
> My blog on Ruby
> http://talklikeaduck.denh...
>
>

--
rm -rf / 2>/dev/null - http...

Dont judge those who try and fail, judge those who fail to try..

Brian Candler

3/28/2007 7:08:00 PM

On Wed, Mar 28, 2007 at 07:46:43PM +0900, Pratik wrote:
> Thanks for your replies. I've modified the code a little so that my
> string "hello" is not the last statement. And also, it is used in
> "puts". Even when it's out of scope, it's not being free.

Objects are not freed when they go out of scope. Objects are freed when the
garbage collector runs, and notices that no outside references point to
these objects. (This is "mark and sweep garbage collection")

You can force this to happen using GC.start, if you wish to play around with
the internals.

Doing it other ways (e.g. reference counting) is expensive and error-prone.
I have a vague recollection that Python does it this way though.

Pratik

3/28/2007 9:41:00 PM

I've already called GC.start and yet some objects are not being freed.
However, I'm still uncertain about memory leaks in ruby code and
looking for an example of any such case.

Thanks,
Pratik

On 3/28/07, Brian Candler <B.Candler@pobox.com> wrote:
> On Wed, Mar 28, 2007 at 07:46:43PM +0900, Pratik wrote:
> > Thanks for your replies. I've modified the code a little so that my
> > string "hello" is not the last statement. And also, it is used in
> > "puts". Even when it's out of scope, it's not being free.
>
> Objects are not freed when they go out of scope. Objects are freed when the
> garbage collector runs, and notices that no outside references point to
> these objects. (This is "mark and sweep garbage collection")
>
> You can force this to happen using GC.start, if you wish to play around with
> the internals.
>
> Doing it other ways (e.g. reference counting) is expensive and error-prone.
> I have a vague recollection that Python does it this way though.
>
>

--
rm -rf / 2>/dev/null - http...

Dont judge those who try and fail, judge those who fail to try..

Robert Klemme

3/30/2007 7:07:00 AM

On 28.03.2007 23:41, Pratik wrote:
> I've already called GC.start and yet some objects are not being freed.
> However, I'm still uncertain about memory leaks in ruby code and
> looking for an example of any such case.

A memory leak comes into existence when some piece of code holds longer
onto an instance than necessary / intended. Look at this:

class Foo
@instances = []

def self.register(obj)
@instances << obj
end

def initialize
Foo.register(self)
end
end

Instances register with their class but never deregister - so they can
never be collected and use up memory. My guess would be that you'll
have a hard time writing a program that is able to track these without
actually running the code.

Kind regards

robert

Brian Candler

3/31/2007 8:25:00 AM

On Fri, Mar 30, 2007 at 04:10:04PM +0900, Robert Klemme wrote:
> On 28.03.2007 23:41, Pratik wrote:
> >I've already called GC.start and yet some objects are not being freed.
> >However, I'm still uncertain about memory leaks in ruby code and
> >looking for an example of any such case.
>
> A memory leak comes into existence when some piece of code holds longer
> onto an instance than necessary / intended. Look at this:
>
> class Foo
> @instances = []
>
> def self.register(obj)
> @instances << obj
> end
>
> def initialize
> Foo.register(self)
> end
> end
>
> Instances register with their class but never deregister - so they can
> never be collected and use up memory. My guess would be that you'll
> have a hard time writing a program that is able to track these without
> actually running the code.

IMO, that's not a memory leak, that's correct behaviour. You created some
objects and purposely held on to references to them. At any point later in
the code someone might do Foo.class_eval { @instances.[n] }

My definition of a memory leak would be when memory has been allocated but
there are *no* remaining references to it available anywhere, meaning that
it has been effectively lost and can never be freed.

This shouldn't/can't happen in pure Ruby code. But it can certainly happen
in C extensions, where memory has been malloc()d but the pointer forgotten
about.

Regards,

Brian.

Dejan Dimic

3/31/2007 9:54:00 AM

The memory leak is a situation where programmer undeliberately keep
some objects in memory but not aware of, usually by poor
implementation of object destructors.
The repercussion of such is that some memory is never released and can
be seen as memory consumption build up with no reasonable explanation.

The above example is not such.

The languages with garbage collectors are less exposed to memory leaks
in a general sense.
If you keep a reference to some object is something you do on
purpose.

To be truly onset the garbage collector is also o piece of code
written by humans ;-).

Stephen Kellett

3/31/2007 9:40:00 PM

In message <1175334819.468800.137310@p77g2000hsh.googlegroups.com>, dima
<dejan.dimic@gmail.com> writes
>If you keep a reference to some object is something you do on
>purpose.

Unless there is an error in your logic. Perhaps you forgot to clear the
reference(s) to the object when you were finished with it? Perhaps the
design changed and you forgot to clear the reference, or forgot to call
a method that cleared the reference.

No different to a memory leak in a non-garbage collected language.
Engineers don't put errors in programs deliberately (well, if you've got
one, you need to sack them), they happen by mistake, design error,
tiredness, Friday afternoons, etc. Thats life, stuff happens.

That is why since Java came on the scene companies like Sitraka (with
JProbe in 1996) and many others since have produced tools to detect such
situations. Its also why we created Ruby Memory Validator, because such
problems do exist for some applications.

http://www.softwareverify.com/ruby/memory/...

The main problem with leaks in garbage collected programs is that by
their very nature (an unwanted reference is held) they are harder to
detect than for non-garbage collected programs.

The statement you write above makes me think you have not truly
experienced a memory leak situation in a GC'd application (regardless of
language - the principles remain the same).

I mention Sitraka as they were the first company I am aware of that
released a commercial tool for this task. They also published an article
in Dr Dobbs detailing their position on this area and the terms they
used. They called an unwanted object a "loiterer" if I recall correctly.
There were three types:

o Unwanted object references held longer than necessary by accident (the
typical memory leak scenario). This reference could be stored in a
static data member (class member) or in a dynamic data member (instance
member)

o Temporary object on stack held for longer than necessary, preventing
potentially large object trees from being collected

someFunc()
{
obj = createALargeNumerOfObjectsAndGiveMeTheRoot();

doSomeWork(obj);

doSomeOtherWorkThatCouldDoWithMemoryReclaim();
}

obj could be reclaimed after doSomeWork() (assuming the stack ref is the
only reference to it) but because it is on the stack it can't be. A
better way to do this would be to create a local scope for it (yes,
there are other variantions, this is just to demo the point)

someFunc()
{
{
obj = createALargeNumerOfObjectsAndGiveMeTheRoot();

doSomeWork(obj);
}

doSomeOtherWorkThatCouldDoWithMemoryReclaim();
}

o Some other case I can't recall right now (sorry I have a horrible cold
and don't feel so good, so memory not working so good...). I'm pretty
sure there was a third case, but it eludes me. Sorry.

It quite interesting watching the Ruby community have the same
discussions (and same misconceptions) about memory as happened in the
Java community over a decade ago.

I've found a reference to Sitraka's work here: Its a powerpoint
presentation. For all of you that think memory leaks can't happen in a
GC'd environment please take a look. Its a not a leak in the terms of a
C memory leak, but you are still chewing through memory.

These are different presentations, found using a simple Google search:

http://www.java-forum-stuttgart.de/jfs/2001/fol...
http://java.quest.com/JUG/meetings/presentations/sep02/JUG%20S....
PPT

I hope you find them interesting. They apply to Ruby just as much as
they do to Java.

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/sof...
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
Reg Office: 24 Windmill Walk, Sutton, Ely, Cambs CB6 2NH.

Robert Klemme

4/1/2007 4:13:00 PM

On 31.03.2007 10:24, Brian Candler wrote:
> On Fri, Mar 30, 2007 at 04:10:04PM +0900, Robert Klemme wrote:
>> On 28.03.2007 23:41, Pratik wrote:
>>> I've already called GC.start and yet some objects are not being freed.
>>> However, I'm still uncertain about memory leaks in ruby code and
>>> looking for an example of any such case.
>> A memory leak comes into existence when some piece of code holds longer
>> onto an instance than necessary / intended. Look at this:
>>
>> class Foo
>> @instances = []
>>
>> def self.register(obj)
>> @instances << obj
>> end
>>
>> def initialize
>> Foo.register(self)
>> end
>> end
>>
>> Instances register with their class but never deregister - so they can
>> never be collected and use up memory. My guess would be that you'll
>> have a hard time writing a program that is able to track these without
>> actually running the code.
>
> IMO, that's not a memory leak, that's correct behaviour.

From the interpreter's point of view.

> You created some
> objects and purposely held on to references to them. At any point later in
> the code someone might do Foo.class_eval { @instances.[n] }

While this might actually happen, the issue I tried to convey is that no
instances of class Foo will never be collected because @instances holds
on to all of them. If this is intended then this is of course not a
leak but as my explanation should have made clear this was not intended
in the example. I probably should have been clearer about this.

> My definition of a memory leak would be when memory has been allocated but
> there are *no* remaining references to it available anywhere, meaning that
> it has been effectively lost and can never be freed.

This cannot happen in Ruby and Java. In these languages the term
"memory leak" is actually used for memory that cannot be freed because
something holds on to it longer than intended.

> This shouldn't/can't happen in pure Ruby code. But it can certainly happen
> in C extensions, where memory has been malloc()d but the pointer forgotten
> about.

That's another story. But I think the OP was looking for memory leaks
in Ruby code.

Kind regards

robert

Robert Klemme

4/3/2007 8:21:00 PM

On 31.03.2007 23:39, Stephen Kellett wrote:
> I mention Sitraka as they were the first company I am aware of that
> released a commercial tool for this task. They also published an article
> in Dr Dobbs detailing their position on this area and the terms they
> used. They called an unwanted object a "loiterer" if I recall correctly.
> There were three types:

I believe OptimizeIt is also around for pretty long time. I knew it
before JProbe but that doesn't mean anything.

Kind regards

robert

comp.lang.ruby

Memory leak in ruby code

Pratik

Rick DeNatale

Pratik

Brian Candler

Pratik

Robert Klemme

Brian Candler

Dejan Dimic

Stephen Kellett

Robert Klemme

Robert Klemme

x Login to ForumsZone