[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

why does this code leak?

ara.t.howard

1/9/2008 3:23:00 AM

cfp2:~ > cat a.rb
#! /usr/bin/env ruby

require 'net/http'

3.times do
Net::HTTP.start('www.google.com') do |http|
http_response = http.get '/'
end

p ObjectSpace.each_object(Net::HTTPResponse){}
end


cfp2:~ > ruby a.rb
1
2
3


why is this leaking Net::HTTPResponse objects?


a @ http://codeforp...
--
share your knowledge. it's a way to achieve immortality.
h.h. the 14th dalai lama
38 Answers

tsuraan

1/9/2008 3:36:00 AM

0

> why is this leaking Net::HTTPResponse objects?

I don't know, but jruby has the exact same behaviour. Maybe that helps?

Jan Dvorak

1/9/2008 3:45:00 AM

0

On Wednesday 09 January 2008 04:23:23 ara howard wrote:
> why is this leaking Net::HTTPResponse objects?
It does not leak, objects are just not immediately released by the GC. Try it
running 1000 times (on local server preferably).

Jan

Michael Bevilacqua-Linn

1/9/2008 3:53:00 AM

0

[Note: parts of this message were removed to make it a legal post.]

Why are you assuming it's leaking without giving the GC a chance to run?

require 'net/http'

3.times do
Net::HTTP.start('www.google.com') do |http|
http_response = http.get '/'
end

p ObjectSpace.each_object(Net::HTTPResponse){}
end

p "Force GC"
ObjectSpace.garbage_collect
p ObjectSpace.each_object(Net::HTTPResponse){}

C:\>ruby test.rb
1
2
3
"Force GC"
0

MBL

On Jan 8, 2008 10:23 PM, ara howard <ara.t.howard@gmail.com> wrote:

> cfp2:~ > cat a.rb
> #! /usr/bin/env ruby
>
> require 'net/http'
>
> 3.times do
> Net::HTTP.start('www.google.com') do |http|
> http_response = http.get '/'
> end
>
> p ObjectSpace.each_object(Net::HTTPResponse){}
> end
>
>
> cfp2:~ > ruby a.rb
> 1
> 2
> 3
>
>
> why is this leaking Net::HTTPResponse objects?
>
>
> a @ http://codeforp...
> --
> share your knowledge. it's a way to achieve immortality.
> h.h. the 14th dalai lama
>
>
>
>

ara.t.howard

1/9/2008 4:28:00 AM

0


On Jan 8, 2008, at 8:53 PM, Michael Bevilacqua-Linn wrote:

> Why are you assuming it's leaking without giving the GC a chance to
> run?

because i'm an idiot and pasted the wrong buffer. here is the code:

#
# distilled behaviour from dike.rb
#
class Object
Methods = instance_methods.inject(Hash.new){|h, m| h.update m =>
instance_method(m)}
end

class Class
Methods = instance_methods.inject(Hash.new){|h, m| h.update m =>
instance_method(m)}

def new *a, &b
object = Methods["new"].bind(self).call *a, &b
ensure
ObjectSpace.define_finalizer object, finalizer
end

def finalizer
lambda{}
end
end


#
# the above makes this code leaks, but *only* Net::HTTPOK objects
#
require "net/http"

def leak!
Net::HTTP.start("localhost") do |http|
puts http.get('/').code
end
end

3.times {
puts "---"
p ObjectSpace.each_object(Net::HTTPResponse){}
leak!
GC.start
}


a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

ara.t.howard

1/9/2008 5:20:00 AM

0


On Jan 8, 2008, at 8:45 PM, Jan Dvorak wrote:

> It does not leak, objects are just not immediately released by the
> GC. Try it
> running 1000 times (on local server preferably).

as i already posted - i posted the wrong code. here is an even more
distilled version:

cfp2:~ > cat a.rb
#
# distilled behaviour from dike.rb
#
class Class
Finalizer = lambda {}

def leak_free_finalizer
Finalizer
end

def leaky_finalizer
lambda{}
end

def finalizer
%r/leak/ =~ ARGV.first ? leaky_finalizer : leak_free_finalizer
end

def new *a, &b
object = allocate
object.send :initialize, *a, &b
object
ensure
ObjectSpace.define_finalizer object, finalizer
end
end


#
# the above makes this code leak iff ARGV has "leak" in it
#
require "yaml"

7.times {
GC.start

y Array.name => ObjectSpace.each_object(Array){}

Array.new
}



cfp2:~ > ruby a.rb
---
Array: 21
---
Array: 49
---
Array: 58
---
Array: 58
---
Array: 58
---
Array: 58
---
Array: 58


cfp2:~ > ruby a.rb leak
---
Array: 21
---
Array: 38
---
Array: 54
---
Array: 67
---
Array: 79
---
Array: 91
---
Array: 103



so.... why does installing a static finalizer work ok, but a dynamic
one leaks memory!?

i'm nice and confused now.


a @ http://codeforp...
--
share your knowledge. it's a way to achieve immortality.
h.h. the 14th dalai lama

ara.t.howard

1/9/2008 4:13:00 PM

0


On Jan 9, 2008, at 2:41 AM, Robert Klemme wrote:

> 'd say you picked the wrong class for your tests - apparently a
> lambda uses an array somehow. This is what I see from the modified
> script (attached):

hmmm. doing this in irb suggests not:

list = []
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}

the number of Array's will remain static. hrrrmmm....

a @ http://codeforp...
--
share your knowledge. it's a way to achieve immortality.
h.h. the 14th dalai lama

Dan Yoder

1/9/2008 4:21:00 PM

0

> so.... why does installing a static finalizer work ok, but a dynamic  
> one leaks memory!?

My guess is that the lambda is keeping the scope of the new invocation
around, which includes a reference to the newly created array.

Would it be a better test not to use Array (instances of which seems
to be created also by ObjectSpace / YAML)? Ex:

class Foo ; end
7.times {
GC.start
count = ObjectSpace.each_object(Foo)
p "Count: #{count}"
Foo.new
}

-Dan

Dan Yoder

1/9/2008 4:35:00 PM

0

Here's an even simpler example. You don't need anything but the
following to demonstrate the problem.

class Foo
Finalizer = lambda{}
def initialize
ObjectSpace.define_finalizer(self,Finalizer)
end
end
def test
10.times do
GC.start
count = ObjectSpace.each_object(Foo) {}
p "Count: #{count}"
Foo.new
end
end
test # -> no leak
"Count: 0"
"Count: 1"
"Count: 2"
"Count: 2"
"Count: 2"
etc.

Now, re-open Foo and add an inline finalizer.

class Foo
def initialize
ObjectSpace.define_finalizer(self,lambda{})
end
end
test # -> now it leaks
"Count: 1"
"Count: 2"
"Count: 3"
"Count: 4"
etc.

I realize the scope of the lambda invocation is different in this
example, but since the behavior is so similar, I thought it likely
pointed to the same underlying issue.

Dan

---
dev.zeraweb.com

ara.t.howard

1/9/2008 4:46:00 PM

0


On Jan 9, 2008, at 9:39 AM, dan yoder wrote:

> I realize the scope of the lambda invocation is different in this
> example, but since the behavior is so similar, I thought it likely
> pointed to the same underlying issue.

i think it's actually some strange interaction with yaml. check this
out:

cfp2:~ > cat a.rb
class Class
def finalizer
lambda{}
end

def new *a, &b
object = allocate
object.send :initialize, *a, &b
object
ensure
ObjectSpace.define_finalizer object, finalizer
end
end

class Foo; end
class Bar < Foo; end


c = Array

if ARGV.detect{|arg| arg["leak"]}
require "yaml"
7.times {
GC.start
y c.name => ObjectSpace.each_object(c){}
c.new
}
else
7.times {
GC.start
puts "---"
puts "#{ c.name }: #{ ObjectSpace.each_object(c){} }"
c.new
}
end

cfp2:~ > ruby a.rb
---
Array: 6
---
Array: 11
---
Array: 14
---
Array: 18
---
Array: 20
---
Array: 20
---
Array: 20


cfp2:~ > ruby a.rb leak
---
Array: 21
---
Array: 38
---
Array: 54
---
Array: 67
---
Array: 79
---
Array: 91
---
Array: 103

a @ http://codeforp...
--
share your knowledge. it's a way to achieve immortality.
h.h. the 14th dalai lama

Rick DeNatale

1/9/2008 6:27:00 PM

0

On Jan 9, 2008 11:45 AM, ara howard <ara.t.howard@gmail.com> wrote:
>
> On Jan 9, 2008, at 9:39 AM, dan yoder wrote:
>
> > I realize the scope of the lambda invocation is different in this
> > example, but since the behavior is so similar, I thought it likely
> > pointed to the same underlying issue.
>
> i think it's actually some strange interaction with yaml. check this
> out:
...
> if ARGV.detect{|arg| arg["leak"]}
> require "yaml"
> 7.times {
> GC.start
> y c.name => ObjectSpace.each_object(c){}
> c.new
> }
> else
> 7.times {
> GC.start
> puts "---"
> puts "#{ c.name }: #{ ObjectSpace.each_object(c){} }"
> c.new
> }
> end
>
> cfp2:~ > ruby a.rb
> ---
> Array: 6

>
> cfp2:~ > ruby a.rb leak
> ---
> Array: 21

Not sure how you got there Ara, I don't see where the OP ever mentioned YAML.

I think the key is where the lambda is created. The lambda is
capturing the binding.

In the first case the lambda is being created in the bindig context of
the class, and in particular self is the class.

In the second case, the lambda is being created in the binding context
of the new instance, and self is that new instance, so the lambda in
the finalizer is hanging on to it.

That's my theory anyway.

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denh...