[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

GC.disable not working?

Eric Mahurin

6/18/2005 9:33:00 PM

From what I can tell, GC.disable doesn't work. I'm wanting to
use GC.disable/GC.enable around code using objects that I get
from ObjectSpace._id2ref to make sure that the id doesn't
become invalid because of GC. Here is a piece of code that
demonstrates the problem:


#!/bin/env ruby

def finalizer(id)
print("<")
GC.enable and
printf("GC disabled while finalizing %x!!!",id)
@ids.delete(id)
print(">")
end

$stdout.sync=true
@ids = {}
1000.times { |i|
print("[")
obj = "X"*i
id = obj.object_id
@ids[id] = true
ObjectSpace.define_finalizer(obj,method(:finalizer))
GC.disable
print("{")
@ids.each_key { |id|
# go make sure all objects seem valid (test size)
ObjectSpace._id2ref(id).size
}
print("}")
GC.enable
print("]")
}


I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
the finalizer when GC is disabled (GC.enable==true). I thought
this would be impossible. I also get a recycled object error.
In the 1.9 version I'm using, it hangs while trying to finalize
one of the objects (@ids.delete(id) is where it hangs).

Any clues to what is going on with this GC stuff. If I can't
get this to work, I don't see how anybody reliably use
ObjectSpace._id2ref. BTW, I found this problem in my code
after running tens of thousands of random tests. If you don't
do enough testing, you may not find problems related to garbage
collection.





__________________________________
Yahoo! Mail Mobile
Take Yahoo! Mail with you! Check email on your mobile phone.
http://mobile.yahoo.com/...


28 Answers

Robert Klemme

6/18/2005 11:22:00 PM

0

Eric Mahurin <eric_mahurin@yahoo.com> wrote:
> From what I can tell, GC.disable doesn't work. I'm wanting to
> use GC.disable/GC.enable around code using objects that I get
> from ObjectSpace._id2ref to make sure that the id doesn't
> become invalid because of GC. Here is a piece of code that
> demonstrates the problem:
>
>
> #!/bin/env ruby
>
> def finalizer(id)
> print("<")
> GC.enable and
> printf("GC disabled while finalizing %x!!!",id)
> @ids.delete(id)
> print(">")
> end
>
> $stdout.sync=true
> @ids = {}
> 1000.times { |i|
> print("[")
> obj = "X"*i
> id = obj.object_id
> @ids[id] = true
> ObjectSpace.define_finalizer(obj,method(:finalizer))
> GC.disable
> print("{")
> @ids.each_key { |id|
> # go make sure all objects seem valid (test size)
> ObjectSpace._id2ref(id).size
> }
> print("}")
> GC.enable
> print("]")
> }
>
>
> I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
> the finalizer when GC is disabled (GC.enable==true).

Doesn't happen with the 1.8.2 I have.

> I thought
> this would be impossible.

AFAIK there is just one guarantee: the finalizer is called *after* the
object is GC'ed and before the interpreter exits. I don't know whether
there are any guarantees that the finalizer is called *immediately* after
the object was collected. So in theory the object could be collected while
GC is enabled and the finalizer can be called some time later when GC is
disabled. Although you might expect finalization to happen immediately
after collection it would not be wise to require this of a runtime system as
it might limit implementation choices in a way that negatively affect
performance.

> I also get a recycled object error.

One reason for this could be that you iterate and manipulate the same
collection (the Hash of oids) at the same time. Also, as I pointed out
above - there is a time lag between collection and finalization. During
this time the id is still in the hash but the object is gone already.

> In the 1.9 version I'm using, it hangs while trying to finalize
> one of the objects (@ids.delete(id) is where it hangs).

You can avoid that by catching the exception:


$ids = {}
$stdout.sync=true

fin = lambda {|oid| $ids.delete oid}

1000.times do
obj = Object.new
$ids[obj.object_id] = true
ObjectSpace.define_finalizer(obj, &fin)
obj = nil

puts "disable: #{GC.disable}"

$ids.each_key do |oid|
begin
raise "mismatch" unless ObjectSpace._id2ref( oid ).object_id == oid
rescue Exception => e
p [oid, e]
end
end

puts "enable: #{GC.enable}"
end



> Any clues to what is going on with this GC stuff. If I can't
> get this to work, I don't see how anybody reliably use
> ObjectSpace._id2ref. BTW, I found this problem in my code
> after running tens of thousands of random tests. If you don't
> do enough testing, you may not find problems related to garbage
> collection.

Kind regards

robert

nobu.nokada

6/19/2005 4:10:00 AM

0

Hi,

At Sun, 19 Jun 2005 06:32:44 +0900,
Eric Mahurin wrote in [ruby-talk:145790]:
> $stdout.sync=true
> @ids = {}
> 1000.times { |i|
> print("[")
> obj = "X"*i
> id = obj.object_id
> @ids[id] = true
> ObjectSpace.define_finalizer(obj,method(:finalizer))
> GC.disable
> print("{")
> @ids.each_key { |id|
> # go make sure all objects seem valid (test size)
> ObjectSpace._id2ref(id).size
> }
> print("}")
> GC.enable

You enable GC here, so it can run by next GC.disable. Then an
exception occurs after GC is disabled, therefore GC will be no
longer enabled.

> print("]")
> }
>
>
> I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
> the finalizer when GC is disabled (GC.enable==true). I thought
> this would be impossible. I also get a recycled object error.

Finailizers don't run immediately after the corresponding
objects get collected. The `finalizer' runs at the process
termination.

> In the 1.9 version I'm using, it hangs while trying to finalize
> one of the objects (@ids.delete(id) is where it hangs).

It's a bug, I'll investigate it.

--
Nobu Nakada


Eric Mahurin

6/19/2005 3:31:00 PM

0

--- nobu.nokada@softhome.net wrote:
> Finailizers don't run immediately after the corresponding
> objects get collected. The `finalizer' runs at the process
> termination.

This is what the documentation says about define_finalizer:

---
Adds aProc as a finalizer, to be called when obj is about to be
destroyed.
---

So, it should run the finalizer right before the object gets
collected. Running the finalizer right after would also work
fine for me. Running it an arbitrary time later doesn't seem
very useful if you plan on using _id2ref. It definitely
doesn't just run finalizers at process termination.

I also tried using Thread.critical= to control the finalizer,
but it doesn't seem to make a difference:

---

#!/bin/env ruby

def finalizer(id)
print("<")
critical = Thread.critical
begin
Thread.critical = true
print(critical ? "F" : "f")
GC.enable and
printf("!GC disabled while finalizing %x!",id)
@ids.delete(id)
ensure
Thread.critical = critical
end
print(">")
end

$stdout.sync=true
@ids = {}
1000.times { |i|
print("[")
obj = "X"*i
id = obj.object_id
@ids[id] = true
ObjectSpace.define_finalizer(obj,method(:finalizer))
GC.start
GC.disable
critical = Thread.critical
begin
Thread.critical = true
print("{")
@ids.each_key { |id|
begin
#finalizer will still run with this instead
#obj2 = "xyzabc"*i
ObjectSpace._id2ref(id).size
rescue RangeError
print(Thread.critical ? "E" : "e")
end
}
print("}")
ensure
Thread.critical = critical
end
GC.enable
print("]")
}

---

I catch the RangeError (recycled object) exceptions and
Thread.critical is still true ("E" gets printed), but the
finalizer still happily runs. I guess GC and the finalizer are
still considered to be part of the same thread even though
functionally it seems like a different one.

I realize that catching RangeError's would fix 99% of the
problems. But, I would still be concerned about the case where
the object would be GCed and then the space reclaimed by an
object that looks just like it. Is it guaranteed that
finalizers of the orginal object be run before its space is
reclaimed?

My goal is for a certain set of objects to maintain which of
those objects are alive. If there are no other references to
them, I want them to be GCed. For the objects still alive I
need to be able to operate on them (see if any are "open" and
also be able to "close" them all). Does anybody have a better
implementation than above? I'm not sure if the above really
works because of the "reclaimed space" case discussed above.
Using WeakRef's may be a solution too. I'm hesitant to trust
much of this stuff until I understand:

a. When are object finalizers called? The docs don't reflect
the behavior.

b. What does GC.disable do?

c. Is GC and/or calling object finalizers considered another
thread? It sure seems like they should.





__________________________________
Yahoo! Mail
Stay connected, organized, and protected. Take the tour:
http://tour.mail.yahoo.com/mai...



ts

6/19/2005 4:12:00 PM

0

>>>>> "E" == Eric Mahurin <eric_mahurin@yahoo.com> writes:


E> I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
E> the finalizer when GC is disabled (GC.enable==true). I thought
E> this would be impossible. I also get a recycled object error.

it's normal : the finalizer is called when ruby stop and it don't care at
this moment if the GC is enabled or not.

E> Any clues to what is going on with this GC stuff. If I can't
E> get this to work, I don't see how anybody reliably use
E> ObjectSpace._id2ref.

it's best to don't use _id2ref


Guy Decoux




nobu.nokada

6/19/2005 5:07:00 PM

0

Hi,

At Mon, 20 Jun 2005 00:30:59 +0900,
Eric Mahurin wrote in [ruby-talk:145818]:
> > Finailizers don't run immediately after the corresponding
> > objects get collected. The `finalizer' runs at the process
> > termination.
>
> This is what the documentation says about define_finalizer:
>
> ---
> Adds aProc as a finalizer, to be called when obj is about to be
> destroyed.
> ---

The documentation is inaccurate or improper. Finalizers have
never be called before the destruction.

> I catch the RangeError (recycled object) exceptions and
> Thread.critical is still true ("E" gets printed), but the
> finalizer still happily runs. I guess GC and the finalizer are
> still considered to be part of the same thread even though
> functionally it seems like a different one.

That guess is right. But I didn't see "E" nor "e" from your
new example, because GC.start does run finalizers. Commenting
the line out, "E" was printed.

> I realize that catching RangeError's would fix 99% of the
> problems. But, I would still be concerned about the case where
> the object would be GCed and then the space reclaimed by an
> object that looks just like it. Is it guaranteed that
> finalizers of the orginal object be run before its space is
> reclaimed?

They are called after all destruction has done.

> a. When are object finalizers called? The docs don't reflect
> the behavior.

Within evaluation loop after a method implemented in C ended.
It is possible to change it more frequently (e.g., for each
instructions), but I've not measured how it affects the
performance.

> b. What does GC.disable do?

Prohibits running GC. If free slots are exhausted while GC is
disabled, the interpreter just tries to allocate new slots with
malloc().

> c. Is GC and/or calling object finalizers considered another
> thread? It sure seems like they should.

Well, though the current implementation doesn't, I guess so.

--
Nobu Nakada


Eric Mahurin

6/19/2005 8:53:00 PM

0

--- nobu.nokada@softhome.net wrote:
> At Mon, 20 Jun 2005 00:30:59 +0900,
> Eric Mahurin wrote in [ruby-talk:145818]:
> > > Finailizers don't run immediately after the corresponding
> > > objects get collected. The `finalizer' runs at the
> process
> > > termination.
> >
> > This is what the documentation says about define_finalizer:
> >
> > ---
> > Adds aProc as a finalizer, to be called when obj is about
> to be
> > destroyed.
> > ---
>
> The documentation is inaccurate or improper. Finalizers have
> never be called before the destruction.
>
> > I catch the RangeError (recycled object) exceptions and
> > Thread.critical is still true ("E" gets printed), but the
> > finalizer still happily runs. I guess GC and the finalizer
> are
> > still considered to be part of the same thread even though
> > functionally it seems like a different one.
>
> That guess is right. But I didn't see "E" nor "e" from your
> new example, because GC.start does run finalizers.
> Commenting
> the line out, "E" was printed.

Sorry. I meant to comment out the GC.start. Putting in the
GC.start (sometimes I need 2 of them back-to-back) gets it to
work, but I don't want the penalty of running GC all the time.

> > I realize that catching RangeError's would fix 99% of the
> > problems. But, I would still be concerned about the case
> where
> > the object would be GCed and then the space reclaimed by an
> > object that looks just like it. Is it guaranteed that
> > finalizers of the orginal object be run before its space is
> > reclaimed?
>
> They are called after all destruction has done.

And before any of that space is reclaimed? It seems like
normal Ruby processing (including memory allocation) can occur
during GC and between GC and finalizers.

> > a. When are object finalizers called? The docs don't
> reflect
> > the behavior.
>
> Within evaluation loop after a method implemented in C ended.
> It is possible to change it more frequently (e.g., for each
> instructions), but I've not measured how it affects the
> performance.

For future Ruby revisions, I think you should consider ensuring
that after an object is GCed its finalizers are called before
normal ruby processing continues. Having normal ruby code have
to deal with the situation where an object has been GCed but
its finalizers haven't been called seems unnecessary.

> > b. What does GC.disable do?
>
> Prohibits running GC. If free slots are exhausted while GC
> is
> disabled, the interpreter just tries to allocate new slots
> with
> malloc().

So, it works immediately, but this GC.disable could have
occured between an object being GCed and its finalizers being
called. The finalizers could still continue while GC is
disabled.




__________________________________
Discover Yahoo!
Stay in touch with email, IM, photo sharing and more. Check it out!
http://discover.yahoo.com/stayin...


Robert Klemme

6/19/2005 9:01:00 PM

0


"ts" <decoux@moulon.inra.fr> schrieb im Newsbeitrag
news:200506191610.j5JGAeIU001389@moulon.inra.fr...
>>>>>> "E" == Eric Mahurin <eric_mahurin@yahoo.com> writes:
>
>
> E> I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
> E> the finalizer when GC is disabled (GC.enable==true). I thought
> E> this would be impossible. I also get a recycled object error.
>
> it's normal : the finalizer is called when ruby stop and it don't care at
> this moment if the GC is enabled or not.

They can be invoked before termination of Ruby. But it's guaranteed that
all finalizers are called before exit.

> E> Any clues to what is going on with this GC stuff. If I can't
> E> get this to work, I don't see how anybody reliably use
> E> ObjectSpace._id2ref.
>
> it's best to don't use _id2ref

I beg to differ: you can safely use _id2ref in conjunction with finalizers -
it just has to be done properly. It will go wrong - as demonstrated - if
the same collection that stores ids is used for iterating / querying.

This approach works ok - you'll have to imagine that x contains information
needed for proper cleanup of a Foo instance, for example, an open IO
instance (although I'm sure that will do proper cleanup on finalization):

class Foo
FOOS = {}

def initialize(x)
self.x=x

ObjectSpace.define_finalizer(self) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end

def x=(y)
@x = y
FOOS[object_id] = y
end

def x() @x end
end

Kind regards

robert

Robert Klemme

6/19/2005 9:25:00 PM

0


"Robert Klemme" <bob.news@gmx.net> schrieb im Newsbeitrag
news:3hm4olFhoppnU1@individual.net...
>
> "ts" <decoux@moulon.inra.fr> schrieb im Newsbeitrag
> news:200506191610.j5JGAeIU001389@moulon.inra.fr...
>>>>>>> "E" == Eric Mahurin <eric_mahurin@yahoo.com> writes:
>>
>>
>> E> I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
>> E> the finalizer when GC is disabled (GC.enable==true). I thought
>> E> this would be impossible. I also get a recycled object error.
>>
>> it's normal : the finalizer is called when ruby stop and it don't care at
>> this moment if the GC is enabled or not.
>
> They can be invoked before termination of Ruby. But it's guaranteed that
> all finalizers are called before exit.
>
>> E> Any clues to what is going on with this GC stuff. If I can't
>> E> get this to work, I don't see how anybody reliably use
>> E> ObjectSpace._id2ref.
>>
>> it's best to don't use _id2ref
>
> I beg to differ: you can safely use _id2ref in conjunction with
> finalizers - it just has to be done properly. It will go wrong - as
> demonstrated - if the same collection that stores ids is used for
> iterating / querying.

To avoid any confusion let me add some clarification: _id2ref has its uses,
but some caution has to be applied:

- don't use it in finalizers

- if objects might have been collected, take precautions (i.e. catch
exceptions)

Also, WeakReferences can be used in many places if you want to be able to
refer an instance but don't want to prevent garbage collection (storing an
oid often serves the same purpose).

The example of course does not show the usage of _id2ref. You can add a
method like this to class Foo:

def self.list
FOO.each do |oid, x|
begin
p ObjectSpace._id2ref(oid)
rescue RangeError => e
puts "#{oid} collected but not finalized: x=#{x.inspect}"
end
end
end


> This approach works ok - you'll have to imagine that x contains
> information needed for proper cleanup of a Foo instance, for example, an
> open IO instance (although I'm sure that will do proper cleanup on
> finalization):
>
> class Foo
> FOOS = {}
>
> def initialize(x)
> self.x=x
>
> ObjectSpace.define_finalizer(self) do |oid|
> puts "Cleanup of #{oid} with #{FOOS[oid]}"
> end
> end
>
> def x=(y)
> @x = y
> FOOS[object_id] = y
> end
>
> def x() @x end
> end
>
> Kind regards
>
> robert
>

Eric Mahurin

6/19/2005 10:55:00 PM

0

--- Robert Klemme <bob.news@gmx.net> wrote:
> "Robert Klemme" <bob.news@gmx.net> schrieb im Newsbeitrag
> def self.list
> FOOS.each do |oid, x|
> begin
> p ObjectSpace._id2ref(oid)
> rescue RangeError => e
> puts "#{oid} collected but not finalized:
> x=#{x.inspect}"
> end
> end
> end

I assume you meant FOOS above. I fixed it.

Just catching a RangeError is not all you need. You better
make sure the object is finalized and the finalizer removes its
oid from FOOS before the space is reclaimed. Otherwise oid
could refer to a completely new object and you wouldn't detect
it.

On top of that, these finalizers can't be called while this
FOOS.each loop is going on. Otherwise you'd get an error about
the hash being modified while your iterating over it. It is
like the GC and finalizers are in another thread, but
unfortunately you can't control it like a thread (i.e.
Thread.critical=). This is my primary dilemma.

These issues may be very difficult to detect problems with and
you may need 1000's of tests to excite the GC differently to
detect the problem. You must be prepared for this using
_id2ref.

> > This approach works ok - you'll have to imagine that x
> contains
> > information needed for proper cleanup of a Foo instance,
> for example, an
> > open IO instance (although I'm sure that will do proper
> cleanup on
> > finalization):
> >
> > class Foo
> > FOOS = {}
> >
> > def initialize(x)
> > self.x=x
> >
> > ObjectSpace.define_finalizer(self) do |oid|
> > puts "Cleanup of #{oid} with #{FOOS[oid]}"
> > end
> > end

Won't work. This is the exact mistake I made when first trying
to use a finalizer. In the above, you gave define_finalizer a
Proc that has a Binding with direct access to self (a Foo).
This creates an unintended reference to the object in
ObjectSpace/GC. It will never be GCed because of this. It
took me a while to figure this out. The Proc/Method needs to
be defined in a context that doesn't have access to the object
you are trying to put a finalizer on. For this reason, I don't
think define_finalizer should even allow the block form. Also,
a Proc#unbind would be nice to have in this situation.

Also, I assume you'd want to delete the oid entry from FOOS in
this finalizer, right?

> > def x=(y)
> > @x = y
> > FOOS[object_id] = y
> > end
> >
> > def x() @x end
> > end




____________________________________________________
Yahoo! Sports
Rekindle the Rivalries. Sign up for Fantasy Football
http://football.fantasysports...


Robert Klemme

6/20/2005 7:37:00 AM

0

Eric Mahurin wrote:
> --- Robert Klemme <bob.news@gmx.net> wrote:
>> "Robert Klemme" <bob.news@gmx.net> schrieb im Newsbeitrag
>> def self.list
>> FOOS.each do |oid, x|
>> begin
>> p ObjectSpace._id2ref(oid)
>> rescue RangeError => e
>> puts "#{oid} collected but not finalized:
>> x=#{x.inspect}"
>> end
>> end
>> end
>
> I assume you meant FOOS above. I fixed it.

Correct.

> Just catching a RangeError is not all you need. You better
> make sure the object is finalized and the finalizer removes its
> oid from FOOS before the space is reclaimed. Otherwise oid
> could refer to a completely new object and you wouldn't detect
> it.

Probably. My assumption was, that oids are not reused. But I may be
wrong here.

> On top of that, these finalizers can't be called while this
> FOOS.each loop is going on. Otherwise you'd get an error about
> the hash being modified while your iterating over it. It is
> like the GC and finalizers are in another thread, but
> unfortunately you can't control it like a thread (i.e.
> Thread.critical=). This is my primary dilemma.

Hm... Did you try Mutex or Monitor?

> These issues may be very difficult to detect problems with and
> you may need 1000's of tests to excite the GC differently to
> detect the problem. You must be prepared for this using
> _id2ref.
>
>>> This approach works ok - you'll have to imagine that x contains
>>> information needed for proper cleanup of a Foo instance, for
>>> example, an open IO instance (although I'm sure that will do proper
>>> cleanup on finalization):
>>>
>>> class Foo
>>> FOOS = {}
>>>
>>> def initialize(x)
>>> self.x=x
>>>
>>> ObjectSpace.define_finalizer(self) do |oid|
>>> puts "Cleanup of #{oid} with #{FOOS[oid]}"
>>> end
>>> end
>
> Won't work. This is the exact mistake I made when first trying
> to use a finalizer. In the above, you gave define_finalizer a
> Proc that has a Binding with direct access to self (a Foo).
> This creates an unintended reference to the object in
> ObjectSpace/GC. It will never be GCed because of this.

Darn, yes you're right!

Two alternatives would be

class Foo
class<<self
alias :_new :new
def new(*a,&b)
obj = _new(*a,&b)
ObjectSpace.define_finalizer(obj) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
obj
end
end
end

class Foo
def initialize(x)
self.x=x

self.class.instance_eval do
ObjectSpace.define_finalizer(obj) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end
end
end


> It
> took me a while to figure this out. The Proc/Method needs to
> be defined in a context that doesn't have access to the object
> you are trying to put a finalizer on. For this reason, I don't
> think define_finalizer should even allow the block form. Also,
> a Proc#unbind would be nice to have in this situation.
>
> Also, I assume you'd want to delete the oid entry from FOOS in
> this finalizer, right?

Yes. Sorry for the errors and omissions - it was late already...

>
>>> def x=(y)
>>> @x = y
>>> FOOS[object_id] = y
>>> end
>>>
>>> def x() @x end
>>> end

What is the real world problem you are trying to solve?

Kind regards

robert