[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Newbie Mem Leak Issue

Barr, Keith

2/21/2008 9:57:00 PM

I am fairly new to Ruby and a program that I have created seems to have
a memory leak. I have generated a small program which does the basics
of my program that leaks, and the test program also seems to leak. Can
anyone spot the design flaw?

Any help you can provide is greatly appreciated.

Thanks,
Keith


=================================
class LeakTest
def initialize
print("initialized\n")
# fill up the standard 8M of space so the leak becomes apparent
faster
str = "0123456789 0123456789 0123456789 0123456789 0123456789
0123456789 0123456789 0123456789 0123456789 0123456789 "
@huge_mem = Array.new(1000000, str)
end

def get_files(pathname)
listing = Array.new

Dir.glob("#{pathname}/*",0) do |f|
# add the files
listing << f
# now add directories.
if(File.directory?(f))
returnlist = get_files(f)
returnlist.each {|filename| listing << filename }
end
end
return listing
end

include GC
def cleanup
printf("cleaning up")
GC.start
5.times do
print"."
sleep(1)
end
print"\n"
end

def monitor
listing = get_files("C:/Program Files")
printf("Found %d files\n", listing.length)
5.times do
print"."
sleep(1)
end
print"\n"
end
end

lt = LeakTest.new
while true
lt.monitor
lt.cleanup
end
==========================================
--
Posted via http://www.ruby-....
13 Answers

Joel VanderWerf

2/21/2008 10:05:00 PM

0

Keith Barr wrote:
> str = "0123456789 0123456789 0123456789 0123456789 0123456789
> 0123456789 0123456789 0123456789 0123456789 0123456789 "
> @huge_mem = Array.new(1000000, str)

That's an array of 1000000 references to the same string. Did you want
lots of strings? This is the way to do that:

Array.new(1000000) {str}

------------------------------------------------------------- Array::new
Array.new(size=0, obj=nil)
Array.new(array)
Array.new(size) {|index| block }
------------------------------------------------------------------------
Returns a new array. In the first form, the new array is empty. In
the second it is created with _size_ copies of _obj_ (that is,
_size_ references to the same _obj_).

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Barr, Keith

2/21/2008 10:17:00 PM

0

Joel VanderWerf wrote:
> Keith Barr wrote:
>> str = "0123456789 0123456789 0123456789 0123456789 0123456789
>> 0123456789 0123456789 0123456789 0123456789 0123456789 "
>> @huge_mem = Array.new(1000000, str)
>
> That's an array of 1000000 references to the same string. Did you want
> lots of strings? This is the way to do that:
>
> Array.new(1000000) {str}
>
> ------------------------------------------------------------- Array::new
> Array.new(size=0, obj=nil)
> Array.new(array)
> Array.new(size) {|index| block }

oops, yeah, good catch.
--
Posted via http://www.ruby-....

John Woods

2/21/2008 10:37:00 PM

0

>> Array.new(1000000) {str}

Isn't the above still just an array of 1000000 refs to the same string?
Perhaps you meant str.dup?

For example:

irb> str = "asdf"
irb> puts Array.new(3){str}.collect{|s|s.object_id}
-605717778
-605717778
-605717778
# the above three object_ids are the same
irb> puts Array.new(3){str.dup}.collect{|s|s.object_id}
-605741458
-605741468
-605741478
# the above three object_ids are different





-----Original Message-----
From: Keith Barr
Sent: 02/21/2008 02:16 PM
> Joel VanderWerf wrote:
>> Keith Barr wrote:
>>> str = "0123456789 0123456789 0123456789 0123456789 0123456789
>>> 0123456789 0123456789 0123456789 0123456789 0123456789 "
>>> @huge_mem = Array.new(1000000, str)
>> That's an array of 1000000 references to the same string. Did you want
>> lots of strings? This is the way to do that:
>>
>> Array.new(1000000) {str}
>>
>> ------------------------------------------------------------- Array::new
>> Array.new(size=0, obj=nil)
>> Array.new(array)
>> Array.new(size) {|index| block }
>
> oops, yeah, good catch.

Joel VanderWerf

2/21/2008 10:47:00 PM

0

John Woods wrote:
> >> Array.new(1000000) {str}
>
> Isn't the above still just an array of 1000000 refs to the same string?
> Perhaps you meant str.dup?

Oops, you're right. Using #dup or the following will work:

a = Array.new(5) {"str"}
p a.map {|s| s.object_id}.uniq.size # => 5

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Joel VanderWerf

2/21/2008 10:52:00 PM

0

Keith Barr wrote:
> I am fairly new to Ruby and a program that I have created seems to have
> a memory leak. I have generated a small program which does the basics
> of my program that leaks, and the test program also seems to leak. Can
> anyone spot the design flaw?
>
> Any help you can provide is greatly appreciated.

Does not seem to leak with ruby-1.8.6p111 on linux. (I replaced
"C:/Program Files" with the mount point of my ntfs partition.) It stays
under 18Mb of VM.

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Barr, Keith

2/22/2008 3:55:00 AM

0

Joel VanderWerf wrote:
> Keith Barr wrote:
>> I am fairly new to Ruby and a program that I have created seems to have
>> a memory leak. I have generated a small program which does the basics
>> of my program that leaks, and the test program also seems to leak. Can
>> anyone spot the design flaw?
>>
>> Any help you can provide is greatly appreciated.
>
> Does not seem to leak with ruby-1.8.6p111 on linux. (I replaced
> "C:/Program Files" with the mount point of my ntfs partition.) It stays
> under 18Mb of VM.

Interesting. I am running (obviously) on Windows and I also have a
Linux box, both on patch 111, and both leaking. It isn't a quick leak,
if you run for an a while it might show up. How long did you let it go?

If you don't ever see anything leaking, I wonder what could be different
in our setup or builds.

I certainly don't see anything in there that looks like a memory leak,
but it certainly grows over time on my boxes. Very difficult problem.

KB
--
Posted via http://www.ruby-....

Joel VanderWerf

2/22/2008 5:43:00 AM

0

Keith Barr wrote:
> Interesting. I am running (obviously) on Windows and I also have a
> Linux box, both on patch 111, and both leaking. It isn't a quick leak,
> if you run for an a while it might show up. How long did you let it go?

Only 10 cycles, but it didn't seem to be growing...

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Joel VanderWerf

2/22/2008 6:33:00 AM

0

Joel VanderWerf wrote:
> Keith Barr wrote:
>> Interesting. I am running (obviously) on Windows and I also have a
>> Linux box, both on patch 111, and both leaking. It isn't a quick
>> leak, if you run for an a while it might show up. How long did you
>> let it go?
>
> Only 10 cycles, but it didn't seem to be growing...

After 150 cycles, still under 18M. (This is the same code as yours with
the one dir path modification.)

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Jano Svitok

2/22/2008 10:36:00 AM

0

On Fri, Feb 22, 2008 at 7:32 AM, Joel VanderWerf
<vjoel@path.berkeley.edu> wrote:
> Joel VanderWerf wrote:
> > Keith Barr wrote:
> >> Interesting. I am running (obviously) on Windows and I also have a
> >> Linux box, both on patch 111, and both leaking. It isn't a quick
> >> leak, if you run for an a while it might show up. How long did you
> >> let it go?
> >
> > Only 10 cycles, but it didn't seem to be growing...
>
> After 150 cycles, still under 18M. (This is the same code as yours with
> the one dir path modification.)

I have run it under winxp, the code and results can be found at
http://bryvece...
It seems that it might slowly leak something, but I don't know for sure.

I have used pslist from sysinternals.com for memory measurements.

Ron Fox

2/22/2008 10:58:00 AM

0

Dunno about the memory leak but are you aware that in get_files, you
only go one level of sub-directory deep, rather than down the whole
depth of the directory tree e.g. (with apologies to the excellent
Xwin32 product):

if I have:
c:\program files +--> Starnet
+---->xwin32
+-----> xwin32.exe

I'll never see xwin32.exe in the list of files?
If you want full depth my suggestion is that you
recursively call get_files (when it encounters a directory)
and then append the results it returns to the original result.

Ron.

Keith Barr wrote:
> I am fairly new to Ruby and a program that I have created seems to have
> a memory leak. I have generated a small program which does the basics
> of my program that leaks, and the test program also seems to leak. Can
> anyone spot the design flaw?
>
> Any help you can provide is greatly appreciated.
>
> Thanks,
> Keith
>
>
> =================================
> class LeakTest
> def initialize
> print("initialized\n")
> # fill up the standard 8M of space so the leak becomes apparent
> faster
> str = "0123456789 0123456789 0123456789 0123456789 0123456789
> 0123456789 0123456789 0123456789 0123456789 0123456789 "
> @huge_mem = Array.new(1000000, str)
> end
>
> def get_files(pathname)
> listing = Array.new
>
> Dir.glob("#{pathname}/*",0) do |f|
> # add the files
> listing << f
> # now add directories.
> if(File.directory?(f))
> returnlist = get_files(f)
> returnlist.each {|filename| listing << filename }
> end
> end
> return listing
> end
>
> include GC
> def cleanup
> printf("cleaning up")
> GC.start
> 5.times do
> print"."
> sleep(1)
> end
> print"\n"
> end
>
> def monitor
> listing = get_files("C:/Program Files")
> printf("Found %d files\n", listing.length)
> 5.times do
> print"."
> sleep(1)
> end
> print"\n"
> end
> end
>
> lt = LeakTest.new
> while true
> lt.monitor
> lt.cleanup
> end
> ==========================================