Jano Svitok
3/17/2009 4:43:00 PM
On Tue, Mar 17, 2009 at 16:23, Edouard Dantes <edrd.dantes@gmail.com> wrote:
>
>> Without seeing code it's hard to comment,
>
> Thanks for answer,
>
> well, the loops are too basic to create problems, it's fill array with
> values & write to file.
>
> RAM jumps are too erratic (5 or 10% for 2GB RAM)
>
> The only processor angry stuff (and possibly RAM too) is the following
> parsing line, imported from another file.
>
> Nokogiri::HTML(page).css('div a').map{|dts|
> dts.text.gsub(/\s+/,"").gsub(/[^A-Za-z0-9\s]/,'Z') }
Use /o (once) flag for regexen (gsub(/\s+/o,"")), or make them
constants out of the loop - you'll save two object creations for each
loop.
Try replacing the block body with
tmp = dts.text.gsub(/\s+/,"") # creates a copy of the string, with
whitespace removed
tmp.gsub!(/[^A-Za-z0-9\s]/,'Z') # avoid creating another copy of the string
tmp # return the temporary string
Run your code under RubyMemoryValidator, or at least under ruby-prof
(or any other similar tool) to see what kind of objects get created
and where. If you post the code and some sample data, I might find
some time to run with MemoryValidator for you.
Jano