[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

How to edit a file without loading it all into memory

Jonathan Dobbie

11/9/2007 8:51:00 PM

I need to change the texture paths in maya files. The lines are easy to
find with grep, and I'm able to create the new line easily enough, but
how can I do this without holding the entire file into memory? The
files can be hundreds of megabytes.

Thanks.
--
Posted via http://www.ruby-....
7 Answers

Reid Thompson

11/9/2007 9:04:00 PM

0

newer versions of sed can replace text in place in files.

there's also perl's

perl -pi -e 's/to change this/to change that/' filename


On Sat, 2007-11-10 at 05:51 +0900, Jonathan Dobbie wrote:
> I need to change the texture paths in maya files. The lines are easy to
> find with grep, and I'm able to create the new line easily enough, but
> how can I do this without holding the entire file into memory? The
> files can be hundreds of megabytes.
>
> Thanks.

Robert Klemme

11/9/2007 9:07:00 PM

0

On 09.11.2007 21:51, Jonathan Dobbie wrote:
> I need to change the texture paths in maya files. The lines are easy to
> find with grep, and I'm able to create the new line easily enough, but
> how can I do this without holding the entire file into memory? The
> files can be hundreds of megabytes.

As you said: do it line by line. This is the usual approach to this
problem. You might want to have a look at Ruby's command line options.
Basically you can do something like this for an inplace edit:

ruby -ni.bak -e 'your replacement code'

Kind regards

robert

7stud --

11/9/2007 9:11:00 PM

0

Jonathan Dobbie wrote:
> The lines are easy to
> find with grep, and I'm able to create the new line easily enough, but
> how can I do this without holding the entire file into memory? The
> files can be hundreds of megabytes.
>

File.foreach("data.txt") do |line|
#look for something in line
end

--
Posted via http://www.ruby-....

Jonathan Dobbie

11/9/2007 9:22:00 PM

0

7stud -- wrote:
> Jonathan Dobbie wrote:
>> The lines are easy to
>> find with grep, and I'm able to create the new line easily enough, but
>> how can I do this without holding the entire file into memory? The
>> files can be hundreds of megabytes.
>>
>
> File.foreach("data.txt") do |line|
> #look for something in line
> end

Here is what I originally wrote, and what I just tried. Am I missing
something obvious? The first one duplicates the lines, and the second
one does nothing. (doesn't save the changes)

def change_texture_paths
File.open(Dir.glob(ENV['HOME'] + "/*.ma").first, "r+") do |file|
file.grep( /setAttr ".fileTextureName" -type "string"/ ) do |line|
texture_file_path = line[52..-2]
texture_file_name = texture_file_path.split("\/").last
line = " setAttr \".fileTextureName\" -type \"string\"
\"" + ENV['HOME'] + "/include/" + texture_file_name + "\n"
file.print line
end
end


File.foreach(Dir.glob(ENV['HOME'] + "/*.ma").first, "r+") do |line|
if (line =~ /setAttr ".fileTextureName" -type "string"/ )
texture_file_path = line[52..-2]
texture_file_name = texture_file_path.split("\/").last
line = " setAttr \".fileTextureName\" -type \"string\"
\"" + ENV['HOME'] + "/include/" + texture_file_name + "\n"
file.print line
end
end
end
--
Posted via http://www.ruby-....

Kyle Schmitt

11/9/2007 9:40:00 PM

0

Err.

Well only one of these textures is open at once right?
If this box is actually capable of running maya well, it's sure to
have more than a few hundred megs of free ram (when maya isn't running
of course ;).

My guess is that you'll get a hefty performance hit for parsing the
files line by line. The box has memory to spare, so why not just use
it?

Of course if there was a way to memap the file under ruby that would
probably be the best...but I dunno how to do that.

--Kyle

Kyle Schmitt

11/9/2007 9:40:00 PM

0

mem map. not memap..but you get the idea.

7stud --

11/10/2007 12:54:00 AM

0

Jonathan Dobbie wrote:
> 7stud -- wrote:
>> Jonathan Dobbie wrote:
>>> The lines are easy to
>>> find with grep, and I'm able to create the new line easily enough, but
>>> how can I do this without holding the entire file into memory? The
>>> files can be hundreds of megabytes.
>>>
>>
>> File.foreach("data.txt") do |line|
>> #look for something in line
>> end
>
> Here is what I originally wrote, and what I just tried. Am I missing
> something obvious? The first one duplicates the lines, and the second
> one does nothing. (doesn't save the changes)
>

1) Are you under the impression that every method in ruby has the same
parameters?

2) When you see an error message, do you just ignore it?

--
Posted via http://www.ruby-....