[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

TempFile

Matt Maycock

10/8/2004 7:23:00 PM

[ummaycoc@localhost ummaycoc]$ ruby -v
ruby 1.8.2 (2004-06-21) [i686-linux]

I have an issue with tempfile. I'm going through a loop which creates
two temp files, writes data, uses them to generate graphs (wonderful
gnuplot), and then destroys them. Sometime after a few
iterations(sometimes 4, sometimes 100, sometimes not at all [once]), I
get to a point where I need to use the contents of the tempfile, and
the file is not there! I've done nothing to erase the file. It's
sorta nondeterministic. If I hit `up' in bash and rexecute my ruby
-ne line, I get the same problem with the same iteration at the same
point in my code. If I `change' any line in a file involved with the
process (my gnuplot wrapper, some random file that gets included) - it
can change where in the code and what iteration - even if the change
is equiv to `touch #{file}`

I can post code if anyone wants to wade through my mess, but I don't
think it's a `me' issue (ie not my fault -- at least I hope :-)

Any similar experiences / advice?

~Me!

--
There's no word in the English language for what you do to a dead
thing to make it stop chasing you.


17 Answers

Ara.T.Howard

10/9/2004 2:46:00 AM

0

Matt Maycock

10/9/2004 8:06:00 PM

0

Nope :-(

Here's my method that I'm using:

def handle.add
unless block_given? then
@done[]
@error[ArgumentError, 'Block not given for Plot # data #
handle.add. A block is required.']
end

file = Tempfile.new('plotdata')
filehandle, point = Object.new, @point
filehandle.singleton_def(:puts) {|*points|
file.puts(point[*points])
}

yield filehandle
file.close(false)

@debug[:data_file, File.readlines(file.path).map {|l| l.chomp}]
@add[file]
end

singleton_def is just this:
class Object
def singleton_class
class << self; self; end
end
def singleton_def(sym, &block)
self.singleton_class.send(:define_method, sym, &block)
end
end

now - I at one point had
p FileTest.exists?(file.path)
after each line after file was defined. at one point, it was true
before the singleton_def, and false afterwards. there are no threads
/ forking, etc.


Ara.T.Howard

10/10/2004 12:24:00 AM

0

YANAGAWA Kazuhisa

10/10/2004 3:22:00 AM

0

In Message-Id: <e86cebfb041009130540c16815@mail.gmail.com>
Matt Maycock <ummaycoc@gmail.com> writes:

> def handle.add
> unless block_given? then
> @done[]
> @error[ArgumentError, 'Block not given for Plot # data #
> handle.add. A block is required.']
> end
>
> file = Tempfile.new('plotdata')
> filehandle, point = Object.new, @point
> filehandle.singleton_def(:puts) {|*points|
> file.puts(point[*points])
> }
>
> yield filehandle
> file.close(false)
>
> @debug[:data_file, File.readlines(file.path).map {|l| l.chomp}]
> @add[file]
> end

For my eyes @done, @error, @debug and @add are a some object which has
method `[]', that do other than reference a something they contain.
They are probably not an Array or Hash or Set or ...., right?

If @add[file] does not store file anywhere, file can be GCed after
handle.add (hmm add is a singleton method of handle?) and a
corresponding temporary file is deleted by finalizer of Tempfile.


--
kjana@dm4lab.to October 10, 2004
Time and tide wait for no man.



Yukihiro Matsumoto

10/10/2004 3:59:00 AM

0

Hi,

In message "Re: TempFile"
on Sun, 10 Oct 2004 05:05:35 +0900, Matt Maycock <ummaycoc@gmail.com> writes:

|Here's my method that I'm using:

Can you show us error reproducing "whole" script?
I don't think of any reason of your problem from your code chunk.

matz.


Matt Maycock

10/10/2004 8:55:00 PM

0

> Can you show us error reproducing "whole" script?
> I don't think of any reason of your problem from your code chunk.
>
> matz.

You asked for it :-)

Here's the script, with all code it calls being included (thus, really
long post)

#------------------------------------------------------------#
#--- COMMAND LINE Script I Use --- #
#--- requires ext/default_plotter and --- #
#--- ext/enum_from - they follow --- #
# reads in a list of csv files that my boss gave me, and takes out
# some experiment data, and adds it into the default plotter to make
# graphs! Not really that complicated.

ls *.csv | ruby -r ext/default_plotter -r ext/enum_from -ne '
lines = File.readlines(chomp).map {|i| i.chomp.split(/\t/)}
header = lines.shift

plotter = `which gnuplot`.chomp

lines.inject([]) {|arr, line|
arr << header.inject(Hash.new) {|h, f| h[f] = line.shift; h}
}.each {|row|
id = row["cloneid"].gsub(/\//, "_")
file = "../../results/#{chomp.gsub(/\.csv$/, "")}/#{id}.png"

Dir.mkdir(File.dirname(file)) unless FileTest.exists?(File.dirname(file))

control, deprived = * %w{CON DEP}.map {|tag|
[0, *[3,6,9,12].map {|i| row["#{i}_#{tag}"].to_f - row["0_CON"].to_f}]
}

xlabel, ylabel, xtics, ytics, xrange, yrange = nil
xlabel = ["Gene Title", "Gene Symbol"].map {|k| "#{k}: #{row[k]}"}
xlabel << ""
table = [%w{Time Control Deprived}]
[0, 3, 6, 9, 12].each_with_index {|e, i|
table << [e.to_s.rjust(3), control[i], deprived[i]].map {|i| i.to_s}
}

frac = /^(-?\d*)(\.?\d*)$/
ilens = (1..2).map {|i|
table.map {|a| a[i] =~ frac ? $1.length : 0}.max
}
flens = (1..2).map {|i|
table.map {|a| a[i] =~ frac ? $2.length : 0}.max
}

(1..2).each {|i|
table.each {|a|
a[i] = "#{$1.rjust(ilens[i-1])}#{$2.ljust(flens[i-1])}" if a[i] =~ frac
}
}

lens = (0...(table[0].length)).map {|i| table.map {|a| a[i].length}.max}
dlens = (0...(table[0].length)).map {|i| table.from(1).map {|a|
a[i].length}.max}
table.each_with_index {|arr, ai|
index = 0
if ai == 0 then
xlabel << arr.map {|item|
actualmax = lens[index]
item.center(actualmax+2)
}.join(" ")
next
end

xlabel << arr.map {|item|
actualmax = lens[index]
datamax = dlens[index]
v = item.rjust(datamax+1)
v.center(actualmax+1)
index += 1
v
}.join(" ")
}

xlabel = xlabel.join("\\n")

DefaultPlotter.data {|files|
[control, deprived].each {|arr|
files.add {|filehandle|
arr.each_with_index {|e, i|
filehandle.puts(e, i*3)
}
}
}

DefaultPlotter.plot(file, plotter, id, xlabel, ylabel, xtics,
ytics, xrange, yrange, "png") {|plot|
files.each_path_with_index {|path, idx|
plot.add(File.expand_path(path), %w{Control Deprived}[idx],
"lines", "2:1")
}
}
}
}
'

#------------------------------------------------------------#
#--- ext/enum_from --- #
# Allows you to take a nth tail...
module Enumerable
def from(index)
result = []
self.each_with_index {|e, i|
next unless i >= index
yield e if block_given?
result << e
}

result
end
end

#------------------------------------------------------------#
#--- ext/default_plotter ---#
#--- requires ext/gnuplot - which follows ---#
# Just an alias for a `plotting' object.
require "ext/gnuplot"
DefaultPlotter = GnuPlot

#------------------------------------------------------------#
#--- ext/gnuplot ---#
#--- requires ext/plot and ext/singleton_class - which follow ---#
# Set up to use gnuplot to make wonderful graphs.
require 'ext/plot'
require 'ext/singleton_class'
require 'tempfile'

module GnuPlot
end

# Take an output (`file' to write to, basically)
# a title (what to call graph)
# an xlabel (what to call x-axis)
# a ylabel (what to call y-axis)
# an xtics (how to mark x-axis)
# a ytics (how to mark y-axis)
# an xrange (what part of x-axis to show)
# a yrange (what part of y-axis to show)
# a format (what to write `as' - png, gif, eps, etc)
# block - takes the `plot' object and interacts with it.
#
# the plot object the block gets has the following methods:
# close : closes the plot command file - now useable for making gnuplots!
# add(path, title, style, use) : add the data from the file with
path, use title and style, with columns use
# path : gives the path of the file containing the gnuplot commands
def GnuPlot.plotter(output, title=nil, xlabel=nil, ylabel=nil,
xtics=nil, ytics=nil, xrange=nil, yrange=nil, format=nil, &block)
raise ArgumentException, "No block given" if block.nil?

if Hash === output then
h = output
syms = [:output, :title, :xlabel, :ylabel, :xtics, :ytics,
:xrange, :yrange, :format]
make_gnuplot(*syms.map {|sym| h[sym]}, &block)
end
raise ArgumentException, "Cannot accept nil output destination." if
output.nil?

gpcmd = Tempfile.new('run_gnuplot_cmds_script')
[["set terminal #{format}", false],
["set output \"#{output}\"", false],
["set pointsize 1.4", false],
["set title \"#{title}\"", title.nil?],
["set xlabel \"#{xlabel}\"", xlabel.nil?],
["set ylabel \"#{ylabel}\"", ylabel.nil?],
["set xrange [#{xrange}]", xrange.nil?],
["set yrange [#{yrange}]", yrange.nil?],
["set xtics (#{xtics})", xtics.nil?],
["set ytics (#{ytics})", ytics.nil?]].each {|str, skip|
gpcmd.puts str unless skip
}

handle = Object.new
closed = false
gpcmds = []

handle.singleton_def(:close) {
next if closed
closed = true

index = 0
space = ' ' * 'plot'.length

cmds = []
gpcmds.each_with_index {|c, i|
cmds << "#{i == 0 ? 'plot' : space} #{c}"
}

gpcmd.puts cmds.join(",\\\n")
gpcmd.close
}

handle.singleton_def(:add) {|path, title, style, using|
cmd = path.inspect
if using.nil? then
cmd += ' using 1:2'
else
cmd += " using #{using}"
end

if title.nil? then
cmd += ' notitle'
else
cmd += " title #{title.inspect}"
end

cmd += " with #{style}" unless style.nil?
gpcmds << cmd
}

handle.singleton_def(:path) {
gpcmd.path
}

result = nil
begin
result = block[handle]
gpcmd.unlink
rescue Exception
begin
gpcmd.unlink
rescue Exception
end
raise
end
result
end

# Turns a set of points into something gnuplot understands as separate
coordinates.
def GnuPlot.make_point(*points)
points.join("\t")
end

# Make GnuPlot a plot!
GnuPlot.extend(Plot)

# Make a gnuplot range! here, we have a name for the range (x?, y?),
and the ends.
# the ends can be numbers, or nolow nohigh...
def GnuPlot.make_range(name, a, b, errorp=nil)
error = errorp.nil? ? Proc.new{|exc, msg|
raise exc, msg
} : Proc.new {|exc, msg|
errorp[exc, msg]
return
}

a = a.strip.downcase
b = b.strip.downcase
error[ArgumentError, "#{name}: both points are #{a.upcase}."] if a
=~ /^no(low|high)$/ && a == b
return make_range[name, b, a] if b == 'nolow' || a == 'nohigh'

floata, floatb = *[a, b].map {|c|
!(/^(-|\+)?(\d+)?\.?(\d+)?$/.match(c).nil? || /\d/.match(c).nil?)}
noa, nob = *[a, b].map {|c| ['nolow', 'nohigh'].include?(c)}
error[ArgumentError, "Invalid points: #{a}, #{b}."] if !(floata ||
noa) && !(floatb || nob)
a = (a == 'nolow') ? '' : a.to_f
b = (b == 'nohigh') ? '' : b.to_f
a, b = b, a if Float === a && Float === b && b < a
a, b = a.to_s, b.to_s
"#{a}:#{b}"
end

#------------------------------------------------------------#
#--- ext/plot ---#
#--- requires ext/singleton_class - which follows ---#
require 'tempfile'
require 'ext/singleton_class'

module Plot
# Output is `where' to go with the graph (file)
# plotter is the actual command to invoke.
# the rest, except errorp and debugp are the same as gnuplot.plotter args
# errorp and debugp are procs toc all with error and debug information.
#
# Needs a block - that block is passed a plot object who's sole method is add,
# which is the same as the add method from the object yieled by
plotter's block.
def plot(output, plotter, title=nil, xlabel=nil, ylabel=nil,
xtics=nil, ytics=nil, xrange=nil, yrange=nil, format=nil, errorp=nil,
debugp=nil)
error = errorp.nil? ? Proc.new{|exc, msg|
raise exc, msg
} : Proc.new {|exc, msg|
errorp[exc, msg]
return
}
debug = debugp.nil? ? Proc.new {} : debugp

error[ArgumentError, 'Block not given for Plot # plot. A block is
required.'] unless block_given?

plotargs = [output, title, xlabel, ylabel, xtics, ytics, xrange,
yrange, format]
plotblock = Proc.new {|plot_handle|
yield(Object.new.singleton_def(:add) {|*args| plot_handle.add(*args)})
plot_handle.close

debug[:command_file, File.readlines(plot_handle.path).map {|i| i.chomp}]

`#{plotter} #{plot_handle.path}`
}

plotter(*plotargs, &plotblock)
end

# Basically, make an object that is a database of plotting files.
# errorp and debugp are the same as above.
# Takes a block - this block gets the `database'
# this database has the following methods:
# add - calling this gets you a filehandle back in the block add gets
# this filehandle can be given points to add, and when the block
# is done, the file gets added to the end of the list of data files
# use_file(fname) - adds fname to the end of the list of data files
# each_path_with_index - use block to go over each file in the list of
# of data files and have the index in the list, too
# each_path - same as above, no index
# at the end of the block, the database's files are `cleared'
def data(errorp=nil, debugp=nil)
error = errorp.nil? ? Proc.new{|exc, msg|
raise exc, msg
} : Proc.new {|exc, msg|
errorp[exc, msg]
return
}
debug = debugp.nil? ? Proc.new {} : debugp

error[ArgumentError, 'Block not given for Plot # data. A block is
required.'] unless block_given?
handle = Object.new

files = []

done = Proc.new {files.each {|f| f.unlink}}
add = Proc.new {|file| files << file}
enum = Proc.new {|block, index|
files.each_with_index {|f, i|
block[f.path, i] if index
block[f.path] unless index
}
}
point = Proc.new {|*args| make_point(*args)}

handle.instance_eval {
@done, @add, @enum,
@error, @debug, @point = done, add, enum,
error, debug, point
}

def handle.add
unless block_given? then
@done[]
@error[ArgumentError, 'Block not given for Plot # data #
handle.add. A block is required.']
end

file = Tempfile.new('plotdata')
filehandle, point = Object.new, @point
filehandle.singleton_def(:puts) {|*points|
file.puts(point[*points])
}

yield filehandle
file.close(false)

@debug[:data_file, File.readlines(file.path).map {|l| l.chomp}]
@add[file]
end

def handle.use_file(fname)
@error[ArgumentError, "File #{fname} does not exist."] unless
FileTest.exists?(fname)
@error[ArgumentError, "File #{fname} isn't a file."] unless
FileTest.file?(fname)
@error[ArgumentError, "File #{fname} isn't readable."] unless
FileTest.readable?(fname)
@add[File.open(fname).singleton_def(:unlink) {self.close}]
end

def handle.each_path_with_index(&block)
if block.nil? then
@done[]
@error[ArgumentError, 'Block not given for Plot # data #
handle.each_path_with_index. A block is required']
end

@enum[block, true]
end

def handle.each_path(&block)
if block.nil? then
@done[]
@error[ArgumentError, 'Block not given for Plot # data #
handle.each_path. A block is required']
end

@enum[block, false]
end

yield handle

files.each {|f| f.unlink}
files.clear

nil
end
end

#------------------------------------------------------------#
#--- ext/singleton_class ---#
class Object
def singleton_class
class << self
self
end
end

def singleton_def(sym, &block)
self.singleton_class.send(:define_method, sym, &block)
self
end
end

--
There's no word in the English language for what you do to a dead
thing to make it stop chasing you.


Yukihiro Matsumoto

10/11/2004 1:12:00 AM

0

Hello,

In message "Re: TempFile"
on Mon, 11 Oct 2004 05:54:57 +0900, Matt Maycock <ummaycoc@gmail.com> writes:

|> Can you show us error reproducing "whole" script?
|> I don't think of any reason of your problem from your code chunk.
|
|You asked for it :-)

I couldn't reproduce a bug even using ruby 1.8.2 (2004-06-21) [i386-linux].
Probably there's something missing, input data or your platform.

matz.


Matt Maycock

10/11/2004 1:30:00 AM

0

[ummaycoc@localhost timepoint]$ uname -a
Linux localhost 2.6.3-7mdk-i686-up-4GB #1 Wed Mar 17 15:17:23 CET 2004
i686 unknown unknown GNU/Linux

data: http://www.cs.drexel.edu/~ummaycoc/da...

thanks!

~Me!

--
There's no word in the English language for what you do to a dead
thing to make it stop chasing you.


Matt Maycock

10/11/2004 7:09:00 PM

0

Problem fixed, I think (let's hope this doesn't introduce new issues)

tempfile.rb: add ObjectSpace.undefine_finalizer(self) at the end of
the unlink method. It was a GC issue - just not mine (well, still
mine cause it messed with me...) :-)

~Me!

--
There's no word in the English language for what you do to a dead
thing to make it stop chasing you.


Yukihiro Matsumoto

10/11/2004 11:29:00 PM

0

Hi,

In message "Re: TempFile"
on Tue, 12 Oct 2004 04:09:02 +0900, Matt Maycock <ummaycoc@gmail.com> writes:

|Problem fixed, I think (let's hope this doesn't introduce new issues)
|
|tempfile.rb: add ObjectSpace.undefine_finalizer(self) at the end of
|the unlink method. It was a GC issue - just not mine (well, still
|mine cause it messed with me...) :-)

You've found and fixed a bug in tempfile.rb. I will merge it in
1.8.2. Thank you.

matz.