[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

file size revisit

python152

11/2/2006 1:38:00 AM

Hi, folks

the subject has been reported before, basically, ruby File.size() won't
work properly on windows when file size exceed certain limit (2GB
maybe). One workaround is to require win32/file. Yes, this will get you
correct file, however, this will break the code (in my case, the find
module) ... the previous recursive scan of file system stop
functioning. The question is, is there a more "generic" or "elegant"
way of getting correct file size on both *nix and windows?

thanks

-Oliver
13 Answers

Daniel Berger

11/2/2006 2:54:00 AM

0

python152@gmail.com wrote:
> Hi, folks
>
> the subject has been reported before, basically, ruby File.size() won't
> work properly on windows when file size exceed certain limit (2GB
> maybe). One workaround is to require win32/file. Yes, this will get you
> correct file, however, this will break the code (in my case, the find
> module) ... the previous recursive scan of file system stop
> functioning.

Whoa, what? Can you please elaborate on how win32-file causes problems
with your filesystem scan?

Thanks,

Dan

python152

11/2/2006 4:10:00 AM

0

please see the code segment, once I put in "require win32/file", the
scanning process is working properly anymore.

require 'find'
require 'win32/file'
....

# the following code recursively go into each sub directory and read
file information
# options[] save certain options such as directory to scan and if run
verbosely.

Find.find(options[:dir]) do |path|
if File.file?(path)
if options[:verbose]
puts "Scaning #{path}"
end
size = File.size?(path)
# ... do some work on the file
end
end
end


On Nov 1, 9:53 pm, "Daniel Berger" <djber...@gmail.com> wrote:
> python...@gmail.com wrote:
> > Hi, folks
>
> > the subject has been reported before, basically, ruby File.size() won't
> > work properly on windows when file size exceed certain limit (2GB
> > maybe). One workaround is to require win32/file. Yes, this will get you
> > correct file, however, this will break the code (in my case, the find
> > module) ... the previous recursive scan of file system stop
> > functioning.Whoa, what? Can you please elaborate on how win32-file causes problems
> with your filesystem scan?
>
> Thanks,
>
> Dan

python152

11/2/2006 4:16:00 AM

0

correction: should be "not working anymore" ...
Also, I am not sure why these two modules (or other reasons) are not
working together. Maybe you or someone can enlighten me.

thanks

Oliver

On Nov 1, 11:09 pm, python...@gmail.com wrote:
> please see the code segment, once I put in "require win32/file", the
> scanning process is working properly anymore.
>
> require 'find'
> require 'win32/file'
> ...
>
> # the following code recursively go into each sub directory and read
> file information
> # options[] save certain options such as directory to scan and if run
> verbosely.
>
> Find.find(options[:dir]) do |path|
> if File.file?(path)
> if options[:verbose]
> puts "Scaning #{path}"
> end
> size = File.size?(path)
> # ... do some work on the file
> end
> end
> end
>
> On Nov 1, 9:53 pm, "Daniel Berger" <djber...@gmail.com> wrote:
>
> > python...@gmail.com wrote:
> > > Hi, folks
>
> > > the subject has been reported before, basically, ruby File.size() won't
> > > work properly on windows when file size exceed certain limit (2GB
> > > maybe). One workaround is to require win32/file. Yes, this will get you
> > > correct file, however, this will break the code (in my case, the find
> > > module) ... the previous recursive scan of file system stop
> > > functioning.Whoa, what? Can you please elaborate on how win32-file causes problems
> > with your filesystem scan?
>
> > Thanks,
>
> > Dan

Daniel Berger

11/2/2006 4:30:00 AM

0

python152@gmail.com wrote:
> please see the code segment, once I put in "require win32/file", the
> scanning process is working properly anymore.
>
> require 'find'
> require 'win32/file'
> ...
>
> # the following code recursively go into each sub directory and read
> file information
> # options[] save certain options such as directory to scan and if run
> verbosely.
>
> Find.find(options[:dir]) do |path|
> if File.file?(path)
> if options[:verbose]
> puts "Scaning #{path}"
> end
> size = File.size?(path)
> # ... do some work on the file
> end
> end
> end

Please define "not working any more". What isn't working exactly? Is
there an error message? Or is File.file?(path) not working? If so,
what does "path" look like?

I'm afraid I need more information in order to help you.

Regards,

Dan

python152

11/2/2006 2:39:00 PM

0

ok, usually, I ran the program and supply directory name "." as
current, that doesn't generate any warning message. However, if I
supply a full path, say "C;\temp", these are the error messages on the
console. This happen as soon as I put in "require win32/file"

c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:35:
warning: method rede
ned; discarding old initialize
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:113:
warning: method red
ined; discarding old <=>
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:119:
warning: method red
ined; discarding old blockdev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:125:
warning: method red
ined; discarding old chardev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:132:
warning: method red
ined; discarding old executable?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:136:
warning: discarding
ld executable_real?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:141:
warning: method red
ined; discarding old file?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:148:
warning: method red
ined; discarding old ftype
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:168:
warning: method red
ined; discarding old grpowned?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:173:
warning: method red
ined; discarding old owned?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:179:
warning: method red
ined; discarding old pipe?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:183:
warning: discarding
ld socket?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:187:
warning: method red
ined; discarding old readable?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:193:
warning: method red
ined; discarding old readable_real?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:199:
warning: method red
ined; discarding old setgid?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:205:
warning: method red
ined; discarding old setuid?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:212:
warning: method red
ined; discarding old size?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:218:
warning: method red
ined; discarding old sticky?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:224:
warning: method red
ined; discarding old symlink?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:230:
warning: method red
ined; discarding old writable?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:236:
warning: method red
ined; discarding old writable_real?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:242:
warning: method red
ined; discarding old zero?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:262:
warning: method red
ined; discarding old directory?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:335:
warning: method red
ined; discarding old atime
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:341:
warning: method red
ined; discarding old blksize
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:352:
warning: method red
ined; discarding old blocks
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:359:
warning: method red
ined; discarding old ctime
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:366:
warning: method red
ined; discarding old dev
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:376:
warning: method red
ined; discarding old gid
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:382:
warning: method red
ined; discarding old ino
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:392:
warning: method red
ined; discarding old mode
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:398:
warning: method red
ined; discarding old mtime
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:404:
warning: method red
ined; discarding old rdev
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:410:
warning: method red
ined; discarding old nlink
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:416:
warning: method red
ined; discarding old size
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:422:
warning: method red
ined; discarding old uid
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:428:
warning: method red
ined; discarding old inspect
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:451:
warning: method red
ined; discarding old pretty_print
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:392:
warning: redefine basename
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:446:
warning: redefine dirname
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:503:
warning: redefine split
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:518:
warning: redefine stat
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:530:
warning: redefine blockdev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:537:
warning: redefine chardev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:546:
warning: redefine size
C:\temp
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:263:
warning: instance v
iable @directory not initialized


On Nov 1, 11:29 pm, "Daniel Berger" <djber...@gmail.com> wrote:
> python...@gmail.com wrote:
> > please see the code segment, once I put in "require win32/file", the
> > scanning process is working properly anymore.
>
> > require 'find'
> > require 'win32/file'
> > ...
>
> > # the following code recursively go into each sub directory and read
> > file information
> > # options[] save certain options such as directory to scan and if run
> > verbosely.
>
> > Find.find(options[:dir]) do |path|
> > if File.file?(path)
> > if options[:verbose]
> > puts "Scaning #{path}"
> > end
> > size = File.size?(path)
> > # ... do some work on the file
> > end
> > end
> > endPlease define "not working any more". What isn't working exactly? Is
> there an error message? Or is File.file?(path) not working? If so,
> what does "path" look like?
>
> I'm afraid I need more information in order to help you.
>
> Regards,
>
> Dan

Daniel Berger

11/2/2006 3:51:00 PM

0


Oliver wrote:
> ok, usually, I ran the program and supply directory name "." as
> current, that doesn't generate any warning message. However, if I
> supply a full path, say "C;\temp", these are the error messages on the
> console. This happen as soon as I put in "require win32/file"

<snip>

These are warnins not errors. They are harmless - I redefine a bunch
of methods in the File class.

> C:\temp
> c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:263:
> warning: instance v
> iable @directory not initialized

This one is strange, since @directory *is* initialized. But, I doubt
it's the source of your problem

None of this helps me, however. You still haven't told me exactly what
line specifically is failing. With or without win32/file, you are
going to get nil for File.size?("C:\\temp") or File.size?("."), since
they are directories. It must be something else.

- Dan

python152

11/2/2006 4:16:00 PM

0

Hi Dan -

Find.find(some path) do |path|
puts path
...
end

suppose to loop all the files and sub directories, this is not the case
once import win32/file, the only print out I can get is the very top
level directory I pass in: say, if I run ruby test.rb -d "c:\temp",
then the only print out is "c:\temp", all files in that directory are
ignored. It should be pretty easy to verify. It almost seems like find
module has a dependency on File module, and win32/file is in odd with
it.

This is probably as much as I can find at this point ...

Oliver



On Nov 2, 10:50 am, "Daniel Berger" <djber...@gmail.com> wrote:
> Oliver wrote:
> > ok, usually, I ran the program and supply directory name "." as
> > current, that doesn't generate any warning message. However, if I
> > supply a full path, say "C;\temp", these are the error messages on the
> > console. This happen as soon as I put in "require win32/file"<snip>
>
> These are warnins not errors. They are harmless - I redefine a bunch
> of methods in the File class.
>
> > C:\temp
> > c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:263:
> > warning: instance v
> > iable @directory not initializedThis one is strange, since @directory *is* initialized. But, I doubt
> it's the source of your problem
>
> None of this helps me, however. You still haven't told me exactly what
> line specifically is failing. With or without win32/file, you are
> going to get nil for File.size?("C:\\temp") or File.size?("."), since
> they are directories. It must be something else.
>
> - Dan

Daniel Berger

11/2/2006 5:27:00 PM

0


Oliver wrote:
> Hi Dan -
>
> Find.find(some path) do |path|
> puts path
> ...
> end
>
> suppose to loop all the files and sub directories, this is not the case
> once import win32/file, the only print out I can get is the very top
> level directory I pass in: say, if I run ruby test.rb -d "c:\temp",
> then the only print out is "c:\temp", all files in that directory are
> ignored. It should be pretty easy to verify. It almost seems like find
> module has a dependency on File module, and win32/file is in odd with
> it.

Aha! The culprit appears to be lstat. In the find module you'll see
this line:

if File.lstat(file).directory? then

That returns nil when you include win32-file:

irb(main):001:0> dir = "C:\\TMP"
=> "C:\\TMP"
irb(main):002:0> File.lstat(dir)
=> #<File::Stat dev=0x2, ino=0, mode=040755, nlink=1, uid=0, gid=0,
rdev=0x2, size=0, blksize=nil, blocks=nil, atime=Thu
Nov 02 09:56:43 -0700 2006, mtime=Fri Oct 21 09:47:09 -0600 2005,
ctime=Fri Oct 21 09:47:09 -0600 2005>
irb(main):003:0> File.lstat(dir).directory?
=> true
irb(main):004:0> require 'win32/file'
=> true
irb(main):005:0> File.lstat(dir).directory?
=> nil

I didn't reallize lstat was implemented on Windows. I'm guessing it's
an alias, so I'll need to check the source. But, that's definitely a
bug in win32-file (or rather, win32-file-stat).

I'll get this fixed and put out a release asap.

Thanks,

Dan

Daniel Berger

11/2/2006 5:54:00 PM

0


Daniel Berger wrote:

<snip>

> I didn't reallize lstat was implemented on Windows. I'm guessing it's
> an alias, so I'll need to check the source. But, that's definitely a
> bug in win32-file (or rather, win32-file-stat).
>
> I'll get this fixed and put out a release asap.

Ok, fixed in CVS. I'll have a release out tonight. If you want the
fix now, just open up win32/file.rb and add this:

def self.lstat(file)
File::Stat.new(file)
end

Regards,

Dan

python152

11/2/2006 6:15:00 PM

0

Dan -

First of all, thanks for fixing this.

A general comment on this issue is: why in Ruby, the basic operation on
files needs special treatment, and short of being cross-platform? If I
am invoking a exotic feature that is windows only, import a win32
module would make more sense to me.

In Python, the operation is uniform cross the platform
import os.path
os.path.getsize("filename")

maybe many folks probably won't agree this, but I tend to regard the
core File.size() won't work with larger file on windows a bug, though a
minor one.

Best,
Oliver




On Nov 2, 12:54 pm, "Daniel Berger" <djber...@gmail.com> wrote:
> Daniel Berger wrote:<snip>
>
> > I didn't reallize lstat was implemented on Windows. I'm guessing it's
> > an alias, so I'll need to check the source. But, that's definitely a
> > bug in win32-file (or rather, win32-file-stat).
>
> > I'll get this fixed and put out a release asap.Ok, fixed in CVS. I'll have a release out tonight. If you want the
> fix now, just open up win32/file.rb and add this:
>
> def self.lstat(file)
> File::Stat.new(file)
> end
>
> Regards,
>
> Dan