[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

remote file glob

greg

11/2/2006 3:00:00 PM

What is the fastest way to do a recursive file glob on a remote server?
Keep in mind that I want to do some filtering based on the directory
name, file properties, and file extensions.
(I want to copy these files, but I assume once I have the list, that
using Net::FTP will be trivial)

I am concerned that just using ftp.chdir and ftp.list will be slow.
Perhaps there is a faster way using Net:SSH. I had an idea to try to
run a ruby program on the server (ruby is installed on the remote
server)
cmd = "ruby -e ' #{ File.read( ruby_glob_program ) } ' "

Net::SSH.start(SERVER, :username => u, :password => p) do |session|
input, output, error = session.process.popen3( cmd )
3 Answers

Robert Klemme

11/2/2006 3:33:00 PM

0

On 02.11.2006 16:00, greg wrote:
> What is the fastest way to do a recursive file glob on a remote server?
> Keep in mind that I want to do some filtering based on the directory
> name, file properties, and file extensions.
> (I want to copy these files, but I assume once I have the list, that
> using Net::FTP will be trivial)
>
> I am concerned that just using ftp.chdir and ftp.list will be slow.
> Perhaps there is a faster way using Net:SSH. I had an idea to try to
> run a ruby program on the server (ruby is installed on the remote
> server)
> cmd = "ruby -e ' #{ File.read( ruby_glob_program ) } ' "

That would rather be something like

ruby -e 'puts Dir["base/**/*.txt"]'
ruby -r find -e 'Find.find("base") {|f| puts f if /\.txt$/ =~ f }'

> Net::SSH.start(SERVER, :username => u, :password => p) do |session|
> input, output, error = session.process.popen3( cmd )

However, if you just want to find files fast and then copy them, a
combination of ssh, find, xargs, cp, tar,... might be more efficient.

Kind regards

robert

M. Edward (Ed) Borasky

11/2/2006 3:41:00 PM

0

greg wrote:
> What is the fastest way to do a recursive file glob on a remote server?
> Keep in mind that I want to do some filtering based on the directory
> name, file properties, and file extensions.
> (I want to copy these files, but I assume once I have the list, that
> using Net::FTP will be trivial)
>
> I am concerned that just using ftp.chdir and ftp.list will be slow.
> Perhaps there is a faster way using Net:SSH. I had an idea to try to
> run a ruby program on the server (ruby is installed on the remote
> server)
> cmd = "ruby -e ' #{ File.read( ruby_glob_program ) } ' "
>
> Net::SSH.start(SERVER, :username => u, :password => p) do |session|
> input, output, error = session.process.popen3( cmd )
>
>
>
If you're talking about a Linux or other Unix-like server, indexing a
filesystem is not something you want to do often. It *is* slow, even
using native utilities like "find", or its cousin "slocate -u". What
most sites do is run a cron job in the off hours to build the "slocate
-u" database, and then do searches using the database.

If you need more current indexing, you probably need to look at the
application and have it do its own index maintenance.

greg

11/2/2006 6:05:00 PM

0

I think I will just use the find command.

I did notice that if I do the following:
session.process.popen3( cmd ) do |input, output, error|
puts output.read
end

For my find command, it will think it is finished reading before find
is done!
I solved this by using:

shell = session.shell.sync
puts shell.exec( find_cmd ).stdout

Thanks guys!