[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Manipulating CSV files over SSH

Drew Olson

2/3/2007 10:48:00 PM

All -

I have a nice little script that takes a large CSV document and splits
it into 65,000 line chunks which are readable in Excel. However, I
currently have a .csv file which I'd like to split that is sitting on a
unix box. I have ssh access to the box, however the file is quite big
and I'd like to avoid downloading the whole file. Is there a simple way
to modify my script such that I use the ssh library to access the
document and perform the splitting over the network?

Thanks in advance,
Drew

--
Posted via http://www.ruby-....
7 Answers

Logan Capaldo

2/3/2007 11:18:00 PM

0

On Sun, Feb 04, 2007 at 07:47:31AM +0900, Drew Olson wrote:
> All -
>
> I have a nice little script that takes a large CSV document and splits
> it into 65,000 line chunks which are readable in Excel. However, I
> currently have a .csv file which I'd like to split that is sitting on a
> unix box. I have ssh access to the box, however the file is quite big
> and I'd like to avoid downloading the whole file. Is there a simple way
> to modify my script such that I use the ssh library to access the
> document and perform the splitting over the network?
>
> Thanks in advance,
> Drew
>
ssh user@host -x 'cat large.csv' | your_script
should work pretty well for this.
> --
> Posted via http://www.ruby-....

Jano Svitok

2/3/2007 11:19:00 PM

0

On 2/3/07, Drew Olson <olsonas@gmail.com> wrote:
> All -
>
> I have a nice little script that takes a large CSV document and splits
> it into 65,000 line chunks which are readable in Excel. However, I
> currently have a .csv file which I'd like to split that is sitting on a
> unix box. I have ssh access to the box, however the file is quite big
> and I'd like to avoid downloading the whole file. Is there a simple way
> to modify my script such that I use the ssh library to access the
> document and perform the splitting over the network?
>
> Thanks in advance,
> Drew

The natural way would be to upload the script there and run it on he
remote host, provided that ruby is installed on the host.

If you don't need CSV escaping, (embedded newlines etc.) then using
split(1) should be enough, i.e.

split -l 65000 filename prefix

for alphanum counter and

split -l 65000 -d filename prefix

for numerical one.

Cameron McBride

2/3/2007 11:23:00 PM

0

On 2/3/07, Logan Capaldo <logancapaldo@gmail.com> wrote:
> On Sun, Feb 04, 2007 at 07:47:31AM +0900, Drew Olson wrote:
> > All -
> >
> > I have a nice little script that takes a large CSV document and splits
> > it into 65,000 line chunks which are readable in Excel. However, I
> > currently have a .csv file which I'd like to split that is sitting on a
> > unix box. I have ssh access to the box, however the file is quite big
> > and I'd like to avoid downloading the whole file. Is there a simple way
> > to modify my script such that I use the ssh library to access the
> > document and perform the splitting over the network?
> >
> > Thanks in advance,
> > Drew
> >
> ssh user@host -x 'cat large.csv' | your_script
> should work pretty well for this.

That is essentially downloading the entire file, which he doesn't want to do.

Wow, hot topic. I was going to suggest what Jan Svitok did: The
easiest thing to do is to upload and run the ruby script on the unix
box.

If the script takes a while to run, you might want to check out
"screen" (but that is a bit off topic - ping me personally if you need
more info on this than you can google).

Cameron

Thomas Hafner

2/3/2007 11:51:00 PM

0

Drew Olson <olsonas@gmail.com> wrote/schrieb <9914f65ae092f7af0f024f289c0282b2@ruby-forum.com>:

> I have a nice little script that takes a large CSV document and splits
> it into 65,000 line chunks which are readable in Excel. However, I
> currently have a .csv file which I'd like to split that is sitting on a
> unix box.

Where shall the result of that splitting reside? Also on the same unix
box where the original CSV document resides?

Regards
Thomas

Drew Olson

2/4/2007 12:59:00 AM

0

> Where shall the result of that splitting reside? Also on the same unix
> box where the original CSV document resides?
>
> Regards
> Thomas

Yes, I'd like the split files to be on the unix box as well. Obviously,
the easy solution is to install rails but I can't (client paranoid,
etc). I do have ksh or perl to work with, but I have a working ruby
script which i'd like to use. Any other ideas?

-Drew

--
Posted via http://www.ruby-....

Drew Olson

2/4/2007 6:49:00 PM

0

Drew Olson wrote:
> Yes, I'd like the split files to be on the unix box as well. Obviously,
> the easy solution is to install rails but I can't (client paranoid,
> etc). I do have ksh or perl to work with, but I have a working ruby
> script which i'd like to use. Any other ideas?
>
> -Drew

Any ideas? If I can't do this with ruby does anyone here have any ksh or
perl - foo to share?

-Drew

--
Posted via http://www.ruby-....

Ara.T.Howard

2/4/2007 7:03:00 PM

0