Asp Forum - yaml format - comp.lang.ruby

Charles L. Snyder

6/9/2006 4:29:00 PM

Hi

Pardon the long post:
I am playing around with one of the ruby quiz problems (stock
portfolio) - trying to modify it to do some other things. I am reading
in data from a csv file, getting the stock symbols from it, and using
the symbols via Yahoo finance to get data, which I output into a .yaml
format. Everything is fine, except that I get a screwy yaml output, and
can't figure out where the extra " and \ are coming from.

Here is the ruby code:

#!/usr/local/bin/ruby

# Stock data via: http://www.gummy-stuff.org/Yaho...

require 'open-uri'
require 'yaml'
require 'ostruct'
require 'csv'

# myfile = 'mystocks_final.yaml' # change this for your file
data = ["s","n","p","p2"] #symbols to check on yahoo

# stock_struct = Struct::new( "StockStruct", :name, :shares, :symbol,
:date, :ass_class)
# get the data from the yaml file
portfolio = YAML.load( open('my_db4.yaml').read )

stocks = []
portfolio.each {|r| stocks.push(r.symbol)} # stick the symbols into the
stock array
data = data.map { |tag| tag[/\w+/] }.join

# put in a delete from array for those stocks with incorrect or
unobtainable symbols
bad_symbols = ["CREF Bond","CREF
Equity","8401765","CREFAnn","8401463","CREF Global
Eq","8806422","CASH", "CREF Stock", "CREF Equity IN"]
stocks.delete_if {|x| bad_symbols.include?(x)}
stocks = stocks.join("+")
stock_struct = Struct::new( "StockStruct", :symbol, :name, :price,
:date, :shares, :ass_class, :ass_subclass)
w = []
open "http://finance.yahoo.com/d/quot...{stocks}&f=#{data}" do
|m|
m.each do |n|
date_now = Time.now.localtime.strftime("%Y-%m-%d")
n = n.split(/,/)
w.push(stock_struct.new(n[0], n[1], n[2],date_now, 5, "Domestic
Equity","Precious Metal"))
end
end

open("my_yaml_output.yaml","w"){|e| e << w.to_yaml}

Here is a sample of the my_yaml_output file:

---
- !ruby/struct:StockStruct
symbol: "'\"AAPL\"'"
name: "\"APPLE COMPUTER\""
price: "60.76"
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: "'\"BGEIX\"'"
name: "\"AMERICAN CENTURY \""
price: "17.79"
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal

what I want is: (get rid of all the extra " and \)

---
- !ruby/struct:StockStruct
symbol: AAPL
name: APPLE COMPUTER
price: 60.76
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: BGEIX
name: AMERICAN CENTURY
price: 17.79
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal

Note - when i try a
puts n
puts n.class

in the line just above this one : w.push(stock_struct.new(n[0], n[1],
n[2],date_now, 5, "Domestic Equity","Precious Metal"))
I get
"AAPL"
string
"BGEIX"
string

so I don't see why feeding in strings gives me all this weird stuff ?

PS Ruby 1.82 , Win XP

Thanks in advance

6 Answers

coachhilton

6/9/2006 5:00:00 PM

Are you certain Yahoo is not giving you back quotes strings?

Ken

Charles L. Snyder wrote:
> Hi
>
> Pardon the long post:
> I am playing around with one of the ruby quiz problems (stock
> portfolio) - trying to modify it to do some other things. I am reading
> in data from a csv file, getting the stock symbols from it, and using
> the symbols via Yahoo finance to get data, which I output into a .yaml
> format. Everything is fine, except that I get a screwy yaml output, and
> can't figure out where the extra " and \ are coming from.
>
> Here is the ruby code:
>
> #!/usr/local/bin/ruby
>
> # Stock data via: http://www.gummy-stuff.org/Yaho...
>
> require 'open-uri'
> require 'yaml'
> require 'ostruct'
> require 'csv'
>
> # myfile = 'mystocks_final.yaml' # change this for your file
> data = ["s","n","p","p2"] #symbols to check on yahoo
>
> # stock_struct = Struct::new( "StockStruct", :name, :shares, :symbol,
> :date, :ass_class)
> # get the data from the yaml file
> portfolio = YAML.load( open('my_db4.yaml').read )
>
> stocks = []
> portfolio.each {|r| stocks.push(r.symbol)} # stick the symbols into the
> stock array
> data = data.map { |tag| tag[/\w+/] }.join
>
> # put in a delete from array for those stocks with incorrect or
> unobtainable symbols
> bad_symbols = ["CREF Bond","CREF
> Equity","8401765","CREFAnn","8401463","CREF Global
> Eq","8806422","CASH", "CREF Stock", "CREF Equity IN"]
> stocks.delete_if {|x| bad_symbols.include?(x)}
> stocks = stocks.join("+")
> stock_struct = Struct::new( "StockStruct", :symbol, :name, :price,
> :date, :shares, :ass_class, :ass_subclass)
> w = []
> open "http://finance.yahoo.com/d/quot...{stocks}&f=#{data}" do
> |m|
> m.each do |n|
> date_now = Time.now.localtime.strftime("%Y-%m-%d")
> n = n.split(/,/)
> w.push(stock_struct.new(n[0], n[1], n[2],date_now, 5, "Domestic
> Equity","Precious Metal"))
> end
> end
>
> open("my_yaml_output.yaml","w"){|e| e << w.to_yaml}
>
>
> Here is a sample of the my_yaml_output file:
>
> ---
> - !ruby/struct:StockStruct
> symbol: "'\"AAPL\"'"
> name: "\"APPLE COMPUTER\""
> price: "60.76"
> date: "2006-06-09"
> shares: 5
> ass_class: Domestic Equity
> ass_subclass: Precious Metal
> - !ruby/struct:StockStruct
> symbol: "'\"BGEIX\"'"
> name: "\"AMERICAN CENTURY \""
> price: "17.79"
> date: "2006-06-09"
> shares: 5
> ass_class: Domestic Equity
> ass_subclass: Precious Metal
>
>
> what I want is: (get rid of all the extra " and \)
>
> ---
> - !ruby/struct:StockStruct
> symbol: AAPL
> name: APPLE COMPUTER
> price: 60.76
> date: "2006-06-09"
> shares: 5
> ass_class: Domestic Equity
> ass_subclass: Precious Metal
> - !ruby/struct:StockStruct
> symbol: BGEIX
> name: AMERICAN CENTURY
> price: 17.79
> date: "2006-06-09"
> shares: 5
> ass_class: Domestic Equity
> ass_subclass: Precious Metal
>
>
> Note - when i try a
> puts n
> puts n.class
>
> in the line just above this one : w.push(stock_struct.new(n[0], n[1],
> n[2],date_now, 5, "Domestic Equity","Precious Metal"))
> I get
> "AAPL"
> string
> "BGEIX"
> string
>
> so I don't see why feeding in strings gives me all this weird stuff ?
>
> PS Ruby 1.82 , Win XP
>
> Thanks in advance

Charles L. Snyder

6/9/2006 5:55:00 PM

coachhilton@gmail.com wrote:
> Are you certain Yahoo is not giving you back quotes strings?

I don't think that is the problem, because I tried this:

> > Note - when i try a
> > puts n
> > puts n.class
> >
> > in the line just above this one : w.push(stock_struct.new(n[0], n[1],
> > n[2],date_now, 5, "Domestic Equity","Precious Metal"))
> > I get
> > "AAPL"
> > string
> > "BGEIX"
> > string

Ross Bamford

6/9/2006 6:19:00 PM

On Fri, 09 Jun 2006 18:54:44 +0100, Charles L. Snyder <clsnyder@gmail.com>
wrote:

>
> coachhilton@gmail.com wrote:
>> Are you certain Yahoo is not giving you back quotes strings?
>
> I don't think that is the problem, because I tried this:
>
>> > Note - when i try a
>> > puts n
>> > puts n.class
>> >
>> > in the line just above this one : w.push(stock_struct.new(n[0], n[1],
>> > n[2],date_now, 5, "Domestic Equity","Precious Metal"))
>> > I get
>> > "AAPL"
>> > string
>> > "BGEIX"
>> > string
>

But those strings *do* contain quotes - puts translates \ stuff for
output. If you did p n rather than puts n, You'd see:

"\"AAPL\""
string
"\"BGEIX\""
string

(p uses inspect, showing the actual form of the string). The backslashed
quote here is the quote that becomes visible in your puts output.

One easy way I can see to fix this, since you're already requiring CSV in
your script, is to change this line:

n = n.split(/,/)

to this:

n = CSV.parse_line(n)

and let CSV worry about correctly parsing out the quoted fields.

--
Ross Bamford - rosco@roscopeco.remove.co.uk

Charles L. Snyder

6/9/2006 9:48:00 PM

> (p uses inspect, showing the actual form of the string). The backslashed
> quote here is the quote that becomes visible in your puts output.
>
> One easy way I can see to fix this, since you're already requiring CSV in
> your script, is to change this line:
>
> n = n.split(/,/)
>
> to this:
>
> n = CSV.parse_line(n)
>
> and let CSV worry about correctly parsing out the quoted fields.
>
> --
> Ross Bamford - rosco@roscopeco.remove.co.uk

Thanks -

I tried that, Ross - now my_yaml_output.yaml looks like this:

---
- !ruby/struct:StockStruct
symbol: !str:CSV::Cell AAPL
name: !str:CSV::Cell APPLE COMPUTER
price: !str:CSV::Cell 60.76
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal
- !ruby/struct:StockStruct
symbol: !str:CSV::Cell BGEIX
name: !str:CSV::Cell "AMERICAN CENTURY "
price: !str:CSV::Cell 17.79
date: "2006-06-09"
shares: 5
ass_class: Domestic Equity
ass_subclass: Precious Metal

and when a place

p n[0]

in the same spot, I get

"AAPL"
"BGEIX"

- good old normal looking strings...
(this time with ruby 1.84 on another win xp machine)

? -v puzzled

Thanks for your suggestions !

Ross Bamford

6/10/2006 6:48:00 AM

On Fri, 09 Jun 2006 22:48:21 +0100, Charles L. Snyder <clsnyder@gmail.com>
wrote:

>> One easy way I can see to fix this, since you're already requiring CSV
>> in
>> your script, is to change this line:
>>
>> n = n.split(/,/)
>>
>> to this:
>>
>> n = CSV.parse_line(n)
>>
>> and let CSV worry about correctly parsing out the quoted fields.
>>
> I tried that, Ross - now my_yaml_output.yaml looks like this:
>
> ---
> - !ruby/struct:StockStruct
> symbol: !str:CSV::Cell AAPL
> name: !str:CSV::Cell APPLE COMPUTER
> price: !str:CSV::Cell 60.76
> [...snipped...]
>
> and when a place
>
> p n[0]
>
> in the same spot, I get
>
> "AAPL"
> "BGEIX"
>
> - good old normal looking strings...
> (this time with ruby 1.84 on another win xp machine)
>
> ? -v puzzled
>

This is just because CSV uses it's own class (CVS::Cell instances) to
represent cells in the parsed string. These mostly act as the data they
hold but if you want to force it to use strings, you could just map the
output:

n = CSV.parse_line(n).map! { |s| s.to_s }

Alternatively, looking at the data Yahoo sends over I guess you should be
okay with a bit of regex foo, e.g:

n = n.split(/,/).map! { |s| s.gsub(/^['"]?(.*?)['"]?$/) { $1 } }

(only lightly tested).

--
Ross Bamford - rosco@roscopeco.remove.co.uk

Charles L. Snyder

6/10/2006 2:10:00 PM

Both of these suggestions work perfectly.

Thanks for your help !!!

comp.lang.ruby

yaml format

Charles L. Snyder

coachhilton

Charles L. Snyder

Ross Bamford

Charles L. Snyder

Ross Bamford

Charles L. Snyder

x Login to ForumsZone