Asp Forum - [QUIZ] Parsing JSON (#155

James Gray

2/1/2008 1:56:00 PM

The three rules of Ruby Quiz:

1. Please do not post any solutions or spoiler discussion for this quiz until
48 hours have passed from the time on this message.

2. Support Ruby Quiz by submitting ideas as often as you can:

http://www.rub...

3. Enjoy!

Suggestion: A [QUIZ] in the subject of emails about the problem helps everyone
on Ruby Talk follow the discussion. Please reply to the original quiz message,
if you can.

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

There has been a lot of talk recently about parsing with Ruby. We're seeing
some parser generator libraries pop up that make the task that much easier and
they've been stirring up interest.

In honor of that, this week's Ruby Quiz is to write a parser for JSON.

JSON turns out to turns out to be a great little example for writing parsers for
two reasons. First, it's pretty easy stuff. You can hand-roll a JSON parser in
under 100 lines of Ruby. The second advantage is that the data format is
wonderfully documented:

http:/...

Since JSON is just a data format and Ruby supports all of the data types, I vote
we just use Ruby itself as the abstract syntax tree produced by the parse.

Feel free to show off your favorite parser generator, if you don't want to roll
your own. Anything goes.

Here are a few tests to get you started:

require "test/unit"

class TestJSONParser < Test::Unit::TestCase
def setup
@parser = JSONParser.new
end

def test_keyword_parsing
assert_equal(true, @parser.parse("true"))
assert_equal(false, @parser.parse("false"))
assert_equal(nil, @parser.parse("null"))
end

def test_number_parsing
assert_equal(42, @parser.parse("42"))
assert_equal(-13, @parser.parse("-13"))
assert_equal(3.1415, @parser.parse("3.1415"))
assert_equal(-0.01, @parser.parse("-0.01"))

assert_equal(0.2e1, @parser.parse("0.2e1"))
assert_equal(0.2e+1, @parser.parse("0.2e+1"))
assert_equal(0.2e-1, @parser.parse("0.2e-1"))
assert_equal(0.2E1, @parser.parse("0.2e1"))
end

def test_string_parsing
assert_equal(String.new, @parser.parse(%Q{""}))
assert_equal("JSON", @parser.parse(%Q{"JSON"}))

assert_equal( %Q{nested "quotes"},
@parser.parse('"nested \"quotes\""') )
assert_equal("\n", @parser.parse(%Q{"\\n"}))
assert_equal( "a",
@parser.parse(%Q{"\\u#{"%04X" % ?a}"}) )
end

def test_array_parsing
assert_equal(Array.new, @parser.parse(%Q{[]}))
assert_equal( ["JSON", 3.1415, true],
@parser.parse(%Q{["JSON", 3.1415, true]}) )
assert_equal([1, [2, [3]]], @parser.parse(%Q{[1, [2, [3]]]}))
end

def test_object_parsing
assert_equal(Hash.new, @parser.parse(%Q{{}}))
assert_equal( {"JSON" => 3.1415, "data" => true},
@parser.parse(%Q{{"JSON": 3.1415, "data": true}}) )
assert_equal( { "Array" => [1, 2, 3],
"Object" => {"nested" => "objects"} },
@parser.parse(<<-END_OBJECT) )
{"Array": [1, 2, 3], "Object": {"nested": "objects"}}
END_OBJECT
end

def test_parse_errors
assert_raise(RuntimeError) { @parser.parse("{") }
assert_raise(RuntimeError) { @parser.parse(%q{{"key": true false}}) }

assert_raise(RuntimeError) { @parser.parse("[") }
assert_raise(RuntimeError) { @parser.parse("[1,,2]") }

assert_raise(RuntimeError) { @parser.parse(%Q{"}) }
assert_raise(RuntimeError) { @parser.parse(%Q{"\\i"}) }

assert_raise(RuntimeError) { @parser.parse("$1,000") }
assert_raise(RuntimeError) { @parser.parse("1_000") }
assert_raise(RuntimeError) { @parser.parse("1K") }

assert_raise(RuntimeError) { @parser.parse("unknown") }
end
end

84 Answers

Eric Mahurin

2/1/2008 4:09:00 PM

[Note: parts of this message were removed to make it a legal post.]

On Feb 1, 2008 7:55 AM, Ruby Quiz <james@grayproductions.net> wrote:

> In honor of that, this week's Ruby Quiz is to write a parser for JSON.
>
> JSON turns out to turns out to be a great little example for writing
> parsers for
> two reasons. First, it's pretty easy stuff. You can hand-roll a JSON
> parser in
> under 100 lines of Ruby. The second advantage is that the data format is
> wonderfully documented:
>
> http:/...
>

I definitely want to find time to do this one. What would be nice to have
is performance benchmark to compare parsers. Maybe just have a little ruby
script that generates a stream of repeatable random (but valid) JSON.

Eric

James Gray

2/1/2008 4:13:00 PM

On Feb 1, 2008, at 10:09 AM, Eric Mahurin wrote:

> On Feb 1, 2008 7:55 AM, Ruby Quiz <james@grayproductions.net> wrote:
>
>> In honor of that, this week's Ruby Quiz is to write a parser for
>> JSON.
>>
>> JSON turns out to turns out to be a great little example for writing
>> parsers for
>> two reasons. First, it's pretty easy stuff. You can hand-roll a
>> JSON
>> parser in
>> under 100 lines of Ruby. The second advantage is that the data
>> format is
>> wonderfully documented:
>>
>> http:/...
>>
>
> I definitely want to find time to do this one. What would be nice
> to have
> is performance benchmark to compare parsers. Maybe just have a
> little ruby
> script that generates a stream of repeatable random (but valid) JSON.

Neat idea.

Just FYI though, I'm probably going to focus more on the parsing in
the summary that the speed.

James Edward Gray II

Eric Mahurin

2/1/2008 4:22:00 PM

[Note: parts of this message were removed to make it a legal post.]

On Feb 1, 2008 10:12 AM, James Gray <james@grayproductions.net> wrote:

> On Feb 1, 2008, at 10:09 AM, Eric Mahurin wrote:
>
> > On Feb 1, 2008 7:55 AM, Ruby Quiz <james@grayproductions.net> wrote:
> >
> >> In honor of that, this week's Ruby Quiz is to write a parser for
> >> JSON.
> >>
> >> JSON turns out to turns out to be a great little example for writing
> >> parsers for
> >> two reasons. First, it's pretty easy stuff. You can hand-roll a
> >> JSON
> >> parser in
> >> under 100 lines of Ruby. The second advantage is that the data
> >> format is
> >> wonderfully documented:
> >>
> >> http:/...
> >>
> >
> > I definitely want to find time to do this one. What would be nice
> > to have
> > is performance benchmark to compare parsers. Maybe just have a
> > little ruby
> > script that generates a stream of repeatable random (but valid) JSON.
>
> Neat idea.
>
> Just FYI though, I'm probably going to focus more on the parsing in
> the summary that the speed.
>
> James Edward Gray II

OK. Once I figure out exactly what JSON is, I'll probably make a random
JSON generator.

Hopefully this will make me do another release of my parser package, which
is long overdue :). At least check-in my local code into CVS.

Trans

2/1/2008 4:24:00 PM

A bit aside, but it seems a good place to plug the thought: JSON is so
close to valid Ruby syntax. It would be great if Ruby could support
the syntax 100%. Then a parse would be as simple as,

data = eval(json)

Or, safety levels withstanding, we could conceive a safe_eval(json).

T.

James Gray

2/1/2008 4:29:00 PM

On Feb 1, 2008, at 10:23 AM, Trans wrote:

> A bit aside, but it seems a good place to plug the thought: JSON is so
> close to valid Ruby syntax. It would be great if Ruby could support
> the syntax 100%.

Conversion is pretty easy and definitely one way to solve this quiz.

James Edward Gray II

Petite Abeille

2/1/2008 4:35:00 PM

On Feb 1, 2008, at 5:23 PM, Trans wrote:

> A bit aside, but it seems a good place to plug the thought: JSON is so
> close to valid Ruby syntax. It would be great if Ruby could support
> the syntax 100%. Then a parse would be as simple as,
>
> data = eval(json)

Brilliant!

But perhaps the other way around: bridge the JSON syntax discrepencies
to valid Ruby syntax, e.g:

eval( to_ruby( json ) )

Cheers,

PA.

ara.t.howard

2/1/2008 5:21:00 PM

On Feb 1, 2008, at 9:28 AM, James Gray wrote:

> Conversion is pretty easy and definitely one way to solve this quiz.

not to mention installing one of the three gem json parsers out
there ;-)

a @ http://codeforp...
--
it is not enough to be compassionate. you must act.
h.h. the 14th dalai lama

James Gray

2/1/2008 5:30:00 PM

On Feb 1, 2008, at 11:20 AM, ara howard wrote:

>
> On Feb 1, 2008, at 9:28 AM, James Gray wrote:
>
>> Conversion is pretty easy and definitely one way to solve this quiz.
>
> not to mention installing one of the three gem json parsers out
> there ;-)

That's so cheating Ara. :D

James Edward Gray II

ara.t.howard

2/1/2008 5:30:00 PM

On Feb 1, 2008, at 9:09 AM, Eric Mahurin wrote:

> Maybe just have a little ruby
> script that generates a stream of repeatable random (but valid) JSON.

cfp2:~ > cat a.rb
require 'rubygems'
require 'json'

def random_json
case rand
when 0 ... 1/3.0
top = Hash.new
add = lambda{|obj| top[obj] = obj}
when 1/3.0 ... 2/3.0
top = Array.new
add = lambda{|obj| top.push obj}
when 2/3.0 .. 1
top = String.new
add = lambda{|obj| top += obj}
end
10.times{ add[rand.to_s] }
top.to_json
end

puts random_json

cfp2:~ > for i in `seq 1 3`;do ruby a.rb ;done
"0.3786779826911330.2475380034343990.7052927081471540.2056530009384740.1367079874315110.6433874613518640.5329060341883540.8932613322492760.9233991888762390.561470121133217
"
{"0.758942077040095":"0.758942077040095","0.740998718448961":"0.740998718448961","0.581975309640819":"0.581975309640819","0.471066491788047":"0.471066491788047","0.150752108985123":"0.150752108985123","0.679712508205116":"0.679712508205116","0.265444532310993":"0.265444532310993","0.43229805237576":"0.43229805237576","0.880407977937905":"0.880407977937905","0.91896885679168":"0.91896885679168"}
["0.140526101058637
","0.647296447390116
","0.419874655921874
","0.67320818546074
","0.847043108967541
","0.479385904117001
","0.378678170026127
","0.707315391952609","0.26064520446906","0.460184583302929"]

a @ http://codeforp...
--
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
h.h. the 14th dalai lama

ThoML

2/1/2008 6:10:00 PM

> A bit aside, but it seems a good place to plug the thought: JSON is so
> close to valid Ruby syntax. It would be great if Ruby could support
> the syntax 100%.

I hoped ruby19 would already support this use of colons as in {"key":
"value"} but unfortunately not.

comp.lang.ruby

[QUIZ] Parsing JSON (#155

James Gray

Eric Mahurin

James Gray

Eric Mahurin

Trans

James Gray

Petite Abeille

ara.t.howard

James Gray

ara.t.howard

ThoML

x Login to ForumsZone