Asp Forum - Why does a test fail when I predicted it's exception class?

RichardOnRails

12/3/2006 2:26:00 AM

Hi,

I have a test suite that invokes on test set that specifies

assert_raise(ZeroDivisionError, MyCalc.new.calc)

When I run the test suite, instead of "1 test, 0 failures, 0 errors",
I get the following. The code follows this output.

What's up?
Richard

K:\_Projects\Ruby\TestUnitTesting\BasicTest>ruby MyCalcTest.rb
..\BasicCalc.rb:5:in `/': divided by 0 (ZeroDivisionError)
from .\BasicCalc.rb:5:in `calc'
from .\BasicCalc.rb:9
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require'
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'
from ./tc_TestSet.rb:1
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`gem_original_require'
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
`require'
from MyCalcTest.rb:4

BasicCalc.rb
=========
# BasicCalc.rb

class MyCalc
def calc
2/0
end
end

tc_TestSet.rb
==========
require '.\BasicCalc.rb'

class Test1 < Test::Unit::TestCase # The long-winded way
def test_excep
assert_raise(ZeroDivisionError, MyCalc.new.calc)
end
end

MyCalcTest.rb
==========
# Reverse-Polish Evaluation Test

require 'test/unit'
require 'tc_TestSet'

9 Answers

Robert Feldt

12/3/2006 2:52:00 AM

On 12/3/06, Richard <RichardDummyMailbox58407@uscomputergurus.com> wrote:
> Hi,
>
> I have a test suite that invokes on test set that specifies
>
> assert_raise(ZeroDivisionError, MyCalc.new.calc)
>
> When I run the test suite, instead of "1 test, 0 failures, 0 errors",
> I get the following. The code follows this output.
>
> What's up?
>
You are using assert_raise in the wrong way. The call you want to test
must be wrapped in a block so that it's exception can be catched and
compared to the expected one.

Try something like

assert_raise(RuntimeError) {raise ""}

/Robert

Louis J Scoras

12/3/2006 2:53:00 AM

On 12/2/06, Richard <RichardDummyMailbox58407@uscomputergurus.com> wrote:

> tc_TestSet.rb
> ==========
> require '.\BasicCalc.rb'
>
> class Test1 < Test::Unit::TestCase # The long-winded way
> def test_excep
> assert_raise(ZeroDivisionError, MyCalc.new.calc)
> end
> end

You need to run the code that raises the exception in a block.

assert_raise(ZeroDivisionError) { MyCalc.new.calc }

should do the trick.

--
Lou.

RichardOnRails

12/3/2006 4:36:00 AM

Hi Robert & Louis,

You guys were right on! Below is one of my actual test sets, the one
with the exception testing. It is more complicated than the toy I
posted, so it took me a few syntax-errors to get it to work. But then
I got my desired: 17 tests, 17 assertions, 0 failures, 0 errors

Thank you very much for your responses.

Best wishes,
Richard

require '.\RevPolishEvaluator.rb'

class ErrorTests < Test::Unit::TestCase # The succinct way
def test1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
def test2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $ 7 --
*}).eval}; end
def test3_missingdata; assert_raise(ArgumentError) {Polish.new(%w{2
+}).eval}; end
def test4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
end

Eric Hodel

12/3/2006 9:58:00 AM

On Dec 2, 2006, at 20:40 , Richard wrote:
> You guys were right on! Below is one of my actual test sets, the one
> with the exception testing. It is more complicated than the toy I
> posted, so it took me a few syntax-errors to get it to work. But then
> I got my desired: 17 tests, 17 assertions, 0 failures, 0 errors
>
> require '.\RevPolishEvaluator.rb'
>
> class ErrorTests < Test::Unit::TestCase # The succinct way
> def test1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
> def test2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $
> 7 --
> *}).eval}; end
> def test3_missingdata; assert_raise(ArgumentError) {Polish.new(%w{2
> +}).eval}; end
> def test4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
> /}).eval}; end
> end

Better:

require 'test/unit'
require 'rev_polish_evaluator'

class TestPolish < Test::Unit::TestCase

def test_eval_bad_operator
# ...
end

def test_eval_divide_by_zero
# ...
end

def test_eval_no_input
e = assert_raise ArgumentError do Polish.new.eval end
assert_equal 'no arguments supplied', e.message
end

def test_eval_missing_data
# ...
end

end

Written this way your tests:

* follow the test/unit naming scheme (TestBlah where Blah is the
class you're testing)
* be in the same order as the output (alphabetical)
* will work with the -n flag:

$ ruby test_polish.rb -n /eval/

* will work with testrb:

$ testrb .

--
Eric Hodel - drbrain@segment7.net - http://blog.se...

I LIT YOUR GEM ON FIRE!

RichardOnRails

12/3/2006 11:50:00 PM

Hi Eric,

Thanks for weighing in on my question.

> def test_eval_no_input
e = assert_raise ArgumentError do Polish.new.eval end
assert_equal 'no arguments supplied', e.message
end

I like the additions here a lot:
1. Capturing the exception object
2. Checking not only the correct exception type is raised but also
checking the programmed message passed to 'raise'

> def test_eval_bad_operator
1. I take it that to conform to unit-test's expectations, the test
names should begin with test_ and not test1_, test2_ etc.

2. How about me using test_1_Whatever, test_2_SomethingElse, etc,?
Wouldn't that conform equally well and cause any error msgs produced by
Test to be presented in lexicographical order (with fewer than 10
tests; double digits if I wanted up to 99 tests, etc.)?

3. I like to produce tests in one-line format if they'll fit on my
screen reasonably so that I can write and scan them more quickly. Do
see any substantive problem in my continuing to do that?

> $ ruby test_polish.rb -n /eval/

I'm running Ruby_1.8.2-15 over WinXP-Pro/SP2. I tried this command and
got nothing:

=== Command Window ====
K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>ruby
RevPolishEvaluator.rb -n /eval/

K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>
====== end ============

I thought the problem might be that I had no errors, so that there was
nothing for Test to report. So I introduced an erroneous assertion
(expressed in three lines rather than my preferred one-line format)
but still got nothing.

Then I ran this expanded test through SciTE and got (both in one-line
format as well as three-line format):

==== SciTE output ====
>ruby RevPolishEvaluatorTest.rb
Loaded suite RevPolishEvaluatorTest
Started
..............F....
Finished in 0.031 seconds.

1) Failure:
test_5_dummy(ErrorTests) [./tc_TestSet2.rb:12]:
<1> expected but was
<2>.

18 tests, 18 assertions, 1 failures, 0 errors
>Exit code: 1
=== end ===

Do you have any idea why the command with the "-n" switch failed on my
system? My code follows.

Again, thanks for your suggestions. I look forward to your comments.

Best wishes,
Richard

=== RevPolishEvaluator.rb ===
### Code same as posted earlier
=== end ===

=== RevPolishEvaluatorTest.rb ===
# Reverse-Polish Evaluation Test

require 'test/unit'
require 'tc_TestSet1' # <= 13 tests that produced no errors;
file not shown here
require 'tc_TestSet2'
=== end ===

=== tc_TestSet2.rb ===
require '.\RevPolishEvaluator.rb'

class ErrorTests < Test::Unit::TestCase # The succinct way
def test_1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
def test_2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $ 7
-- *}).eval}; end
def test_3_missingdata
e = assert_raise(ArgumentError) {Polish.new(%w{2 +}).eval}
assert_equal('Stack underflow', e.message) #
<= Had to change the msg to confirm to my coding
end
def test_4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
def test_5_dummy
assert_equal(1, 2)
end
end
=== end ===

Eric Hodel wrote:
> On Dec 2, 2006, at 20:40 , Richard wrote:
> > You guys were right on! Below is one of my actual test sets, the one
> > with the exception testing. It is more complicated than the toy I
> > posted, so it took me a few syntax-errors to get it to work. But then
> > I got my desired: 17 tests, 17 assertions, 0 failures, 0 errors
> >
> > require '.\RevPolishEvaluator.rb'
> >
> > class ErrorTests < Test::Unit::TestCase # The succinct way
> > def test1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
> > def test2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $
> > 7 --
> > *}).eval}; end
> > def test3_missingdata; assert_raise(ArgumentError) {Polish.new(%w{2
> > +}).eval}; end
> > def test4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
> > /}).eval}; end
> > end
>
> Better:
>
> require 'test/unit'
> require 'rev_polish_evaluator'
>
> class TestPolish < Test::Unit::TestCase
>
> def test_eval_bad_operator
> # ...
> end
>
> def test_eval_divide_by_zero
> # ...
> end
>
> def test_eval_no_input
> e = assert_raise ArgumentError do Polish.new.eval end
> assert_equal 'no arguments supplied', e.message
> end
>
> def test_eval_missing_data
> # ...
> end
>
> end
>
> Written this way your tests:
>
> * follow the test/unit naming scheme (TestBlah where Blah is the
> class you're testing)
> * be in the same order as the output (alphabetical)
> * will work with the -n flag:
>
> $ ruby test_polish.rb -n /eval/
>
> * will work with testrb:
>
> $ testrb .
>
> --
> Eric Hodel - drbrain@segment7.net - http://blog.se...
>
> I LIT YOUR GEM ON FIRE!

Eric Hodel

12/4/2006 6:25:00 AM

On Dec 3, 2006, at 15:55 , Richard wrote:

>> def test_eval_bad_operator
> 1. I take it that to conform to unit-test's expectations, the test
> names should begin with test_ and not test1_, test2_ etc.

Numbering tests is not typical behavior (I've seen it very rarely).
Using #setup and #teardown its easy to avoid the need to number tests.

The only restrictions test/unit has are that the must start with
'test' and accept 0 arguments.

> 2. How about me using test_1_Whatever, test_2_SomethingElse, etc,?
> Wouldn't that conform equally well and cause any error msgs
> produced by
> Test to be presented in lexicographical order (with fewer than 10
> tests; double digits if I wanted up to 99 tests, etc.)?

I match test methods to implementation methods and test classes to
implementation classes. This allows easy auditing (for example with
ZenTest) or even visually of your test coverage.

def test_foo() end

matches

def foo() end

and if I need to test a couple edge-cases of foo:

def test_foo_empty_foopy() end
def test_foo_no_blah() end

And everything stays in run order.

> 3. I like to produce tests in one-line format if they'll fit on my
> screen reasonably so that I can write and scan them more quickly. Do
> see any substantive problem in my continuing to do that?

I don't care, they're your tests (but I hate ; so I had to get rid of
them.)

I sometimes write:

def some_method() do_the_stuff end

because the ; is ugly, but almost always for test stub objects.

Most people add newlines.

>> $ ruby test_polish.rb -n /eval/
>
> I'm running Ruby_1.8.2-15 over WinXP-Pro/SP2. I tried this command
> and
> got nothing:
>
> === Command Window ====
> K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>ruby
> RevPolishEvaluator.rb -n /eval/
>
> K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>
> ====== end ============
>
> I thought the problem might be that I had no errors, so that there was
> nothing for Test to report. So I introduced an erroneous assertion
> (expressed in three lines rather than my preferred one-line format)
> but still got nothing.

You ran your implementation, not your tests.

> Then I ran this expanded test through SciTE and got (both in one-line
> format as well as three-line format):
>
> ==== SciTE output ====
>> ruby RevPolishEvaluatorTest.rb
> Loaded suite RevPolishEvaluatorTest
> Started
> .............F....
> Finished in 0.031 seconds.

This one ran with -n

> Do you have any idea why the command with the "-n" switch failed on my
> system? My code follows.

Try again running your test file :)

> Again, thanks for your suggestions. I look forward to your comments.

Oh, also you can make your test shorter with setup:

> === tc_TestSet2.rb ===
> require '.\RevPolishEvaluator.rb'
>
> class ErrorTests < Test::Unit::TestCase # The succinct way

def setup() @p = Polish.new end

> def test_1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end

def test_1_noarg() assert_raise ArgumentError do @p.eval end end

(As you can see, I hate punctuation.)

--
Eric Hodel - drbrain@segment7.net - http://blog.se...

I LIT YOUR GEM ON FIRE!

RichardOnRails

12/6/2006 4:42:00 AM

Hi Erik,

Hi Eric,

> Using #setup and #teardown its easy to avoid the need to number tests.

Thanks for mentioning them.

> The only restrictions test/unit has are that the must start with
> 'test' and accept 0 arguments.
Excellent

> I match test methods to implementation methods and test classes to
> implementation classes. This allows easy auditing [snip]
I like that, too.

> And everything stays in run order.
Ah, that's what I didn't know. I thought my numbering might be
required to achieve that.

> I don't care, they're your tests [snip]
Great.

> >> $ ruby test_polish.rb -n /eval/
> > Do you have any idea why the command with the "-n" switch failed on my
> > system? My code follows.
>
> Try again running your test file :)
Woops!

Your answers here solved a problem I posted yesterday about another
messed up test. I have now deleted that post because "I've seen the
light."

Below is a toy testing setup. I stuck with my numbered approach for
the moment because it fit these tests. This setup worked quite nicely.
If you're in the mood to comment on them, I'd be a grateful recipient.
But I think I'm OK on this topic now. Incidentally, I thought about
trying to use lamda expressions to reduce the duplication, but enough
is enough.

Again, thank you for providing such thoughtful comments.

Best wishes,
Richard

SetupTeardown.rb
=============
class Foo
MyConst = "Foo's Constant"
def initialize(n)
@n = n
end
def bar
"I'm Foo#bar with #{MyConst}#{@n}"
end
end

puts Foo.new(1).bar if __FILE__ == $0

TestSetupTeardown.rb
================
# "TestSetupTeardown"

require 'test/unit'
require 'tc_Test1'

tc_Test.rb
========
require './SetupTeardown'
require 'test/unit'
class TestST < Test::Unit::TestCase
def setup
@instance1 = Foo.new(1)
@instance2 = Foo.new(2)
end
def teardown
@instance1 = nil
@instance2 = nil
end
def test1
assert_equal("I'm Foo#bar with Foo's Constant1",
@instance1.bar)
assert_equal("I'm Foo#bar with Foo's Constant2",
@instance2.bar)
end
end

Result (using SciTE)
=====
>ruby TestSetupTeardown.rb
Loaded suite TestSetupTeardown
Started
.
Finished in 0.0 seconds.

1 tests, 2 assertions, 0 failures, 0 errors
>Exit code: 0

Eric Hodel wrote:
> On Dec 3, 2006, at 15:55 , Richard wrote:
>
> >> def test_eval_bad_operator
> > 1. I take it that to conform to unit-test's expectations, the test
> > names should begin with test_ and not test1_, test2_ etc.
>
> Numbering tests is not typical behavior (I've seen it very rarely).
> Using #setup and #teardown its easy to avoid the need to number tests.
>
> The only restrictions test/unit has are that the must start with
> 'test' and accept 0 arguments.
>
> > 2. How about me using test_1_Whatever, test_2_SomethingElse, etc,?
> > Wouldn't that conform equally well and cause any error msgs
> > produced by
> > Test to be presented in lexicographical order (with fewer than 10
> > tests; double digits if I wanted up to 99 tests, etc.)?
>
> I match test methods to implementation methods and test classes to
> implementation classes. This allows easy auditing (for example with
> ZenTest) or even visually of your test coverage.
>
> def test_foo() end
>
> matches
>
> def foo() end
>
> and if I need to test a couple edge-cases of foo:
>
> def test_foo_empty_foopy() end
> def test_foo_no_blah() end
>
> And everything stays in run order.
>
> > 3. I like to produce tests in one-line format if they'll fit on my
> > screen reasonably so that I can write and scan them more quickly. Do
> > see any substantive problem in my continuing to do that?
>
> I don't care, they're your tests (but I hate ; so I had to get rid of
> them.)
>
> I sometimes write:
>
> def some_method() do_the_stuff end
>
> because the ; is ugly, but almost always for test stub objects.
>
> Most people add newlines.
>
> >> $ ruby test_polish.rb -n /eval/
> >
> > I'm running Ruby_1.8.2-15 over WinXP-Pro/SP2. I tried this command
> > and
> > got nothing:
> >
> > === Command Window ====
> > K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>ruby
> > RevPolishEvaluator.rb -n /eval/
> >
> > K:\_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>
> > ====== end ============
> >
> > I thought the problem might be that I had no errors, so that there was
> > nothing for Test to report. So I introduced an erroneous assertion
> > (expressed in three lines rather than my preferred one-line format)
> > but still got nothing.
>
> You ran your implementation, not your tests.
>
> > Then I ran this expanded test through SciTE and got (both in one-line
> > format as well as three-line format):
> >
> > ==== SciTE output ====
> >> ruby RevPolishEvaluatorTest.rb
> > Loaded suite RevPolishEvaluatorTest
> > Started
> > .............F....
> > Finished in 0.031 seconds.
>
> This one ran with -n
>
> > Do you have any idea why the command with the "-n" switch failed on my
> > system? My code follows.
>
> Try again running your test file :)
>
> > Again, thanks for your suggestions. I look forward to your comments.
>
> Oh, also you can make your test shorter with setup:
>
> > === tc_TestSet2.rb ===
> > require '.\RevPolishEvaluator.rb'
> >
> > class ErrorTests < Test::Unit::TestCase # The succinct way
>
> def setup() @p = Polish.new end
>
> > def test_1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
>
> def test_1_noarg() assert_raise ArgumentError do @p.eval end end
>
> (As you can see, I hate punctuation.)
>
> --
> Eric Hodel - drbrain@segment7.net - http://blog.se...
>
> I LIT YOUR GEM ON FIRE!

Rob Sanheim

12/6/2006 4:52:00 AM

Hi,

<snip>
> Your answers here solved a problem I posted yesterday about another
> messed up test. I have now deleted that post because "I've seen the
> light."
>
> Below is a toy testing setup. I stuck with my numbered approach for
> the moment because it fit these tests. This setup worked quite nicely.
> If you're in the mood to comment on them, I'd be a grateful recipient.
> But I think I'm OK on this topic now. Incidentally, I thought about
> trying to use lamda expressions to reduce the duplication, but enough
> is enough.

Why do you want to number your tests? Your tests should be written in
such a way that they don't depend upon the order they run, so
numbering them to try and get a specific order is a bad idea. If its
just to get a nice output when you run the suite, you should look at
tweaking the test runner or autotest or whatever you use to run your
tests.

> Again, thank you for providing such thoughtful comments.
>
> Best wishes,
> Richard
>
> SetupTeardown.rb
> =============
> class Foo
> MyConst = "Foo's Constant"
> def initialize(n)
> @n = n
> end
> def bar
> "I'm Foo#bar with #{MyConst}#{@n}"
> end
> end
>
> puts Foo.new(1).bar if __FILE__ == $0
>
> TestSetupTeardown.rb
> ================
> # "TestSetupTeardown"
>
> require 'test/unit'
> require 'tc_Test1'
>
> tc_Test.rb
> ========
> require './SetupTeardown'
> require 'test/unit'
> class TestST < Test::Unit::TestCase
> def setup
> @instance1 = Foo.new(1)
> @instance2 = Foo.new(2)
> end
> def teardown
> @instance1 = nil
> @instance2 = nil

You don't need to set these to nil, as a new instance of the test
class will be create for each test case. You normally only see
teardown used for things like closing a database connection, making
sure a stubbed method gets removed, etc.

> end
> def test1
> assert_equal("I'm Foo#bar with Foo's Constant1",
> @instance1.bar)
> assert_equal("I'm Foo#bar with Foo's Constant2",
> @instance2.bar)
> end
> end

- Rob

RichardOnRails

12/7/2006 3:41:00 AM

Hi Robm,

Thanks for adding your views to this discussion.

> Why do you want to number your tests? Your tests should be written in
> such a way that they don't depend upon the order they run, so
> numbering them to try and get a specific order is a bad idea. If its
> just to get a nice output when you run the suite, you should look at
> tweaking the test runner or autotest or whatever you use to run your
> tests.

I like the numbering so that when I start to develop a slew of tests
for a real app and get a failure(s), I can quickly locate the offending
test in cases where the app is right and the test needs to be
corrected. Maybe when I get some real experience in Test Driven
Development I'll find that's not as helpful as I anticipate it to be.

I don't do it to get the tests to run in any particular order.

> > def teardown
> > @instance1 = nil
> > @instance2 = nil
>
> You don't need to set these to nil, as a new instance of the test
> class will be create for each test case. You normally only see
> teardown used for things like closing a database connection, making
> sure a stubbed method gets removed, etc.
OK.

Again, thanks for responding

Best wishes,
Richard

comp.lang.ruby

Why does a test fail when I predicted it's exception class?

RichardOnRails

Robert Feldt

Louis J Scoras

RichardOnRails

Eric Hodel

RichardOnRails

Eric Hodel

RichardOnRails

Rob Sanheim

RichardOnRails

x Login to ForumsZone