[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Best way to skip tests

Daniel Berger

2/15/2007 7:23:00 PM

Hi all,

What's the general approach folks use for skipping tests? Sometimes I
have some tests that I want to skip based on platform (usually MS
Windows). I saw the 'flunk' method, but that's considered a failed
test. I'm looking for something that doesn't treat it as success or
failure.

I guess I'd like something like this:

class TC_Foo < Test::Unit::TestCase
def test_bar
if RUBY_PLATFORM.match('mswin')
skip('There's no Foo::Bar on MS Windows - skipped')
else
assert_equal(42, Foo::Bar)
end
end
end

And then output that looked like this:

1 tests, 0 assertions, 0 failures, 0 errors, 1 skipped

If there isn't anything like this currently, would patches to add this
be accepted? I'd be happy to work on it.

Thanks,

Dan

PS - Yes, I realize I could wrap the whole test_bar method in an 'if'
clause. I don't want to do that. I want to be explicit.
16 Answers

Alex Young

2/15/2007 7:32:00 PM

0

Daniel Berger wrote:
> Hi all,
>
> What's the general approach folks use for skipping tests? Sometimes I
> have some tests that I want to skip based on platform (usually MS
> Windows). I saw the 'flunk' method, but that's considered a failed
> test. I'm looking for something that doesn't treat it as success or
> failure.
>
If you were to factor the platform-dependent tests out into their own
module which you can conditionally include into the test case, I think
you'd get what you were after.

--
Alex

Daniel Berger

2/15/2007 7:41:00 PM

0

On Feb 15, 12:32 pm, Alex Young <a...@blackkettle.org> wrote:
> Daniel Berger wrote:
> > Hi all,
>
> > What's the general approach folks use for skipping tests? Sometimes I
> > have some tests that I want to skip based on platform (usually MS
> > Windows). I saw the 'flunk' method, but that's considered a failed
> > test. I'm looking for something that doesn't treat it as success or
> > failure.
>
> If you were to factor the platform-dependent tests out into their own
> module which you can conditionally include into the test case, I think
> you'd get what you were after.

It's not a bad idea, but that still wouldn't explicitly indicate to a
user that tests had been skipped - they would merely see fewer tests
run. Plus, it's more work and I'm lazy. :)

Regards,

Dan

James Gray

2/15/2007 7:52:00 PM

0

On Feb 15, 2007, at 1:45 PM, Daniel Berger wrote:

> On Feb 15, 12:32 pm, Alex Young <a...@blackkettle.org> wrote:
>> Daniel Berger wrote:
>>> Hi all,
>>
>>> What's the general approach folks use for skipping tests?
>>> Sometimes I
>>> have some tests that I want to skip based on platform (usually MS
>>> Windows). I saw the 'flunk' method, but that's considered a failed
>>> test. I'm looking for something that doesn't treat it as success or
>>> failure.
>>
>> If you were to factor the platform-dependent tests out into their own
>> module which you can conditionally include into the test case, I
>> think
>> you'd get what you were after.
>
> It's not a bad idea, but that still wouldn't explicitly indicate to a
> user that tests had been skipped - they would merely see fewer tests
> run. Plus, it's more work and I'm lazy. :)

I think it's a much better design though.

For an example of my concerns, what happens if your proposed skip()
is called after a few assertions are run in a test?

James Edward Gray II

Jacob Fugal

2/15/2007 7:58:00 PM

0

On 2/15/07, James Edward Gray II <james@grayproductions.net> wrote:
> On Feb 15, 2007, at 1:45 PM, Daniel Berger wrote:
> > It's not a bad idea, but that still wouldn't explicitly indicate to a
> > user that tests had been skipped - they would merely see fewer tests
> > run. Plus, it's more work and I'm lazy. :)
>
> I think it's a much better design though.
>
> For an example of my concerns, what happens if your proposed skip()
> is called after a few assertions are run in a test?

I agree that the skipping should be done outside of the test method,
not inside. But explicitness in the output would still be valuable.
Maybe something like the following:

class MyTest < Test::Unit::TestCase
def test_foo
...
end

def test_bar
...
end

if Time.now.wday == 0
skip :test_foo, "The frambulator isn't available on Sundays"
end
end

Jacob Fugal

James Gray

2/15/2007 7:59:00 PM

0

On Feb 15, 2007, at 1:25 PM, Daniel Berger wrote:

> I guess I'd like something like this:
>
> class TC_Foo < Test::Unit::TestCase
> def test_bar
> if RUBY_PLATFORM.match('mswin')
> skip('There's no Foo::Bar on MS Windows - skipped')

at_exit { warn "The Windows tests were skipped." }


> else
> assert_equal(42, Foo::Bar)
> end
> end
> end

Just a thought.

James Edward Gray II

Tim Pease

2/15/2007 8:01:00 PM

0

On 2/15/07, James Edward Gray II <james@grayproductions.net> wrote:
> On Feb 15, 2007, at 1:45 PM, Daniel Berger wrote:
>
> > On Feb 15, 12:32 pm, Alex Young <a...@blackkettle.org> wrote:
> >> Daniel Berger wrote:
> >>> Hi all,
> >>
> >>> What's the general approach folks use for skipping tests?
> >>> Sometimes I
> >>> have some tests that I want to skip based on platform (usually MS
> >>> Windows). I saw the 'flunk' method, but that's considered a failed
> >>> test. I'm looking for something that doesn't treat it as success or
> >>> failure.
> >>
> >> If you were to factor the platform-dependent tests out into their own
> >> module which you can conditionally include into the test case, I
> >> think
> >> you'd get what you were after.
> >
> > It's not a bad idea, but that still wouldn't explicitly indicate to a
> > user that tests had been skipped - they would merely see fewer tests
> > run. Plus, it's more work and I'm lazy. :)
>
> I think it's a much better design though.
>
> For an example of my concerns, what happens if your proposed skip()
> is called after a few assertions are run in a test?
>

That would be a design error, and an exception should be raised.

I like the idea of having a skip method that prints to the screen. It
is a way to let the user know that something is not being tested for
one reason or another.

We have a unit test framework for our embedded software code that has
a skip method. We use it all the time when a test can't be run either
because hardware is not available or it's running on the wrong
platform. The caveat is that the skip method has to be the first
assertion in the test -- otherwise an exception is raised.

Future addition to Test::Unit ??

TwP

Tim Pease

2/15/2007 8:04:00 PM

0

On 2/15/07, Jacob Fugal <lukfugl@gmail.com> wrote:
> On 2/15/07, James Edward Gray II <james@grayproductions.net> wrote:
> > On Feb 15, 2007, at 1:45 PM, Daniel Berger wrote:
> > > It's not a bad idea, but that still wouldn't explicitly indicate to a
> > > user that tests had been skipped - they would merely see fewer tests
> > > run. Plus, it's more work and I'm lazy. :)
> >
> > I think it's a much better design though.
> >
> > For an example of my concerns, what happens if your proposed skip()
> > is called after a few assertions are run in a test?
>
> I agree that the skipping should be done outside of the test method,
> not inside. But explicitness in the output would still be valuable.
> Maybe something like the following:
>
> class MyTest < Test::Unit::TestCase
> def test_foo
> ...
> end
>
> def test_bar
> ...
> end
>
> if Time.now.wday == 0
> skip :test_foo, "The frambulator isn't available on Sundays"
> end
> end
>

Now you have test names in two places, and when someone renames
:test_foo to :test_frambulator ?? Whoops!

I like the idea of having a skip assertion inside the test, and it has
to be first assertion used otherwise an exception is raised.

TwP

Ara.T.Howard

2/15/2007 8:05:00 PM

0

James Gray

2/15/2007 8:08:00 PM

0

On Feb 15, 2007, at 2:05 PM, ara.t.howard@noaa.gov wrote:

> On Fri, 16 Feb 2007, Daniel Berger wrote:
>
>> Hi all,
>>
>> What's the general approach folks use for skipping tests?
>
> oh -- i just don't write them...
>
> ;-)

Cracked me up.

James Edward Gray II

Gregory Brown

2/15/2007 8:08:00 PM

0

On 2/15/07, Tim Pease <tim.pease@gmail.com> wrote:

> I like the idea of having a skip method that prints to the screen. It
> is a way to let the user know that something is not being tested for
> one reason or another.

I like this idea too. I usually will print a message to STDERR when
I skip tests, but this gets obscured by the test/unit output and makes
it hard to tell what's going on.