Asp Forum - unit testing advice

Shadowfirebird

8/1/2008 9:32:00 PM

Forgive me if this is a stupid question.

(Actually, I know it's a stupid question -- they're the only ones
worth asking...)

* How do you unit test a method who's job is to interface with the
outside world? For example, a method that outputs to a file? *

I've done a lot of coding, but in a dinosaur language -- automated
unit tests are completely new to me. I understand how to use
unit/test; but it seems to me that that's only half the story. I need
some suggestions on how to design my code in such a way as it can be
tested. Suggestions, anyone?

Shadowfirebird

57 Answers

Gregory Brown

8/1/2008 9:37:00 PM

On Fri, Aug 1, 2008 at 5:31 PM, Shadowfirebird <shadowfirebird@gmail.com> wrote:
> Forgive me if this is a stupid question.
>
> (Actually, I know it's a stupid question -- they're the only ones
> worth asking...)
>
> * How do you unit test a method who's job is to interface with the
> outside world? For example, a method that outputs to a file? *
>
> I've done a lot of coding, but in a dinosaur language -- automated
> unit tests are completely new to me. I understand how to use
> unit/test; but it seems to me that that's only half the story. I need
> some suggestions on how to design my code in such a way as it can be
> tested. Suggestions, anyone?

Usually, mock objects are best for this. There exist some special
purpose ones, but it is relatively straightforward to put one together
using something like Mocha or Flexmock, both available on RubyForge.
If you are using RSpec, support for mock objects is built in.

-greg

--
Killer Ruby PDF Generation named after a magnificent sea creature:
http://github.com/sa... | Non-tech stuff at:
http://metametta.bl...

Shadowfirebird

8/1/2008 10:04:00 PM

Many thanks. Clearly I'm going to have to go back to research mode on
this one.

I've found this lovely bit of example code for mocha, though -- see
below. If I understand you correctly, you seem to be saying that you
*don't* test the output routine; you use a mock to fake it so that you
can test the rest of the code?

That would imply that you farm out the difficult bits of the output
routine into other methods in the same way that you would with
functional programming -- ideally, leaving the output routine as
something that is so simple it doesn't need testing?

class Enterprise
def initialize(dilithium); @dilithium = dilithium; end

def go(warp_factor); warp_factor.times { @dilithium.nuke(:anti_matter) }; end
end

require 'test/unit'
require 'rubygems'
require 'mocha'

class EnterpriseTest < Test::Unit::TestCase

def test_should_boldly_go
dilithium = mock()
dilithium.expects(:nuke).with(:anti_matter).at_least_once #
auto-verified at end of test
enterprise = Enterprise.new(dilithium)
enterprise.go(2)
end

end

On Fri, Aug 1, 2008 at 10:37 PM, Gregory Brown
<gregory.t.brown@gmail.com> wrote:
> On Fri, Aug 1, 2008 at 5:31 PM, Shadowfirebird <shadowfirebird@gmail.com> wrote:
>> Forgive me if this is a stupid question.
>>
>> (Actually, I know it's a stupid question -- they're the only ones
>> worth asking...)
>>
>> * How do you unit test a method who's job is to interface with the
>> outside world? For example, a method that outputs to a file? *
>>
>> I've done a lot of coding, but in a dinosaur language -- automated
>> unit tests are completely new to me. I understand how to use
>> unit/test; but it seems to me that that's only half the story. I need
>> some suggestions on how to design my code in such a way as it can be
>> tested. Suggestions, anyone?
>
> Usually, mock objects are best for this. There exist some special
> purpose ones, but it is relatively straightforward to put one together
> using something like Mocha or Flexmock, both available on RubyForge.
> If you are using RSpec, support for mock objects is built in.
>
> -greg
>
> --
> Killer Ruby PDF Generation named after a magnificent sea creature:
> http://github.com/sa... | Non-tech stuff at:
> http://metametta.bl...
>
>

--
Me, I imagine places that I have never seen / The colored lights in
fountains, blue and green / And I imagine places that I will never go
/ Behind these clouds that hang here dark and low
But it's there when I'm holding you / There when I'm sleeping too /
There when there's nothing left of me / Hanging out behind the
burned-out factories / Out of reach but leading me / Into the
beautiful sea

Gregory Brown

8/1/2008 10:20:00 PM

On Fri, Aug 1, 2008 at 6:03 PM, Shadowfirebird <shadowfirebird@gmail.com> wrote:
> Many thanks. Clearly I'm going to have to go back to research mode on
> this one.
>
> I've found this lovely bit of example code for mocha, though -- see
> below. If I understand you correctly, you seem to be saying that you
> *don't* test the output routine; you use a mock to fake it so that you
> can test the rest of the code?

Mocks do check to make sure that certain calls are actually made with
constraints on the way they are called and how they respond.
So if you have a method like load_text_file(name) that appends a .txt
to the end of the filename and then reads the file, like so:

def load_text_file(name)
File.read("#{name}.txt")
end

You'd want to do a mock like:

# Set up the expectation
File.expects(:read).with("foo.txt")
load_text_file "foo"

The point here is that this tests your code without unnecessarily
verifying that File.read() works.

Of course, be sure that your mocks reflect reality when you design
them, as it's possible to build nonsense mocks that lead you astray.
However, as soon as you really try to use your code, you'd notice that
and be able to fix it in your tests...

> That would imply that you farm out the difficult bits of the output
> routine into other methods in the same way that you would with
> functional programming -- ideally, leaving the output routine as
> something that is so simple it doesn't need testing?

Essentially the idea behind mocking is that you replace an external
resource with an object that handles the sorts of messages that form
the interface between your external resource and your code. This
should behave in the same way you'd expect your real resource to
behave given the way that you are using it, so that you can test
things like exception handling and also test your code that wraps and
invokes these resources.

From this point of view, the assumption is that you can rely on thing
like File handles, Database connections, or other external resources
to work as expected, so you don't need to actually test them directly.
What you do need to test is the interaction between your code and
these resources, and for this purpose, a suitable mock object that
verifies these things works great.

And yes, testing this way does encourage you to make your wrappers of
external resources clean and easy to work with, which is a side
benefit.

-greg

--
Killer Ruby PDF Generation named after a magnificent sea creature:
http://github.com/sa... | Non-tech stuff at:
http://metametta.bl...

Robert Dober

8/1/2008 10:29:00 PM

On Sat, Aug 2, 2008 at 12:03 AM, Shadowfirebird
<shadowfirebird@gmail.com> wrote:

> def go(warp_factor); warp_factor.times { @dilithium.nuke(:anti_matter) }; end
I thought that antimatter consumption was exponential to warp speed!
R.

Frederick Cheung

8/1/2008 11:46:00 PM

On 1 Aug 2008, at 23:03, Shadowfirebird wrote:

> Many thanks. Clearly I'm going to have to go back to research mode on
> this one.
>
> I've found this lovely bit of example code for mocha, though -- see
> below. If I understand you correctly, you seem to be saying that you
> *don't* test the output routine; you use a mock to fake it so that you
> can test the rest of the code?
>>

The following article is well worth reading: http://martinfowler.com/articles/mocksArent...

Fred

Phlip

8/2/2008 1:23:00 AM

Shadowfirebird wrote:

> I've found this lovely bit of example code for mocha, though -- see
> below. If I understand you correctly, you seem to be saying that you
> *don't* test the output routine; you use a mock to fake it so that you
> can test the rest of the code?

Consider a test T that calls A, which calls B.

Sometimes, a B is cheap to assemble, so T can assemble B, activate A, and test
that A did its thing. T considers B's behavior just a side-effect.

Most of the time, B should be real. And its state should be static (such as a
Rails "fixture", with static data.) B should not have runaway dependencies. The
test T should be easy to set-up.

You should mock B if it's too expensive to set up. For example, if B reads the
system clock, and if T would prefer the date is 2008 July 31, the test T should
not wait an infinite amount of time, until the cosmos oscillates and 2008 July
31 occurs again. Tests should run as fast as possible, to avoid any hesitation
running them.

You should mock B, so it behaves as if the date is correct. Or you should mock
(in Ruby) Time.now, so all Ruby methods that use the date will read the mocked date.

Other examples of things too expensive to directly test:

- live users
- random numbers
- hardware - networks, robots, tape drives, the clock, etc
- system errors

If your B object is not on the list, you should not mock it. Unit tests work
best when they cross-test everything. The only thing better than a test that
fails because A broke is many tests that all accurately fail because B broke. If
your B is too expensive to assemble, you should refactor it, so it bypasses the
behavior that T and A did not care about.

> class Enterprise
> def initialize(dilithium); @dilithium = dilithium; end
>
> def go(warp_factor); warp_factor.times { @dilithium.nuke(:anti_matter) }; end
> end

Very nice. And note it obeys my list, by mocking both hardware and space-time
distortions.

--
Phlip

Shadowfirebird

8/2/2008 10:36:00 AM

Thanks everyone.

Now I need to go meditate to get my head around the idea of designing
methods to be testable. It's quite a shift.

Shadowfirebird.

On Sat, Aug 2, 2008 at 2:24 AM, Phlip <phlip2005@gmail.com> wrote:
> Shadowfirebird wrote:
>
>> I've found this lovely bit of example code for mocha, though -- see
>> below. If I understand you correctly, you seem to be saying that you
>> *don't* test the output routine; you use a mock to fake it so that you
>> can test the rest of the code?
>
> Consider a test T that calls A, which calls B.
>
> Sometimes, a B is cheap to assemble, so T can assemble B, activate A, and
> test that A did its thing. T considers B's behavior just a side-effect.
>
> Most of the time, B should be real. And its state should be static (such as
> a Rails "fixture", with static data.) B should not have runaway
> dependencies. The test T should be easy to set-up.
>
> You should mock B if it's too expensive to set up. For example, if B reads
> the system clock, and if T would prefer the date is 2008 July 31, the test T
> should not wait an infinite amount of time, until the cosmos oscillates and
> 2008 July 31 occurs again. Tests should run as fast as possible, to avoid
> any hesitation running them.
>
> You should mock B, so it behaves as if the date is correct. Or you should
> mock (in Ruby) Time.now, so all Ruby methods that use the date will read the
> mocked date.
>
> Other examples of things too expensive to directly test:
>
> - live users
> - random numbers
> - hardware - networks, robots, tape drives, the clock, etc
> - system errors
>
> If your B object is not on the list, you should not mock it. Unit tests work
> best when they cross-test everything. The only thing better than a test that
> fails because A broke is many tests that all accurately fail because B
> broke. If your B is too expensive to assemble, you should refactor it, so it
> bypasses the behavior that T and A did not care about.
>
>> class Enterprise
>> def initialize(dilithium); @dilithium = dilithium; end
>>
>> def go(warp_factor); warp_factor.times { @dilithium.nuke(:anti_matter)
>> }; end
>> end
>
> Very nice. And note it obeys my list, by mocking both hardware and
> space-time distortions.
>
> --
> Phlip
>
>

--
Me, I imagine places that I have never seen / The colored lights in
fountains, blue and green / And I imagine places that I will never go
/ Behind these clouds that hang here dark and low
But it's there when I'm holding you / There when I'm sleeping too /
There when there's nothing left of me / Hanging out behind the
burned-out factories / Out of reach but leading me / Into the
beautiful sea

Phlip

8/2/2008 12:17:00 PM

Shadowfirebird wrote:

> Now I need to go meditate to get my head around the idea of designing
> methods to be testable. It's quite a shift.

It's easy if you write the tests first. Get them to fail for the right reason,
then write code to pass them.

--
Phlip

David A. Black

8/2/2008 12:56:00 PM

Hi --

On Sat, 2 Aug 2008, Phlip wrote:

> Shadowfirebird wrote:
>
>> I've found this lovely bit of example code for mocha, though -- see
>> below. If I understand you correctly, you seem to be saying that you
>> *don't* test the output routine; you use a mock to fake it so that you
>> can test the rest of the code?
>
> Consider a test T that calls A, which calls B.
>
> Sometimes, a B is cheap to assemble, so T can assemble B, activate A, and
> test that A did its thing. T considers B's behavior just a side-effect.
>
> Most of the time, B should be real. And its state should be static (such as a
> Rails "fixture", with static data.) B should not have runaway dependencies.
> The test T should be easy to set-up.
>
> You should mock B if it's too expensive to set up.

Mocking is also good for pinpointing exactly what you want to test and
what you don't, even if not mocking wouldn't be that expensive in
resources. For example, there's the classic Rails create method, which
goes like this:

if new record is saved successfully
go do something
else
do something else
end

In this case, you can certainly do a real saving of the object. But if
you want to test *only* the conditional logic, and make sure that the
controller takes the right branch given a record whose answer to "Did
you save?" is always "Yes" or always "No", then you can use a mock
object.

> For example, if B reads the system clock, and if T would prefer the
> date is 2008 July 31, the test T should not wait an infinite amount
> of time, until the cosmos oscillates and 2008 July 31 occurs again.
> Tests should run as fast as possible, to avoid any hesitation
> running them.
>
> You should mock B, so it behaves as if the date is correct. Or you should
> mock (in Ruby) Time.now, so all Ruby methods that use the date will read the
> mocked date.
>
> Other examples of things too expensive to directly test:
>
> - live users
> - random numbers
> - hardware - networks, robots, tape drives, the clock, etc
> - system errors
>
> If your B object is not on the list, you should not mock it.

I wouldn't narrow it down that strictly. It can depend on the purpose
of the test, as well as the profile of the thing you're mocking.

David

--
Rails training from David A. Black and Ruby Power and Light:
* Advancing With Rails August 18-21 Edison, NJ
* Co-taught by D.A. Black and Erik Kastner
See http://www.r... for details and updates!

Phlip

8/2/2008 1:23:00 PM

David A. Black wrote:

>> - live users
>> - random numbers
>> - hardware - networks, robots, tape drives, the clock, etc
>> - system errors
>>
>> If your B object is not on the list, you should not mock it.
>
> I wouldn't narrow it down that strictly. It can depend on the purpose
> of the test, as well as the profile of the thing you're mocking.

You can also avoid mocking the clock by setting a time to 2.minutes.ago, for
example.

(And "hardware" covers "profile". We don't care if B takes a trillion clock
cycles, on a magic CPU that can run them all instantly.)

However, some teams go mock-crazy (even those subjected to high-end
consultants), and mock everything for no reason. Don't do that!

--
Phlip

comp.lang.ruby

unit testing advice

Shadowfirebird

Gregory Brown

Shadowfirebird

Gregory Brown

Robert Dober

Frederick Cheung

Phlip

Shadowfirebird

Phlip

David A. Black

Phlip

x Login to ForumsZone