[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Automatically finding gaps in unit test coverage ?

Tomasz Wegrzanowski

8/4/2006 4:12:00 PM

Hello,

Is it possible to somehow automatically check which parts of code
are not covered by unit tests ?

--
Tomasz Wegrzanowski [ http://t-a-w.blo... ]
7 Answers

Logan Capaldo

8/4/2006 4:32:00 PM

0


On Aug 4, 2006, at 12:12 PM, Tomasz Wegrzanowski wrote:

> Hello,
>
> Is it possible to somehow automatically check which parts of code
> are not covered by unit tests ?
>
> --
> Tomasz Wegrzanowski [ http://t-a-w.blo... ]
>

Check out rcov. This is exactly what rcov was created for. It makes
these awesome HTML coverage reports.

Robert Klemme

8/4/2006 4:35:00 PM

0

Tomasz Wegrzanowski wrote:
> Hello,
>
> Is it possible to somehow automatically check which parts of code
> are not covered by unit tests ?

I don't know of any mechanism built into RubyUnit or a code coverage
tool. But this is something you could do: use set_trace_func to trace
all methods invoked, then at the end of a single class's test case or
when all tests are done go through your classes and check their
#instance_methods to verify that all of them were called. But that
still does not give you info whether you had a proper test case for the
method or the method was called by some other method etc. Maybe you can
extend the tracing to record only methods called from a unit test.
Still you would not know whether the method was simply called or whether
return values were checked or whether they were checked properly.

Kind regards

robert

Tomasz Wegrzanowski

8/4/2006 6:41:00 PM

0

On 8/4/06, Logan Capaldo <logancapaldo@gmail.com> wrote:
> Check out rcov. This is exactly what rcov was created for. It makes
> these awesome HTML coverage reports.

Thanks, it was exactly what I needed. :-)

By the way how are people usually testing the convenience functions
that print something to STDOUT.

--
Tomasz Wegrzanowski [ http://t-a-w.blo... ]

Tomasz Wegrzanowski

8/4/2006 6:51:00 PM

0

On 8/4/06, Robert Klemme <shortcutter@googlemail.com> wrote:
> Tomasz Wegrzanowski wrote:
> > Hello,
> >
> > Is it possible to somehow automatically check which parts of code
> > are not covered by unit tests ?
>
> I don't know of any mechanism built into RubyUnit or a code coverage
> tool. But this is something you could do: use set_trace_func to trace
> all methods invoked, then at the end of a single class's test case or
> when all tests are done go through your classes and check their
> #instance_methods to verify that all of them were called. But that
> still does not give you info whether you had a proper test case for the
> method or the method was called by some other method etc. Maybe you can
> extend the tracing to record only methods called from a unit test.
> Still you would not know whether the method was simply called or whether
> return values were checked or whether they were checked properly.

Well, one cannot have everything :-). rcov is pretty useful for finding
missing tests. My only problem with it so far is its inability to test
coverage of metaprogrammed code, but I guess that can't be helped.

--
Tomasz Wegrzanowski [ http://t-a-w.blo... ]

James Gray

8/4/2006 7:20:00 PM

0

On Aug 4, 2006, at 1:40 PM, Tomasz Wegrzanowski wrote:

> By the way how are people usually testing the convenience functions
> that print something to STDOUT.

Use the standard StringIO library for this. You can swap out the
STDOUT object for the test and replace it when you are done but I
prefer to modify the method to be more testable:

def meth_to_test(io = STDOUT)
# use io here...
end

Then you can just pass the StringIO object when testing.

James Edward Gray II

Mauricio Fernández

8/4/2006 8:54:00 PM

0

On Sat, Aug 05, 2006 at 03:50:35AM +0900, Tomasz Wegrzanowski wrote:
> On 8/4/06, Robert Klemme <shortcutter@googlemail.com> wrote:
> >I don't know of any mechanism built into RubyUnit or a code coverage
> >tool. But this is something you could do: use set_trace_func to trace
> >all methods invoked, then at the end of a single class's test case or
> >when all tests are done go through your classes and check their
> >#instance_methods to verify that all of them were called. But that
> >still does not give you info whether you had a proper test case for the
> >method or the method was called by some other method etc. Maybe you can
> >extend the tracing to record only methods called from a unit test.
[...]

You can do that in rcov with the --test-unit-only option.

> Well, one cannot have everything :-). rcov is pretty useful for finding
> missing tests. My only problem with it so far is its inability to test
> coverage of metaprogrammed code, but I guess that can't be helped.

Are you referring to this?

$ rcov --no-html --text-coverage --no-color a.rb
================================================================================
a.rb
================================================================================

module StupidMeta
def ugly_make_meth(meth)
module_eval <<-EOF, __FILE__, __LINE__+1
def #{meth}(a,b)
a + #{meth.to_s.inspect} + b
end
EOF
end

def make_meth(meth)
define_method(meth) do |a,b|
!! a + meth.to_s + b
!! end
end

end

class X
extend StupidMeta
ugly_make_meth :foo
make_meth :bar
end

# we don't actually call them, so they should both be uncovered
#x = X.new
#x.foo("method ", " here")
#x.bar("method ", " here")


As you can see, Module#define_method will work better, if you can use it at
all (that is, if you don't need a block argument, under 1.8, and don't care
about the closure and the speed hit). Another advantage is that it will tell
you if the code is well formed (syntactically correct) much earlier.

The problem with the string forms of eval is that I've had to go to great
lengths to make sure rcov marks heredocs correctly, and the above annotation
*is* correct.

Indeed, in the above snippet the heredoc in ugly_make_meth has been
"executed", in some sense (this interpretation gains some weight due to the
interpolation going on; there's actually something being evaluated there).

But there are two interpretations for that string: as a mere heredoc, and as
executable code that defines another method. There are thus also two ways to
think about it as far as coverage is concerned.

I'm thinking that maybe some sort of pragma would make sense:

def ugly_make_meth(meth)
module_eval <<-EOF, __FILE__, __LINE+1__ # rcov:noheredoc
def #{meth}(a,b)
a + #{meth.to_s.inspect} + b
end
EOF
end


As for other limitations, I think I know how to handle generated code that
cannot be meaningfully associated to your sources, but I feel it's not worth
the effort.

Is there anything else you'd like to see addressed?

--
Mauricio Fernandez - http://eige... - singular Ruby

Robert Klemme

8/5/2006 8:40:00 AM

0

Mauricio Fernandez wrote:
> You can do that in rcov with the --test-unit-only option.

[snip]

Mauricio, thanks for the detailed explanation! Learn something new
every day...

Kind regards

robert