Asp Forum - [ANN] rcov 0.2.0 - code coverage tool for Ruby

Mauricio Fernández

2/28/2006 4:31:00 PM

Source code, additional information, screenshots... available at

http://eige.../hi...

Overview
--------

rcov is a tool for simple code coverage analysis in Ruby. It improves on
the better known coverage[1] in a number of ways:
* 20-300 times faster: typically, the program being inspect runs only ~3
times slower than without rcov (i.e. not 200 times slower as with
some other tools)
* more convenient interface
* prettier output (since 0.2.0)
* more accurate coverage information through code linkage inference using
simple heuristics
* additional features like bogo-profile mode, colorblind-friendliness...

Code coverage shouldn't be abused (in few words, C0 coverage guarantees
nothing) but it's still useful for testing: it will at least tell you when
your tests need more work, and most importantly where.

What's new in 0.2.0
-------------------
A change summary is available at
http://eige.../hi...+0.2.0

Visible changes since 0.1.0:
* improved HTML output (cleaner and more readable) --- using xx under the hood
* textual output: execution counts or code decorated with ANSI sequences
* more convenient interface: run several scripts/tests in one go, argument
propagation...

Downloading
-----------

The last version is available at
http://eige.../hi...

How do I use it? What does it look like?
----------------------------------------

Just use rcov to run your program (instead of ruby), and a number of XHTML
files with the code coverage information will be generated, e.g.

rcov test/*.rb

will execute all the .rb files under test/ and generate the code coverage report
under coverage/. rcov can also operate in "bogo-profiling mode"
and output the relevant information in alternative formats (for the time
being, plain text with execution count annotations and colored text). Use
rcov -h
for more information.

Sample output
-------------

See http://eige.../hi... (once again) for screenshots.
The (undecorated) textual output with execution count information looks like this:

$ rcov --no-html -t b.rb
================================================================================
./b.rb
================================================================================
| 2
a, b, c = (1..3).to_a | 2
10.times do | 1
a += 1 | 10
20.times do |i| | 10
b += i | 200
b.times do | 200
c += (j = (b-a).abs) > 0 ? j : 0 | 738800
end | 0
end | 0
end | 0

License
-------
rcov is released under the terms of Ruby's license.
rcov includes xx 0.1.0, which is subject to the following conditions:

ePark Labs Public License version 1
Copyright (c) 2005, ePark Labs, Inc. and contributors
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of ePark Labs nor the names of its contributors may be
used to endorse or promote products derived from this software without
specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

[1] http://freshmeat.net/projects/ru...
--
Mauricio Fernandez - http://eige... - non-trivial Ruby

13 Answers

Robert Feldt

3/1/2006 1:43:00 AM

Thanks Mauricio, this looks great and arrives just at the right momen
when the students in my testing course goes into structural testing of
their programs. I'll point them to this!

Keep up the good work!

Thanks,

Robert Feldt

Simon Strandgaard

3/1/2006 1:47:00 AM

On 2/28/06, Mauricio Fernandez <mfp@acm.org> wrote:
> Source code, additional information, screenshots... available at
>
> http://eigenclass.org/hi...
>
[snip]

very nice :-)

--
Simon Strandgaard

Shashank Date

3/1/2006 1:56:00 AM

Hi,

Mauricio Fernandez <mfp@acm.org> wrote:
Source code, additional information, screenshots... available at

http://eigenclass.org/hi...

On Windows XP (HOME SP2) using MSVC ++ 6.0 and latest one-click installer:

C:\extract\_ruby\rcov>ruby setup.rb
---> bin
<--- bin
---> ext
---> ext/rcovrt
C:/ruby/bin/ruby.exe C:/extract/_ruby/rcov/ext/rcovrt/extconf.rb
creating Makefile
<--- ext/rcovrt
<--- ext
---> bin
updating shebang: rcov
<--- bin
---> ext
---> ext/rcovrt
nmake
Microsoft (R) Program Maintenance Utility Version 6.00.8168.0
Copyright (C) Microsoft Corp 1988-1998. All rights reserved.
cl -nologo -MD -Zi -O2b2xg- -G6 -I. -IC:/ruby/lib/ruby/1.8/i386-mswin32 -IC:/ruby/lib/ruby/1.8/i386-mswin32 -IC:/extract/_ruby/rcov/ext/rcovrt -I. -I./.. -I./../missing -c -Tcrcov.c
rcov.c
rcov.c(21) : error C2146: syntax error : missing ')' before identifier 'event'
rcov.c(21) : error C2061: syntax error : identifier 'event'
rcov.c(21) : error C2059: syntax error : ';'
rcov.c(21) : error C2059: syntax error : ','
rcov.c(22) : error C2059: syntax error : ')'
rcov.c(70) : error C2065: 'coverage_event_hook' : undeclared identifier
rcov.c(71) : error C2065: 'RUBY_EVENT_ALL' : undeclared identifier
rcov.c(71) : error C2065: 'RUBY_EVENT_C_CALL' : undeclared identifier
rcov.c(72) : error C2065: 'RUBY_EVENT_C_RETURN' : undeclared identifier
rcov.c(72) : error C2065: 'RUBY_EVENT_CLASS' : undeclared identifier
NMAKE : fatal error U1077: 'cl' : return code '0x2'
Stop.
setup.rb:655:in `command': system("nmake") failed (RuntimeError)
from setup.rb:664:in `make'
from setup.rb:1258:in `setup_dir_ext'
from setup.rb:1532:in `__send__'
from setup.rb:1532:in `traverse'
from setup.rb:1530:in `dive_into'
from setup.rb:1530:in `traverse'
from setup.rb:1534:in `traverse'
from setup.rb:1533:in `each'
... 7 levels...
from setup.rb:996:in `exec_setup'
from setup.rb:813:in `invoke'
from setup.rb:773:in `invoke'
from setup.rb:1578
C:\extract\_ruby\rcov>

---------------------------------
Yahoo! Mail
Bring photos to life! New PhotoMail makes sharing a breeze.

Meinrad Recheis

3/1/2006 2:12:00 AM

many thanks for rcov, batsman!!
i was just trying it out. what a great tool. i will be using it heavily for
sure!
-- henon

On 2/28/06, Mauricio Fernandez <mfp@acm.org> wrote:
>
> Source code, additional information, screenshots... available at
>
> http://eige.../hi...
>
>
> Overview
> --------
>
> rcov is a tool for simple code coverage analysis in Ruby. It improves on
> the better known coverage[1] in a number of ways:
> * 20-300 times faster: typically, the program being inspect runs only ~3
> times slower than without rcov (i.e. not 200 times slower as with
> some other tools)
> * more convenient interface
> * prettier output (since 0.2.0)
> * more accurate coverage information through code linkage inference using
> simple heuristics
> * additional features like bogo-profile mode, colorblind-friendliness...
>
> Code coverage shouldn't be abused (in few words, C0 coverage guarantees
> nothing) but it's still useful for testing: it will at least tell you when
> your tests need more work, and most importantly where.
>
>
> What's new in 0.2.0
> -------------------
> A change summary is available at
> http://eige.../hi...+0.2.0
>
> Visible changes since 0.1.0:
> * improved HTML output (cleaner and more readable) --- using xx under the
> hood
> * textual output: execution counts or code decorated with ANSI sequences
> * more convenient interface: run several scripts/tests in one go, argument
> propagation...
>
>
> Downloading
> -----------
>
> The last version is available at
> http://eige.../hi...
>
>
> How do I use it? What does it look like?
> ----------------------------------------
>
> Just use rcov to run your program (instead of ruby), and a number of XHTML
> files with the code coverage information will be generated, e.g.
>
> rcov test/*.rb
>
> will execute all the .rb files under test/ and generate the code coverage
> report
> under coverage/. rcov can also operate in "bogo-profiling mode"
> and output the relevant information in alternative formats (for the time
> being, plain text with execution count annotations and colored text). Use
> rcov -h
> for more information.
>
>
> Sample output
> -------------
>
> See http://eige.../hi... (once again) for screenshots.
> The (undecorated) textual output with execution count information looks
> like this:
>
> $ rcov --no-html -t b.rb
>
> ================================================================================
> ./b.rb
>
> ================================================================================
>
> | 2
> a, b, c = (1..3).to_a
> | 2
> 10.times do
> | 1
> a +=
> 1 | 10
> 20.times do |i|
> | 10
> b +=
> i | 200
> b.timesdo | 200
> c += (j = (b-a).abs) > 0 ? j : 0 |
> 738800
> end
> | 0
> end
> | 0
> end
> | 0
>
>
> License
> -------
> rcov is released under the terms of Ruby's license.
> rcov includes xx 0.1.0, which is subject to the following conditions:
>
> ePark Labs Public License version 1
> Copyright (c) 2005, ePark Labs, Inc. and contributors
> All rights reserved.
>
> Redistribution and use in source and binary forms, with or without
> modification,
> are permitted provided that the following conditions are met:
>
> 1. Redistributions of source code must retain the above copyright
> notice, this
> list of conditions and the following disclaimer.
> 2. Redistributions in binary form must reproduce the above copyright
> notice,
> this list of conditions and the following disclaimer in the
> documentation
> and/or other materials provided with the distribution.
> 3. Neither the name of ePark Labs nor the names of its contributors may
> be
> used to endorse or promote products derived from this software
> without
> specific prior written permission.
>
> THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
> IS" AND
> ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> IMPLIED
> WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
> DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
> LIABLE FOR
> ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
> DAMAGES
> (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> SERVICES;
> LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
> AND ON
> ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
> THIS
> SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>
>
> [1] http://freshmeat.net/projects/ru...
> --
> Mauricio Fernandez - http://eige... - non-trivial Ruby
>
>

Shashank Date

3/1/2006 5:17:00 AM

Hi ,

Shashank Date <shanko_date@yahoo.com> wrote:
Mauricio Fernandez wrote:
Source code, additional information, screenshots... available at

http://eigenclass.org/hi...

On Windows XP (HOME SP2) using MSVC ++ 6.0 and latest one-click installer:

I was wrong ... this error only occurs if I use the older version of one-click installer.
Works fine with the latest version.

However, now after successful compile and install, when I do this:

C:\atest>rcov --no-html -t tst_rcov.rb

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb --no-html -t tst_rcovrb
c:/ruby/lib/ruby/1.8/optparse.rb:1445:in `complete': invalid option: --no-html (OptionParser::Invalid
from c:/ruby/lib/ruby/1.8/optparse.rb:1443:in `complete'
from c:/ruby/lib/ruby/1.8/optparse.rb:1278:in `order!'
from c:/ruby/lib/ruby/1.8/optparse.rb:1271:in `order!'
from c:/ruby/lib/ruby/1.8/optparse.rb:1351:in `permute!'
from c:/ruby/lib/ruby/1.8/optparse.rb:1378:in `parse!'
from c:/ruby/bin/rcov.rb:627

C:\atest>rcov -h

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb -h
rcov 0.1.0 2006-01-23
Usage: rcov [options] <script.rb>
Options:
-o, --output PATH Destination directory.
-I, --include PATHS Prepend PATHS to $: (colon separated list)
-n, --no-color Create colorblind-safe output.
-x, --exclude PATTERNS Don't generate info for the files matching any
of the given patterns (comma-separated regexp list)
-p, --profile Generate bogo-profiling info.
-r, --range Color scale range for profiling info (dB).
-h, --help Show this message
--version Show version
C:\atest>

---------------------------------
Relax. Yahoo! Mail virus scanning helps detect nasty viruses!

Shashank Date

3/1/2006 5:28:00 AM

Hi,

It works ... if I use the correct version !

Sorry for the noise ....

And now, I have the Windows binary in case anybody wants to try it out with the _latest_ one click installer.

-- Shashank

Shashank Date <shanko_date@yahoo.com> wrote:
Hi ,

Shashank Date wrote:
Mauricio Fernandez wrote:
Source code, additional information, screenshots... available at

http://eigenclass.org/hi...

On Windows XP (HOME SP2) using MSVC ++ 6.0 and latest one-click installer:

I was wrong ... this error only occurs if I use the older version of one-click installer.
Works fine with the latest version.

However, now after successful compile and install, when I do this:

C:\atest>rcov --no-html -t tst_rcov.rb

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb --no-html -t tst_rcov.rb
c:/ruby/lib/ruby/1.8/optparse.rb:1445:in `complete': invalid option: --no-html (OptionParser::Invalid
from c:/ruby/lib/ruby/1.8/optparse.rb:1443:in `complete'
from c:/ruby/lib/ruby/1.8/optparse.rb:1278:in `order!'
from c:/ruby/lib/ruby/1.8/optparse.rb:1271:in `order!'
from c:/ruby/lib/ruby/1.8/optparse.rb:1351:in `permute!'
from c:/ruby/lib/ruby/1.8/optparse.rb:1378:in `parse!'
from c:/ruby/bin/rcov.rb:627

C:\atest>rcov -h

C:\atest>c:\ruby\bin\ruby.exe c:\ruby\bin\rcov.rb -h
rcov 0.1.0 2006-01-23
Usage: rcov [options]
Options:
-o, --output PATH Destination directory.
-I, --include PATHS Prepend PATHS to $: (colon separated list)
-n, --no-color Create colorblind-safe output.
-x, --exclude PATTERNS Don't generate info for the files matching any
of the given patterns (comma-separated regexp list)
-p, --profile Generate bogo-profiling info.
-r, --range Color scale range for profiling info (dB).
-h, --help Show this message
--version Show version
C:\atest>

---------------------------------
Relax. Yahoo! Mail virus scanning helps detect nasty viruses!

---------------------------------
Yahoo! Mail
Bring photos to life! New PhotoMail makes sharing a breeze.

Ville Mattila

3/1/2006 5:40:00 AM

Hello,

It seems like you are using the old rcov eg. rcov 0.1 instead of 0.2

- Ville

Mauricio Fernández

3/2/2006 10:23:00 PM

On Wed, Mar 01, 2006 at 02:27:37PM +0900, Shashank Date wrote:
> It works ... if I use the correct version !

I think I might be to blame for that; I didn't really think about
win32 issues like whether the executable is associated to the Ruby
interpreter. I'll try to do better next time.

--
Mauricio Fernandez - http://eige... - singular Ruby

David Holroyd

3/14/2006 12:27:00 AM

On Wed, Mar 01, 2006 at 01:31:11AM +0900, Mauricio Fernandez wrote:
> Source code, additional information, screenshots... available at
>
> http://eigenclass.org/hi...

Fabulous!

I would like to get coverage for my test cases too. Could this be
accomodated (without tweaking @ignore_files in the source)?

many, many thanks for this tool,
dave

--
http://david.holr...

Mauricio Fernández

3/14/2006 5:23:00 PM

On Tue, Mar 14, 2006 at 09:27:14AM +0900, David Holroyd wrote:
> On Wed, Mar 01, 2006 at 01:31:11AM +0900, Mauricio Fernandez wrote:
> > Source code, additional information, screenshots... available at
> >
> > http://eigenclass.org/hi...
>
> Fabulous!
>
> I would like to get coverage for my test cases too. Could this be
> accomodated (without tweaking @ignore_files in the source)?

It will be possible in the next release; in the meantime, could you use the
following patch?

--- old-rcov/bin/rcov 2006-03-14 18:12:47.000000000 +0100
+++ new-rcov/bin/rcov 2006-03-14 09:53:07.000000000 +0100
@@ -790,6 +790,11 @@
regexps = list.split(/,/).map{|x| Regexp.new x}
options.skip += regexps
end
+ opts.on("--exclude-only PATTERNS",
+ "Skip info generation only for the files matching the",
+ "given patterns (i.e. discard the default regexp list).") do |list|
+ options.skip = list.split(/,/).map{|x| Regexp.new(x) }
+ end
opts.on("-p", "--profile", "Generate bogo-profiling info.") do
options.profiling = true
options.destdir ||= "profiling"

After applying: (note that in rcov 0.2.0 -n == --no-color is ignored in -T
rich-text mode, so you'll get ANSI-decorated output instead of the following)

$ rcov --exclude-only lib/ruby --no-html -n -T tc_foo.rb
Loaded suite tc_foo
Started
6

comp.lang.ruby

[ANN] rcov 0.2.0 - code coverage tool for Ruby

Mauricio Fernández

Robert Feldt

Simon Strandgaard

Shashank Date

Meinrad Recheis

Shashank Date

Shashank Date

Ville Mattila

Mauricio Fernández

David Holroyd

Mauricio Fernández

x Login to ForumsZone