Florian Gross
12/21/2004 6:13:00 PM
Stephen Kellett wrote:
> In message <32qvlvF3p991aU1@individual.net>, Florian Gross
> <flgr@ccan.de> writes
>
>>> Ruby Coverage Validator graphical, stats updated in real time as the
>>> app executes - enabling you to direct your testing sequence to
>>> ensure maximum coverage if you are running interactively rather than
>>> as a regression test. You can run merge results from one run into
>>> another, or a central session - ideal for regression testing. You can
>>> export the results in HTML or XML. Multiple views onto the same data,
>>> etc.
>>
>> That sounds like it could be useful, though I'm not 100% sure how it
>> would be used in practice.
>
> Well, our C++/Java/Python customers seem to find it useful. If you were
> performing a unit test, you'd probably find it useful to see which lines
> a particular test executed. If you had all your tests running off a GUI
> and could run each one on demand, you could view the code coverage
> results as you executed each command. If the commands were separate Ruby
> scripts you could view the results in the central session merged into
> the current session (automatically if you choose) at the end of each
> script run.
>
> The HTML/XML export allow you to run your regression test suite
> overnight and come in to work the next day with a session ready to use
> (in the GUI) and also HTML or XML results for management. We supply XML
> as well as HTML in case people want to build a custom report for
> management rather than use the HTML report.
>
> One of our customers (for the C++ product) went from spending 33 hours
> to perform their regression test with a competing product to 8.5 hours
> our C++ version of Coverage Validator. Ruby Coverage Validator uses a
> similar GUI and data collection framework with custom bits for Ruby.
>
> We've added various data export and data visualisation options as people
> ask for them.
Ah, that's very interesting points. I can see how those features are
useful in practice now.
>>>> I especially wonder if you are using trace_funcs (which can be quite
>>>> slow)
>>>
>>> Yes. Although the slowness would be compounded by using a trace func
>>> written in Ruby. Our trace func is written in C++. Matz and a few
>>> helpful people in this newsgroup provided enough information for us
>>> to put things together after examining the source code and writing
>>> quite a few test applications.
>>
>> Interesting -- I've not done this before, but if writing a trace_func
>> in a lower level language speeds it up severely that might be a very
>> important option. Thanks for mentioning this.
>
> I thought it would be obvious :-) - I can't speak for Ruby users, but
> the Python people routinely move more expensive operations over to
> C/C++. I'd expect the same to be true for Ruby.
Sure, we do that, too. But I always felt that trace_funcs were too slow
to have them implemented permanently. That might not be true when
writing them in C, however. I'll have a look at that.