Asp Forum - [RDOC] Using unit-tests as examples for a documentation

benny

11/25/2004 9:32:00 PM

dear list,

suddenly this idea jumped on my brain:

when using unit tests we got this pretty methods like
assert_equal()
and
assert_match()

in which we are telling what we expect to get back from a method.
For this methods we invent more or less realistical examples for
how we could use the tested method.
We only need to select realistic examples to let them be meaningful and if
they are, they could in fact serve as examples for a documentation as well.

In a similar way we use the "don'ts" - the prohibited usage of our method
as flesh for the
assert_raise()

tests (although I am not sure if negative examples are that useful for a
documentation).

As we all know: in the documentation nothing beats a good example!

Now the idea: Why not use the examples in the unit-tests (at least the
"positive ones") as link to a popup window "examples" in the
HTML-documentation created by rdoc?

For this to work properly we might need a convention for the locations of
the unit-tests (e.g. as given in Progr. RubyV.2, page 159*) and an optional
switch for rdoc to tell it to search for unit-tests and make the examples
for the corresponding methods.

what do you think about it? might that be possible or would there be so much
divergence in the modelling of unit tests that rdoc won't have a chance to
tie that all together?

I see a big advantage coming out of this approach.
Consider you were using unit-tests anyway on a regular basis. then you would
run them at least anytime you make changes to your old code. so you will
have to fix the unit-tests as well if you do some changes that affect the
usage of your method. once changed the automatically build examples would
be up to date if you are running rdoc the next time. (and even an outdated,
"forgotten" documentation might stay useful when there are working examples
coming with it).

Surely that would affect the way we organize unit-test name parameter
therein etc. but that should not be that bad.

regards, benny

* the example given is:

roman
|-----lib/
| |---- roman.rb
| |----- other files....
|----test/
| |-----test_roman.rb
| |----- other tests......

12 Answers

Dave Thomas

11/25/2004 9:41:00 PM

On Nov 25, 2004, at 15:32, benny wrote:

> Now the idea: Why not use the examples in the unit-tests (at least the
> "positive ones") as link to a popup window "examples" in the
> HTML-documentation created by rdoc?

This has been raised before. The problem is that it's hard to work out
which tests test which methods: method naming conventions would help,
but even so you quite often have three test methods testing one
production method.

An alternative might be to embed the UTs in the comment for the method,
and then have TestUnit extract and run them automatically. I personally
quite like that approach, although it would need some special markup
that RDoc doesn't yet support.

Cheers

Dave

benny

11/25/2004 9:59:00 PM

Dave Thomas wrote:

>
> On Nov 25, 2004, at 15:32, benny wrote:
>
>> Now the idea: Why not use the examples in the unit-tests (at least the
>> "positive ones") as link to a popup window "examples" in the
>> HTML-documentation created by rdoc?
>
> This has been raised before.
sorry, I didn't notice that. Don't want to bore the list.

> The problem is that it's hard to work out
> which tests test which methods: method naming conventions would help,
> but even so you quite often have three test methods testing one
> production method.
but that could be made by a really simple convention:
the name of each test method for method "foo" should be

test_foo_individual_name

Anyway this would be an *option*, i.e. test-methods not following that
convention would silently slip away (which might also be a useful feature).

>
> An alternative might be to embed the UTs in the comment for the method,
> and then have TestUnit extract and run them automatically. I personally
> quite like that approach, although it would need some special markup
> that RDoc doesn't yet support.

I don't think it is good to embed the testing methods into the
documentation. Think of packaging your sources: it might be useful to
separate the tests from the code for smaller packages. the user when
generating the documentation would then not have the examples in the docs
but the developer might offer them on the online docs.

also think of a replacement for a set of given classes methods. you might
want to use the same tests on different development branches to ensure the
compatibility. for using rdoc in this case you start it in the
corresponding branch and make a simple symlink named "test" to the
unit-test directories.

regards,
benny

>
>
> Cheers
>
> Dave

Florian Gross

11/25/2004 10:50:00 PM

Dave Thomas wrote:

> On Nov 25, 2004, at 15:32, benny wrote:
>> Now the idea: Why not use the examples in the unit-tests (at least the
>> "positive ones") as link to a popup window "examples" in the
>> HTML-documentation created by rdoc?
>
> This has been raised before. The problem is that it's hard to work out
> which tests test which methods: method naming conventions would help,
> but even so you quite often have three test methods testing one
> production method.
>
> An alternative might be to embed the UTs in the comment for the method,
> and then have TestUnit extract and run them automatically. I personally
> quite like that approach, although it would need some special markup
> that RDoc doesn't yet support.

I'm currently using indentation as a sample code marker in
test-extract.rb and would welcome official tags for this. (Maybe
prefixing each line of code with optional whitespace and "|"?)
Integrating support for this into RDoc might also have other benefits
for output formatting and so on.

Here's a sample of what I am currently doing:

> # Causes a pattern which would otherwise match
> # greedily to match non-greedily.
> #
> # This can only be applied to #count and #optional
> # based constructs.
> #
> # re_greedy = Regexp::English.new do
> # literal("<") + something + literal(">")
> # end
> # re_non_greedy = Regexp::English.new do
> # literal("<") + something.minimal + literal(">")
> # end
> # str = "<html><body>"
> # re_greedy.match(str)[0] # => "<html><body>"
> # re_non_greedy.match(str)[0] # => "<html>"
> #
> # This method will do nothing when applied to a Node
> # which is already minimal:
> #
> # re_minimal = Regexp::English.something.minimal
> # re_minimal.minimal == re_minimal # => true
> #
> # However, it will raise a NameError Exception when
> # you try to call it on a Regexp Node that can't be
> # made non-greedy:
> #
> # re = Regexp::English.literal("foo")
> # re.minimal # raises NameError
> def minimal
> <implementation snipped>
> end

The next version of test/unit will also feature a new syntax for this.
That one will use sample { assert_equal(code, result) } style test
cases directly before the method IIRC.

Dave Thomas

11/26/2004 4:11:00 AM

On Nov 25, 2004, at 16:52, Florian Gross wrote:
> I'm currently using indentation as a sample code marker in
> test-extract.rb and would welcome official tags for this. (Maybe
> prefixing each line of code with optional whitespace and "|"?)
> Integrating support for this into RDoc might also have other benefits
> for output formatting and so on.

I'm not sure what you're asking for here--could you explain at bit
more? (RDoc will already set this indented code as code, won't it?)

Cheers

Dave

Florian Gross

11/26/2004 3:40:00 PM

Dave Thomas wrote:

> I'm not sure what you're asking for here--could you explain at bit more?
> (RDoc will already set this indented code as code, won't it?)

I'm thinking along the lines of a "sample code" icon or something
similar that can be clicked to display or hide the embedded sample code
and possibly also setup code that needs to be run for actually making
the sample code work for the HTML output.

Maybe something like in this simplified example:

class KeyedList
...

# Removes all items from a keyed list.
#
# % aList = KeyedList["foo", "bar", "qux"]
# | aList.clear
# | aList.size # => 0
def clear
size.times { pop }
end
end

Where "%" would by default be hidden and could be uncovered by clicking
the icon. I'm not sure about the symbols so feel free to change them to
something that makes more sense for you.

I think the biggest point would however be formatting samples
consistently so that they can be distinguished easily from other things
that make sense in a whitespace preserved area.

benny

11/26/2004 7:13:00 PM

Florian Gross wrote:

>
>
> Maybe something like in this simplified example:
>
> class KeyedList
> ...
>
> # Removes all items from a keyed list.
> #
> # % aList = KeyedList["foo", "bar", "qux"]
> # | aList.clear
> # | aList.size # => 0
> def clear
> size.times { pop }
> end
> end
could you please explain how that syntax would fit into Test::Unit?

benny

Florian Gross

11/27/2004 1:20:00 AM

benny wrote:

> Florian Gross wrote:
>># Removes all items from a keyed list.
>>#
>># % aList = KeyedList["foo", "bar", "qux"]
>># | aList.clear
>># | aList.size # => 0
> could you please explain how that syntax would fit into Test::Unit?

I have written a tool that scans such samples out of source code and
turns them into test cases.

Dave Thomas

11/27/2004 4:20:00 PM

On Nov 25, 2004, at 15:32, benny wrote:

> Now the idea: Why not use the examples in the unit-tests (at least the
> "positive ones") as link to a popup window "examples" in the
> HTML-documentation created by rdoc?

I've been thinking a bit about this. I'm nervous about imposing my own
rules on how tests are organized and how methods are named: I'd like
RDoc to be as transparent as possible.

So here's another take on it. You can already link from the method to
its test using standard RDoc hyperlinks. How about adding a slight bit
of new linking smarts: if you hyperlink to a class/method where the
class name starts Test and the method name starts test_, RDoc will
hyperlink to it in a different window. That way you can read the
mainline code, and scroll to the appropriate test in the test window by
clicking. You comment block might contain:

# burble burble burble. Burble burble burble. Burble burble burble
# burble burble. See the unit test in TestThingie#test_create

def create
...
end

It'll use a named window, so all test code will share just one
additional window.

Cheers

Dave

Florian Gross

11/27/2004 4:46:00 PM

Dave Thomas wrote:

> On Nov 25, 2004, at 15:32, benny wrote:
>> Now the idea: Why not use the examples in the unit-tests (at least the
>> "positive ones") as link to a popup window "examples" in the
>> HTML-documentation created by rdoc?
>
> I've been thinking a bit about this. I'm nervous about imposing my own
> rules on how tests are organized and how methods are named: I'd like
> RDoc to be as transparent as possible.

Well, it would just be a style suggestion. There's no reason to enforce
tests being inlined into the documentation of methods, but there are use
cases where it makes good sense to do so. (Specification by example
comes to mind.) I'm not sure I understand the point about method names.

> So here's another take on it. You can already link from the method to
> its test using standard RDoc hyperlinks. How about adding a slight bit
> of new linking smarts: if you hyperlink to a class/method where the
> class name starts Test and the method name starts test_, RDoc will
> hyperlink to it in a different window. That way you can read the
> mainline code, and scroll to the appropriate test in the test window by
> clicking. You comment block might contain:
>
> # burble burble burble. Burble burble burble. Burble burble burble
> # burble burble. See the unit test in TestThingie#test_create

I think this is too indirect. I'd really like to see the sample code
inlined in the documentation, but formatted differently so it stands
out. Being able to toggle the visibility of sample setup code would also
be a nice thing, but it isn't as important to me. Of course this is just
my personal opinion and other's are likely to have other thoughts about
this.

Dave Thomas

11/27/2004 4:55:00 PM

On Nov 27, 2004, at 10:47, Florian Gross wrote:

> I think this is too indirect. I'd really like to see the sample code
> inlined in the documentation, but formatted differently so it stands
> out. Being able to toggle the visibility of sample setup code would
> also be a nice thing, but it isn't as important to me. Of course this
> is just my personal opinion and other's are likely to have other
> thoughts about this.

Are you talking about sample code or unit tests here. For sample code,
I don't see why you'd want to hide it--wouldn't it be part of the
normal comment of the method?

Cheers

Dave

comp.lang.ruby

[RDOC] Using unit-tests as examples for a documentation

benny

Dave Thomas

benny

Florian Gross

Dave Thomas

Florian Gross

benny

Florian Gross

Dave Thomas

Florian Gross

Dave Thomas

x Login to ForumsZone