Faisal N Jawdat
2/8/2007 6:59:00 PM
On Feb 6, 2007, at 6:18 PM, Trans wrote:
> The more these test/specification/dbc system evolve the more they seem
> like double-entry bookkeeping. Aren't we just writing the code twice
> but from two differnt perspectives and seeing if they match up?
Given that double-entry bookkeeping is often cited as one of the
subtle advancements that led to global the dominance of the Western
world, I'm not sure this is a bad thing.
But.
"Yes and no". For a single piece of code, it's generally a lot
simpler to test that the code did the right thing than to write the
functional code itself. If you add in edge cases, the problem
multiplies: it's a lot easier to write 10 individual test cases,
each for one edge case, than to write one piece of functional code
that elegantly (or even inelegantly) handles all the edge cases. As
your app grows, this aspect grows by leaps and bounds.
I've been asked something along the lines of "well how do we know
that the test code doesn't have bugs in it?". The answer is that we
don't. However, think of this as a probability problem. The
following are gross oversimplifications, but should get the gist.
- If you have one test case per chunk of code, and that test case
also has a 10% chance of being wrong, and if they're wrong in exactly
the same way, then you have an 81% chance that the code executes
correctly and tests cleanly, an 18% chance that the test fails (9%
false positive, 9% false negative), and a 1% chance that the test
passes incorrectly. Since a failure will likely to look through your
code until you either find what's wrong in your code or your test,
then you've just dramatically decreased the chance of an uncaught bug.
- As code increases the likelyhood of bug-free code decreases
exponentially with the amount of code. However, since tests often
must implicitly test things other than the specific thing they're
testing, the likelihood of bugs remaining uncaught also decreases at
an exponential rate.
There are limits to all of this, and I do think that people are prone
to putting too much faith in automated testing, but it still runs
circles around a lack of it.
-faisal