Samuel Williams
11/27/2007 2:20:00 AM
> This is known, and well discussed and misunderstood in many, many,
> many places (I'm talking well outside of ruby here). You can also
> create this problem in C. An ascii formatted number is *not* going to
> be the same as the machine representation. It's sometimes a reason
> people use a binary format.
>
> In any case, you confusion is understandable - but there isn't
> anything inconsistent or problematic here. It's just a rehashing of
> the standard floating point representation.
>
> Cameron
I understand this is not a problem for just Ruby, but in this case the
Ruby behaviour is not as obvious as Python. I am going to guess that
it is IRB doing the formatting, so it may be something that can be
changed in IRB.
When you are debugging something, it is important that you see the
actual data. For example, I can forgive puts for writing "-14014.0",
but I can't forgive IRB for printing that as the value of a variable.
Even in GDB, we will not get this kind of result
--- test.cpp ---
#include <iostream>
int main (int argc, char ** argv) {
double v = (-140.14 * 100.0);
std::cout << v << std::endl;
}
(gdb) break test.cpp:7
Breakpoint 1 at 0x1da8: file test.cpp, line 7.
(gdb) run
Starting program: /private/tmp/test
Reading symbols for shared libraries +++. done
Breakpoint 1, main (argc=1, argv=0xbffff9dc) at test.cpp:7
7 std::cout << v << std::endl;
(gdb) p v
$1 = -14013.999999999998 <=== This line is like "=> v"
(gdb) step
-14014 <=== This line is like "=> puts v"
8 }
(gdb)
So does this make clear what my issue is with this kind of formatting?
For example, in this situation, the debugger correctly shows me the
value, this is what I would expect:
>> (-140.14 * 100)
=> -14013.999999999998 <=== Great!
We know the number isn't -14014.0 - which is mathematically incorrect
- this is the kind of information we need to know in a debugger.
>> puts (-140.14 * 100)
=> -14014.0 <=== Acceptable
Having .0 on the output is not really a good default. The output
should be -14014 in this case, without any trailing .0 - as it stands,
this indicates that that number is accurate to 1dp. If numbers are
going to be formatted and rounded by default, best to do it correctly,
right?
Obviously, string representation is not accurate, I'm not stupid
enough to dispute that! However, I think it is important these things
are done consistently for the benefit of the programmer. Both GDB and
Python, and many other languages are consistently different from Ruby
in this respect.
I understand that you are trying to tell me that the number is the
same - I'm not arguing that, what I am arguing is that the way this is
revealed to the programmer is a problem.
Regards,
Samuel