Yossef Mendelssohn
10/23/2007 1:33:00 PM
On Oct 22, 10:14 pm, Gary Wright <gwtm...@mac.com> wrote:
> On Oct 22, 2007, at 10:30 PM, Yossef Mendelssohn wrote:
>
> > Yes, I know the troubles with floating point. I think what I found
> > most interesting was a number that claimed to be 19.76 when it
> > actually wasn't, simply because it's more convenient (?) to display it
> > as such.
>
> Well, the culprit is actually Float#to_s, which has it's own idea of
> how to convert floating point numbers to text. In particular it
> defaults
> to 15 digits of precession and also works hard to strip trailing zero's:
>
> $ irb
> irb(main):001:0> a = 19.76
> => 19.76
> irb(main):002:0> a.to_s
> => "19.76"
> irb(main):003:0> sprintf "%#.15g", a
> => "19.7600000000000"
> irb(main):004:0> sprintf "%#.32g", a
> => "19.760000000000001563194018672220"
> irb(main):005:0>
>
> You can see from this session that 15 digits of precision is not
> enough to 'show' that a isn't really 19.76.
>
> What was surprising to me is that Float#to_s doesn't just use a standard
> sprintf format string:
>
> static VALUE
> flo_to_s(flt)
> VALUE flt;
> {
> char buf[32];
> double value = RFLOAT(flt)->value;
> char *p, *e;
>
> if (isinf(value))
> return rb_str_new2(value < 0 ? "-Infinity" : "Infinity");
> else if(isnan(value))
> return rb_str_new2("NaN");
>
> sprintf(buf, "%#.15g", value); /* ensure to print decimal point */
> if (!(e = strchr(buf, 'e'))) {
> e = buf + strlen(buf);
> }
> if (!ISDIGIT(e[-1])) { /* reformat if ended with decimal point
> (ex 111111111111111.) */
> sprintf(buf, "%#.14e", value);
> if (!(e = strchr(buf, 'e'))) {
> e = buf + strlen(buf);
> }
> }
> p = e;
> while (p[-1]=='0' && ISDIGIT(p[-2]))
> p--;
> memmove(p, e, strlen(e)+1);
> return rb_str_new2(buf);
>
> }
>
> Gary Wright
Thanks for this, Gary. I wasn't expecting that Float#inspect would
call #to_s, and obviously didn't even bother to test anything out with
sprintf.
This also explains another Float niggle, which is why it will display
"Infinity" but trying to use Infinity as a constant doesn't work.
Maybe I should spend more time looking at the Ruby source.
--
-yossef