[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Floating point conversion error?

Brandon Hoult

1/23/2008 3:26:00 PM

>> (2241.18 * 100).to_i
=> 224117

Anyone know what is up with that?
--
Posted via http://www.ruby-....
4 Answers

Sebastian Hungerecker

1/23/2008 3:32:00 PM

0

Brandon Hoult wrote:
> >> (2241.18 * 100).to_i
>
> => 224117
>
> Anyone know what is up with that?

Floating point arithmetic isn't entirely accurate.
>> "%.30f" % (2241.18 * 100)
=> "224117.999999999970896169543266296387"

And since 224117.999999999970896169543266296387.to_i is 224117, that's what
you get.

HTH,
Sebastian
--
NP: Immortal - Withstand The Fall Of Time
Jabber: sepp2k@jabber.org
ICQ: 205544826

Thomas Wieczorek

1/23/2008 3:34:00 PM

0

Hello!

On Jan 23, 2008 4:25 PM, Brandon Hoult <bhoult@gmail.com> wrote:
> >> (2241.18 * 100).to_i
> => 224117
>
> Anyone know what is up with that?
>

Floating point numbers are never 100% accurate. See
http://en.wikipedia.org/wiki/Floa... for a detailed
explanation.

Brandon Hoult

1/23/2008 3:34:00 PM

0

Sebastian Hungerecker wrote:
> Brandon Hoult wrote:
>> >> (2241.18 * 100).to_i
>>
>> => 224117
>>
>> Anyone know what is up with that?
>
> Floating point arithmetic isn't entirely accurate.
>>> "%.30f" % (2241.18 * 100)
> => "224117.999999999970896169543266296387"
>
> And since 224117.999999999970896169543266296387.to_i is 224117, that's
> what
> you get.
>
> HTH,
> Sebastian


Thanks... found this after poking around... guess I should have spent
more time looking.

http://www.ruby-...topic/...
--
Posted via http://www.ruby-....

Gary Wright

1/23/2008 3:38:00 PM

0


On Jan 23, 2008, at 10:25 AM, Brandon Hoult wrote:

>>> (2241.18 * 100).to_i
> => 224117
>
> Anyone know what is up with that?

What Every Computer Scientist Should Know About Floating-Point
Arithmetic

<http://docs.sun.com/source/806-3568/ncg_goldber...