[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

ruby float calculations

Alexandru Toma

3/9/2006 4:58:00 PM

Could someone please explain to me why the following happens:

irb(main):001:0> 0.29 - 0.38 + 0.1
=> 0.00999999999999998
irb(main):002:0> 0.29 + (-0.38 + 0.1)
=> 0.00999999999999995
irb(main):003:0> 0.29 + 0.1 - 0.38
=> 0.01

How can I prevent something like this from happening, or at the very
least how can I work around such errors?

Many thanks in advance.

--
Posted via http://www.ruby-....
4 Answers

Ara.T.Howard

3/9/2006 5:06:00 PM

0

Cameron McBride

3/9/2006 5:06:00 PM

0

On 3/9/06, Alexandru Toma <flash3001@yahoo.com> wrote:
> Could someone please explain to me why the following happens:
>
> irb(main):001:0> 0.29 - 0.38 + 0.1
> => 0.00999999999999998
> irb(main):002:0> 0.29 + (-0.38 + 0.1)
> => 0.00999999999999995
> irb(main):003:0> 0.29 + 0.1 - 0.38
> => 0.01

Check this out:

d1 = 0.29 - 0.38 + 0.1
d2 = 0.29 + (-0.38 + 0.1)
d3 = 0.29 + 0.1 - 0.38
p (d3 - d2)
p Float::EPSILON
(d3 - d2) < Float::EPSILON #=> true

Computers do not store numbers indefinitely, so there is an "accuracy"
associated with floating point operations. For more information,
check out:
http://en.wikipedia.org/wiki/Floa...

Cameron

Alexandru Toma

3/9/2006 5:42:00 PM

0

> http://en.wikipedia.org/wiki/Floa...

Thanks for your answers. It's clear now. Expecially this link on the IBM
site made it very clear:
http://www2.hursley.ibm.com/decimal/decifaq1.ht...

My question now would be if there is an easy way to implement
floating-point decimal arithmetic in Ruby.

--
Posted via http://www.ruby-....

Alexandru Toma

3/9/2006 5:48:00 PM

0

> My question now would be if there is an easy way to implement
> floating-point decimal arithmetic in Ruby.

Forget about this... it has already been answered... I should have been
more careful

>
> with ruby, at least, you can use BigDecimal or the like - but with a
> serious
> speed penalty. in short this is just the way computers work:

Thanks again


--
Posted via http://www.ruby-....