[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

ruby numbers

djreid

6/9/2006 9:15:00 AM

Hi I'm new to ruby.
Started with learning-ruby.
In first_steps it looks at using ruby as a calculator
>> 1 + 2
=> 3
>> 3 / 2
=> 1
>> 3.0 / 2
=> 1.5

all this I understand BUT

>> 4.1 % 2
=> 0.0999999999999996

Should be .1
and wants more there has only been one decimal place used so the answer
should have only 1 decimal place.

another
>> 3.945 * 2.62
=> 10.3359 4 decimals


Dave
3 Answers

Robert Klemme

6/9/2006 10:03:00 AM

0

djreid@ihug.co.nz wrote:
> Hi I'm new to ruby.
> Started with learning-ruby.
> In first_steps it looks at using ruby as a calculator
>>> 1 + 2
> => 3
>>> 3 / 2
> => 1
>>> 3.0 / 2
> => 1.5
>
> all this I understand BUT
>
>>> 4.1 % 2
> => 0.0999999999999996
>
> Should be .1
> and wants more there has only been one decimal place used so the answer
> should have only 1 decimal place.
>
> another
>>> 3.945 * 2.62
> => 10.3359 4 decimals
>
>
> Dave
>

This are usual numeric errors you see because a float is not represented
as a decimal number but as a binary internally.

robert

Tadashi Saito

6/16/2006 3:24:00 AM

0

Hi all,

On 9 Jun 2006 02:15:22 -0700
djreid@ihug.co.nz wrote:

> all this I understand BUT
>
> >> 4.1 % 2
> => 0.0999999999999996
>
> Should be .1

Yes, I think so, too. The shortest solution is:

>> require "bigdecimal"
=> true
>> BigDecimal("4.1") % 2
=> #<BigDecimal:125e8,'0.1E0',4(16)>
>> BigDecimal("4.1") % 2 == BigDecimal("0.1")
=> true

--
Tadashi Saito

Chris Hulan

6/16/2006 6:51:00 PM

0


> >>> 4.1 % 2
> > => 0.0999999999999996
> >
> > Should be .1

An interesting thing I noticed is that:
0.1 % 2
=> 0.1

I would think it would have the same issue? Unless % is doing
something different
when the left value is les than the right? Have to look at the code if
I get a chance...