Brian Candler
3/19/2007 8:16:00 PM
On Tue, Mar 20, 2007 at 04:40:08AM +0900, Greg Lorriman wrote:
> This
>
> ('40.12'.to_f*100).to_i
>
> results in this integer :
>
> 4011
>
> This is a bit of surprise. I'm using 1.8.4 on windows. Is this a bug?
Only standard numeric rounding error. The problem is that decimal 0.1 cannot
be represented exactly in binary floating point - it's a recurring fraction,
like 1/3 in decimal - so you get only an approximation.
> Is there another floating type that avoids this problem? I;m not
> seeing anything in the docs that might have warned me.
irb(main):002:0> require 'bigdecimal'
=> true
irb(main):003:0> f = BigDecimal.new('40.12')
=> #<BigDecimal:b7cb7548,'0.4012E2',8(8)>
irb(main):004:0> (f*100).to_i
=> 4012
BigDecimal stores decimal numbers as decimal. It's not exactly well
documented though (I only came across it because Ruby on Rails uses it).
Another solution is to multiply all your numbers by 100 at source, i.e.
works in cents rather than dollars or whatever.
Regards,
Brian.