Asp Forum - Unexpected BigDecimal rounding issue

Samuel Lown

5/6/2009 1:02:00 AM

Hi,

Some more not so fun rounding issues.

Using Floats:

>> ((30 / 1.16) * 1.16) == 30
=> true

Cool, so you'd think it would work with BigDecimal:

>> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")) == 30
=> false
>> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")).to_s
=> "30.0000000000000000000000002"

I'm clearly missing something here as I thought BigDecimal was supposed
to fix this type of thing.

Tested in:
ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]
ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0]

Can anyone offer an explanation?

Cheers, sam
--
Posted via http://www.ruby-....

5 Answers

Nobuyoshi Nakada

5/6/2009 3:06:00 AM

Hi,

At Wed, 6 May 2009 10:01:54 +0900,
Samuel Lown wrote in [ruby-talk:335867]:
> >> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")) == 30
> => false
> >> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")).to_s
> => "30.0000000000000000000000002"
>
> I'm clearly missing something here as I thought BigDecimal was supposed
> to fix this type of thing.

BigDecimal is another kind of floating point number, which uses
decimal base instead of binary, but has finite digits.
Therefore, it is impossible to represent exactly a recurring
decimal theoretically.

--
Nobu Nakada

Rick DeNatale

5/6/2009 3:30:00 AM

On Tue, May 5, 2009 at 9:01 PM, Samuel Lown <me@samlown.com> wrote:
> Hi,
>
> Some more not so fun rounding issues.
>
> Using Floats:
>
>>> ((30 / 1.16) * 1.16) == 30
> => true
>
> Cool, so you'd think it would work with BigDecimal:
>
>>> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")) == 30
> => false
>>> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")).to_s
> => "30.0000000000000000000000002"
>
> I'm clearly missing something here as I thought BigDecimal was supposed
> to fix this type of thing.
>
> Tested in:
> ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]
> ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0]
>
> Can anyone offer an explanation?

BigDecimal is still a kind of float, albeit a decimal float rather
than a binary float.

So it helps with problems when the fractional part of a number can be
expressed exactly as a finite string of decimal digits, but just as
certain fractional parts can't be expressed exactly as a finite string
of binary digits, there are some which have the same problem when the
base is 10.

For example 1/3 cannot be expressed as a decimal float, no matter how
many digits 0.33333.....

Similarly 3000/116 (which is the same as 30/1.16) can't be expressed
by a finite sequence of decimal digits.

It comes out as

25.862068965517241379310344827[5862068965517241379310344827]...

where the digits in the brackets repeat infinitely.

--
Rick DeNatale

Blog: http://talklikeaduck.denh...
Twitter: http://twitter.com/Ri...
WWR: http://www.workingwithrails.com/person/9021-ric...
LinkedIn: http://www.linkedin.com/in/ri...

Samuel Lown

5/6/2009 10:09:00 AM

Rick Denatale wrote:
> On Tue, May 5, 2009 at 9:01 PM, Samuel Lown <me@samlown.com> wrote:
>>
>> Can anyone offer an explanation?
>
> BigDecimal is still a kind of float, albeit a decimal float rather
> than a binary float.
>

Many thanks Nobuyoshi and Rick, that makes complete sense now.

I guess the safest rule of thumb is if you're doing comparisons for
humans, use humanized numbers (i.e. integers :-)

Cheers, sam
--
Posted via http://www.ruby-....

Julian Leviston

5/6/2009 10:11:00 AM

Yeah I think they would be better off with rationals

Blog: http://random8.ze...
Learn: http://sensei.ze...
Twitter: http://twitter.co...

On 06/05/2009, at 1:30 PM, Rick DeNatale <rick.denatale@gmail.com>
wrote:

> On Tue, May 5, 2009 at 9:01 PM, Samuel Lown <me@samlown.com> wrote:
>> Hi,
>>
>> Some more not so fun rounding issues.
>>
>> Using Floats:
>>
>>>> ((30 / 1.16) * 1.16) == 30
>> => true
>>
>> Cool, so you'd think it would work with BigDecimal:
>>
>>>> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")) == 30
>> => false
>>>> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")).to_s
>> => "30.0000000000000000000000002"
>>
>> I'm clearly missing something here as I thought BigDecimal was
>> supposed
>> to fix this type of thing.
>>
>> Tested in:
>> ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]
>> ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0]
>>
>> Can anyone offer an explanation?
>
> BigDecimal is still a kind of float, albeit a decimal float rather
> than a binary float.
>
> So it helps with problems when the fractional part of a number can be
> expressed exactly as a finite string of decimal digits, but just as
> certain fractional parts can't be expressed exactly as a finite string
> of binary digits, there are some which have the same problem when the
> base is 10.
>
> For example 1/3 cannot be expressed as a decimal float, no matter how
> many digits 0.33333.....
>
> Similarly 3000/116 (which is the same as 30/1.16) can't be expressed
> by a finite sequence of decimal digits.
>
> It comes out as
>
> 25.862068965517241379310344827[5862068965517241379310344827]...
>
> where the digits in the brackets repeat infinitely.
>
>
> --
> Rick DeNatale
>
> Blog: http://talklikeaduck.denh...
> Twitter: http://twitter.com/Ri...
> WWR: http://www.workingwithrails.com/person/9021-ric...
> LinkedIn: http://www.linkedin.com/in/ri...
>

Robert Klemme

5/6/2009 11:06:00 AM

2009/5/6 Samuel Lown <me@samlown.com>:
>
> Rick Denatale wrote:
>> On Tue, May 5, 2009 at 9:01 PM, Samuel Lown <me@samlown.com> wrote:
>>>
>>> Can anyone offer an explanation?
>>
>> BigDecimal is still a kind of float, albeit a decimal float rather
>> than a binary float.
>>
>
> Many thanks Nobuyoshi and Rick, that makes complete sense now.
>
> I guess the safest rule of thumb is if you're doing comparisons for
> humans, use humanized numbers (i.e. integers :-)

Or define a maximum error, i.e. do not rely on identical values but
rather the difference of the quotient.

Kind regards

robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestprac...

comp.lang.ruby

Unexpected BigDecimal rounding issue

Samuel Lown

Nobuyoshi Nakada

Rick DeNatale

Samuel Lown

Julian Leviston

Robert Klemme

x Login to ForumsZone