[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

[BUG?] Strange behavior in Integer

Joe Davison

8/4/2008 9:59:00 PM

new user on MacOsX.
Ruby version: ruby 1.8.2 (2004-12-25) [universal-darwin8.0]
intel MacBook Pro

Working thru the examples in Programming Ruby, I got what I didn't
expect:

Integer(33.80 * 100) ==> 3379

however,

Integer(3.380 * 1000) ==> 3380

Doesn't seem like a floating point round off error.

joe
4 Answers

Sebastian Hungerecker

8/4/2008 10:03:00 PM

0

Joe Davison wrote:
> Integer(33.80 * 100) ==> 3379
>
> however,
>
> Integer(3.380 * 1000) ==> 3380
>
> Doesn't seem like a floating point round off error.

>> "%.30f" % (33.80 * 100)
=> "3379.999999999999545252649113535881"

Seems like one to me.

HTH,
Sebastian
--
Jabber: sepp2k@jabber.org
ICQ: 205544826

Todd Benson

8/5/2008 12:18:00 AM

0

On Mon, Aug 4, 2008 at 5:02 PM, Sebastian Hungerecker
<sepp2k@googlemail.com> wrote:
> Joe Davison wrote:
>> Integer(33.80 * 100) ==> 3379
>>
>> however,
>>
>> Integer(3.380 * 1000) ==> 3380
>>
>> Doesn't seem like a floating point round off error.
>
>>> "%.30f" % (33.80 * 100)
> => "3379.999999999999545252649113535881"
>
> Seems like one to me.
>
> HTH,
> Sebastian

Just playing around a little (no exhaustive testing), but it seems
like this only happens with a number string matching
/\A3[2-9]\.[38]0*\Z/ (for nubies, a number that is in the closed
interval 32 to 40 with a following 3 or 8 in the decimal place and
nothing after it except for zero or more zeros). Weird. There might
be other cases.

I'm using the 1.8.7 ruby ver, btw.

Todd

Joost Diepenmaat

8/5/2008 12:58:00 AM

0

Todd Benson <caduceass@gmail.com> writes:

> On Mon, Aug 4, 2008 at 5:02 PM, Sebastian Hungerecker
> <sepp2k@googlemail.com> wrote:
>> Joe Davison wrote:
>>>> "%.30f" % (33.80 * 100)
>> => "3379.999999999999545252649113535881"
>>
>> Seems like one to me.
>>
>> HTH,
>> Sebastian
>
> Just playing around a little (no exhaustive testing), but it seems
> like this only happens with a number string matching
> /\A3[2-9]\.[38]0*\Z/ (for nubies, a number that is in the closed
> interval 32 to 40 with a following 3 or 8 in the decimal place and
> nothing after it except for zero or more zeros). Weird. There might
> be other cases.

http://docs.sun.com/source/806-3568/ncg_gol...

--
Joost Diepenmaat | blog: http://joost.... | work: http://...

Todd Benson

8/5/2008 3:23:00 AM

0

On Mon, Aug 4, 2008 at 7:58 PM, Joost Diepenmaat <joost@zeekat.nl> wrote:
> Todd Benson <caduceass@gmail.com> writes: There might
>> be other cases.
>
> http://docs.sun.com/source/806-3568/ncg_gol...

Indeed. These errors occur starting with 32, then 512, then 8192,
etc. with expanding ranges. The pattern I get, but I don't
understand, though, why 3 and 8?

Todd