Prev: hash example
Next: [ANN] Free shell accounts on new ruby-versions.net site
From: Karthick S. on 12 Jul 2010 04:18
Hi,

I am new to ruby and found this while experimenting with ruby.

irb(main):010:0> 5.1%0.5
=> 0.0999999999999996
irb(main):011:0>

Is this similar to the float related operations that MS Excel has?

- Karthick S.
From: Brian Candler on 12 Jul 2010 06:15
Karthick S. wrote:
> I am new to ruby and found this while experimenting with ruby.
>
> irb(main):010:0> 5.1%0.5
> => 0.0999999999999996

Ruby is no different to any other language which uses binary floating
point arithmetic:

$ perl -e 'print 5.1 - 10*0.5, "\n"'
0.0999999999999996

The problem is that 1/10 does not have an exact representation in binary
floating point, in the same way that 1/3 does not have an exact
representation in decimal floating point (0.33333...)

To learn more see http://docs.sun.com/source/806-3568/ncg_goldberg.html

For some applications, you might use the Rational or BigDecimal classes
instead. If dealing with currency, you can work in Integers for
cents/pence etc.

Or maybe you just want to use fewer significant digits when printing:

>> v = 5.1 % 0.5
=> 0.0999999999999996
>> "%.8f" % v
=> "0.10000000"

although you're still likely to be tripped up by this:

>> v == 0.1
=> false

HTH,

Brian.
--
Posted via http://www.ruby-forum.com/.

From: Rick DeNatale on 12 Jul 2010 09:59
On Mon, Jul 12, 2010 at 6:15 AM, Brian Candler <b.candler(a)pobox.com> wrote:
> Karthick S. wrote:
>> I am new to ruby and found this while experimenting with ruby.
>>
>> irb(main):010:0> 5.1%0.5
>> => 0.0999999999999996
>
> Ruby is no different to any other language which uses binary floating
> point arithmetic

It isn't relevant to the OPs example, but not all languages produce
the results for modulo when presented with negative arguments:

http://en.wikipedia.org/wiki/Modulo_operation#Common_pitfalls

Ruby, Perl, and MS Excel all return a result with the sign of the
divisor, while Java returns a result with the sign of the dividend.

--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Github: http://github.com/rubyredrick
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

 | 
Pages: 1
Prev: hash example
Next: [ANN] Free shell accounts on new ruby-versions.net site