Increase significant digits in Float
in [Ruby]
|
From: Jason Lillywhite on 2 Mar 2010 10:26 If I want to increase my significant digits beyond 15 in a result of a math expression involving floats, how do I do it? So, for example, I would like more than 15 sig digits in the result of: irb(main):001:0> 4.005 / 7 => 0.572142857142857 Thank you! -- Posted via http://www.ruby-forum.com/.
From: Jesús Gabriel y Galán on 2 Mar 2010 10:37 On Tue, Mar 2, 2010 at 4:26 PM, Jason Lillywhite <jason.lillywhite(a)gmail.com> wrote: > If I want to increase my significant digits beyond 15 in a result of a > math expression involving floats, how do I do it? So, for example, I > would like more than 15 sig digits in the result of: > > irb(main):001:0> 4.005 / 7 > => 0.572142857142857 irb is calling inspect on the value of the expression. It's the inspect method the one that truncates to 15 digits when it creates the string. You can create your own strings with greater precision like this: irb(main):001:0> result = 4.005 / 7 => 0.572142857142857 irb(main):002:0> "%.20f" % result => "0.57214285714285717521" irb(main):003:0> "%.30f" % result => "0.572142857142857175212213860505" Check the String#% method: http://ruby-doc.org/core/classes/String.html#M000770 and the Kernel#sprintf method: http://ruby-doc.org/core/classes/Kernel.html#M005962 Jesus.
From: Jason Lillywhite on 2 Mar 2010 11:00 Jesús Gabriel y Galán wrote: > irb is calling inspect on the value of the expression. It's the > inspect method the one that truncates to 15 digits when it creates the > string. Thank you! I forgot that IRB is returning a truncated string to the screen. -- Posted via http://www.ruby-forum.com/.
From: Xavier Noria on 2 Mar 2010 11:25 On Tue, Mar 2, 2010 at 5:00 PM, Jason Lillywhite <jason.lillywhite(a)gmail.com> wrote: > Thank you! I forgot that IRB is returning a truncated string to the > screen. That 15 is hard-coded in the definition of Float#to_s.
From: Siep Korteling on 2 Mar 2010 12:27
Xavier Noria wrote: > On Tue, Mar 2, 2010 at 5:00 PM, Jason Lillywhite > <jason.lillywhite(a)gmail.com> wrote: > >> Thank you! I forgot that IRB is returning a truncated string to the >> screen. > > That 15 is hard-coded in the definition of Float#to_s. Yes, and exposing more digits is not accurate: puts "%.70f"%(4.005 / 7) #=>0.5721428571428571752122138605045620352029800415039062500000000000000000 require 'bigdecimal' a = BigDecimal.new("4.005") b = a/7 puts b #=>0.572142857142857142857143E0 p b.precs #=>[24, 32] #first number is the number of significant digits in b; the second number is the maximum precision my system can handle (not sure about this). The float is bogus after the 16th digit. hth, Siep -- Posted via http://www.ruby-forum.com/. |