FAQ Topic - Why does simple decimal arithmetic give strange results? (2010-07-29)
in [JavaScript]
|
Prev: Cairo& Giza sightseeing tour, quick overday trip shore excursion from Alexandria port
Next: A "does global variable exist" function
From: FAQ server on 28 Jul 2010 19:00 ----------------------------------------------------------------------- FAQ Topic - Why does simple decimal arithmetic give strange results? ----------------------------------------------------------------------- For example, `5 * 1.015` does not give exactly `5.075` and `0.06+0.01` does not give exactly `0.07` in javascript. ECMAScript numbers are represented in binary as IEEE-754 (IEC 559) Doubles, with a resolution of 53 bits, giving an accuracy of 15-16 decimal digits; integers up to just over `9e15` are precise, but few decimal fractions are. Given this, arithmetic is as exact as possible, but no more. Operations on integers are exact if the true result and all intermediates are integers within that range. In particular, non-integer results should not normally be compared for equality; and non-integer computed results commonly need rounding; see How do I format a Number as a String with exactly 2 decimal places? [ref 1] <URL: http://msdn.microsoft.com/en-us/library/7wkd9z69%28VS.85%29.aspx> <URL: http://www.merlyn.demon.co.uk/js-misc0.htm#DW4> Otherwise, use `Math.round` on the results of expressions which should be of integer value. References: ----------- [1] http://jibbering.com/faq/#formatNumber The complete comp.lang.javascript FAQ is at http://jibbering.com/faq/ -- The sendings of these daily posts are proficiently hosted by http://www.pair.com.
From: Dr J R Stockton on 30 Jul 2010 17:26
In comp.lang.javascript message <4c50b679$0$276$14726298(a)news.sunsite.dk >, Wed, 28 Jul 2010 23:00:03, FAQ server <javascript(a)dotinternet.be> posted: >FAQ Topic - Why does simple decimal arithmetic give strange >results? >For example, `5 * 1.015` does not give exactly >`5.075` and `0.06+0.01` does >not give exactly `0.07` in javascript. That's not really right, since JavaScript cannot store any of those values in a Number. Could be : For example, `"5" * "1.015"` does not give exactly `"5.075"` and `"0.06" + "0.01"` does not give exactly `"0.07"` in javascript. or X = 5 ; Y = 1.015 ; Z = (X*Y).toString() // -> "5.074999999999999" -- (c) John Stockton, nr London UK. ?@merlyn.demon.co.uk Turnpike v6.05 MIME. Web <URL:http://www.merlyn.demon.co.uk/> - FAQish topics, acronyms, & links. Proper <= 4-line sig. separator as above, a line exactly "-- " (RFCs 5536/7) Do not Mail News to me. Before a reply, quote with ">" or "> " (RFCs 5536/7) |