Python -- floating point arithmetic
in [Python]
|
From: david mainzer on 7 Jul 2010 08:05 Dear Python-User, today i create some slides about floating point arithmetic. I used an example from http://docs.python.org/tutorial/floatingpoint.html so i start the python shell on my linux machine: dm(a)maxwell $ python Python 2.6.5 (release26-maint, May 25 2010, 12:37:06) [GCC 4.3.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> >>> sum = 0.0 >>> >>> for i in range(10): .... sum += 0.1 .... >>> >>> sum 0.99999999999999989 >>> >>> But thats looks a little bit wrong for me ... i must be a number greater then 1.0 because 0.1 = 0.100000000000000005551115123125782702118158340454101562500000000000 in python ... if i print it. So i create an example program: sum = 0.0 n = 10 d = 1.0 / n print "%.60f" % ( d ) for i in range(n): print "%.60f" % ( sum ) sum += d print sum print "%.60f" % ( sum ) -------- RESULTs ------ 0.100000000000000005551115123125782702118158340454101562500000 0.000000000000000000000000000000000000000000000000000000000000 0.100000000000000005551115123125782702118158340454101562500000 0.200000000000000011102230246251565404236316680908203125000000 0.300000000000000044408920985006261616945266723632812500000000 0.400000000000000022204460492503130808472633361816406250000000 0.500000000000000000000000000000000000000000000000000000000000 0.599999999999999977795539507496869191527366638183593750000000 0.699999999999999955591079014993738383054733276367187500000000 0.799999999999999933386618522490607574582099914550781250000000 0.899999999999999911182158029987476766109466552734375000000000 1.0 0.999999999999999888977697537484345957636833190917968750000000 and the jump from 0.50000000000000*** to 0.59999999* looks wrong for me ... do i a mistake or is there something wrong in the representation of the floating points in python? my next question, why could i run print "%.66f" % ( sum ) but not print "%.67f" % ( sum ) can anybody tell me how python internal represent a float number?? Best and many thanks in advanced, David
From: Mark Dickinson on 7 Jul 2010 08:55 On Jul 7, 1:05 pm, david mainzer <d...(a)tu-clausthal.de> wrote: > Dear Python-User, > > today i create some slides about floating point arithmetic. I used an > example from > > http://docs.python.org/tutorial/floatingpoint.html > > so i start the python shell on my linux machine: > > dm(a)maxwell $ python > Python 2.6.5 (release26-maint, May 25 2010, 12:37:06) > [GCC 4.3.4] on linux2 > Type "help", "copyright", "credits" or "license" for more information.>>> >>> sum = 0.0 > >>> >>> for i in range(10): > > ... sum += 0.1 > ...>>> >>> sum > 0.99999999999999989 > > But thats looks a little bit wrong for me ... i must be a number greater > then 1.0 because 0.1 = 0.100000000000000005551115123125782702118158340454101562500000000000 > in python ... if i print it. So you've identified one source of error here, namely that 0.1 isn't exactly representable (and you're correct that the value stored internally is actually a little greater than 0.1). But you're forgetting about the other source of error in your example: when you do 'sum += 0.1', the result typically isn't exactly representable, so there's another rounding step going on. That rounding step might produce a number that's smaller than the actual exact sum, and if enough of your 'sum += 0.1' results are rounded down instead of up, that would easily explain why the total is still less than 1.0. > > So i create an example program: > > sum = 0.0 > n = 10 > d = 1.0 / n > print "%.60f" % ( d ) > for i in range(n): > print "%.60f" % ( sum ) > sum += d > > print sum > print "%.60f" % ( sum ) > > -------- RESULTs ------ > 0.100000000000000005551115123125782702118158340454101562500000 > 0.000000000000000000000000000000000000000000000000000000000000 > 0.100000000000000005551115123125782702118158340454101562500000 > 0.200000000000000011102230246251565404236316680908203125000000 > 0.300000000000000044408920985006261616945266723632812500000000 > 0.400000000000000022204460492503130808472633361816406250000000 > 0.500000000000000000000000000000000000000000000000000000000000 > 0.599999999999999977795539507496869191527366638183593750000000 > 0.699999999999999955591079014993738383054733276367187500000000 > 0.799999999999999933386618522490607574582099914550781250000000 > 0.899999999999999911182158029987476766109466552734375000000000 > 1.0 > 0.999999999999999888977697537484345957636833190917968750000000 > > and the jump from 0.50000000000000*** to 0.59999999* looks wrong > for me ... do i a mistake or is there something wrong in the > representation of the floating points in python? Look at this more closely: you're adding 0.500000000000000000000000.... to 0.1000000000000000055511151231257827021181583404541015625 The *exact* result is, of course 0.6000000000000000055511151231257827021181583404541015625 However, that's not a number that can be exactly represented as a C double (which is how Python stores floats internally). This value falls between the two (consecutive) representable values: 0.59999999999999997779553950749686919152736663818359375 and 0.600000000000000088817841970012523233890533447265625 But of these two, the first is closer to the exact value than the second, so that's the result that you get. You can convince yourself of these results by using the fractions module to do exact arithmetic: >>> from fractions import Fraction >>> tenth = Fraction.from_float(0.1) >>> half = Fraction.from_float(0.5) >>> point_six = Fraction.from_float(0.6) # 0.599999999999 >>> point_six_plus = Fraction.from_float(0.6 + 2**-53) # next float up, 0.6000000 >>> sum = tenth + half # exact value of the sum >>> point_six < sum < point_six_plus # lies between point_six and point_six_plus True >>> sum - point_six < point_six_plus - sum # but it's closer to point_six True > my next question, why could i run > > print "%.66f" % ( sum ) > > but not > > print "%.67f" % ( sum ) That's a historical artefact resulting from use of a fixed-length buffer somewhere deep in Python's internals. This restriction is removed in Python 2.7 and Python 3.x. > can anybody tell me how python internal represent a float number?? In CPython, it's stored as a C double, which typically means in IEEE 754 binary64 format. (Of course, since it's a Python object, it's not just storing the C double itself; it also has fields for the object type and the reference count, so a Python float typically takes 16 bytes of memory on a 32-bit machine, and 24 bytes on a 64-bit machine.) -- Mark
From: Thomas Jollans on 7 Jul 2010 09:08 On 07/07/2010 02:05 PM, david mainzer wrote: > today i create some slides about floating point arithmetic. I used an > example from > > http://docs.python.org/tutorial/floatingpoint.html > > so i start the python shell on my linux machine: > > dm(a)maxwell $ python > Python 2.6.5 (release26-maint, May 25 2010, 12:37:06) > [GCC 4.3.4] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>>>>> sum = 0.0 >>>>>>> for i in range(10): > ... sum += 0.1 > ... >>>>>>> sum > 0.99999999999999989 >>>>>>> > But thats looks a little bit wrong for me ... i must be a number greater > then 1.0 because 0.1 = 0.100000000000000005551115123125782702118158340454101562500000000000 > in python ... if i print it. The simple fact of the matter is: floating point arithmetic isn't accurate. This has nothing to do with Python, it's the fault of your processor's floating point handling. It's good enough in most cases, but you should never rely on a floating-point number to have exactly a certain value. It won't work. To generalize your example a bit: >>> def test_floating_product(a, b): .... sum = 0 .... for _ in range(int(b)): .... sum += a .... return sum, a * int(b), sum == a * b .... >>> test_floating_product(0.1, 1) (0.1, 0.1, True) >>> test_floating_product(0.1, 10) (0.9999999999999999, 1.0, False) >>> test_floating_product(0.2, 4) (0.8, 0.8, True) >>> test_floating_product(0.2, 5) (1.0, 1.0, True) >>> test_floating_product(0.2, 6) (1.2, 1.2000000000000002, False) >>> > -------- RESULTs ------ > 0.100000000000000005551115123125782702118158340454101562500000 > 0.000000000000000000000000000000000000000000000000000000000000 > 0.100000000000000005551115123125782702118158340454101562500000 > 0.200000000000000011102230246251565404236316680908203125000000 > 0.300000000000000044408920985006261616945266723632812500000000 > 0.400000000000000022204460492503130808472633361816406250000000 > 0.500000000000000000000000000000000000000000000000000000000000 > 0.599999999999999977795539507496869191527366638183593750000000 > 0.699999999999999955591079014993738383054733276367187500000000 > 0.799999999999999933386618522490607574582099914550781250000000 > 0.899999999999999911182158029987476766109466552734375000000000 > 1.0 > 0.999999999999999888977697537484345957636833190917968750000000 > > and the jump from 0.50000000000000*** to 0.59999999* looks wrong > for me ... do i a mistake or is there something wrong in the > representation of the floating points in python? the difference is almost exactly 0.1, so that looks okay > > my next question, why could i run > > print "%.66f" % ( sum ) > > but not > > print "%.67f" % ( sum ) I can run either... with Python 3.1. Using 2.6, I get a nice error message: >>> "%.129f" % 0.1 Traceback (most recent call last): File "<stdin>", line 1, in <module> OverflowError: formatted float is too long (precision too large?) There just isn't anything like 67 decimals of information available. Having that information wouldn't help you a bit. basically, floating point number are stored in the format N * (2 ** E) And use a lot of guesswork. as E gets larger, the precision decreases. Rounding errors occur at the last few decimals, in either direction, depending on the numbers. > > can anybody tell me how python internal represent a float number?? >
From: Christian Heimes on 7 Jul 2010 09:24 > can anybody tell me how python internal represent a float number?? It's an IEEE 754 double precision float on all hardware platforms that support IEEE 754 semantics. Python follows the C99 standards for double and complex numbers. Christian
From: Philip Semanchuk on 7 Jul 2010 10:05
On Jul 7, 2010, at 9:08 AM, Thomas Jollans wrote: > On 07/07/2010 02:05 PM, david mainzer wrote: >> today i create some slides about floating point arithmetic. I used an >> example from >> >> http://docs.python.org/tutorial/floatingpoint.html >> >> so i start the python shell on my linux machine: >> >> dm(a)maxwell $ python >> Python 2.6.5 (release26-maint, May 25 2010, 12:37:06) >> [GCC 4.3.4] on linux2 >> Type "help", "copyright", "credits" or "license" for more >> information. >>>>>>>> sum = 0.0 >>>>>>>> for i in range(10): >> ... sum += 0.1 >> ... >>>>>>>> sum >> 0.99999999999999989 >>>>>>>> >> But thats looks a little bit wrong for me ... i must be a number >> greater >> then 1.0 because 0.1 = >> 0.100000000000000005551115123125782702118158340454101562500000000000 >> in python ... if i print it. > > The simple fact of the matter is: floating point arithmetic isn't > accurate. This has nothing to do with Python, it's the fault of your > processor's floating point handling. It's good enough in most cases, > but > you should never rely on a floating-point number to have exactly a > certain value. It won't work. Yes, this might be a good time to review the dense but interesting document, "What Every Computer Scientist Should Know About Floating- Point Arithmetic": http://docs.sun.com/source/806-3568/ncg_goldberg.html Cheers Philip |