Python -- floating point arithmetic
in [Python]
|
From: Adam Skutt on 8 Jul 2010 06:58 On Jul 8, 12:53 am, "Zooko O'Whielacronx" <zo...(a)zooko.com> wrote: > I don't understand. I described two different problems: problem one is > that the inputs, outputs and literals of your program might be in a > different encoding (in my experience they have most commonly been in > decimal). Problem two is that your computations may be lossy. If the > inputs, outputs and literals of your program are decimal (as they have > been for most of my programs) then using decimal is better than using > float because of problem one. Neither has a strict advantage over the > other in terms of problem two. I can't think of any program I've ever written where the inputs are actually intended to be decimal. Consider a simple video editing program, and the user specifies a frame rate 23.976 fps. Is that what they really wanted? No, they wanted 24000/1001 but didn't feel like typing that. It should be instructive and telling that mplayer now strongly prefers the rational expressions over the decimal ones. Just because you haven't rounded your input in parsing it doesn't change the fact your input may have be rounded before it was presented to the program, which is the reality of the matter more often than not. Even in fields where the numbers are decimal, like finance, the rules are considerably more complicated than what you were taught in the "schoolbook". This is one of the reasons IBM can still make a killing selling mainframes, they got all of those rules right decades ago. > > And that will be never true once you > > start using non trivial computation (i.e. transcendental functions > > like log, exp, etc...). > > I'm sorry, what will never be true? Are you saying that decimals have > a disadvantage compared to floats? If so, what is their disadvantage? He's saying that once you get past elementary operations, you quickly run into irrational numbers, which you will not be representing accurately. Moreover, in general, it's impossible to even round operations involving transcendental functions to an arbitrary fixed- precision, you may need effectively infinite precision in order to the computation. In practice, this means the error induced by a lossy input conversion (assuming you hadn't already lost information) is entirely negligible compared to inherent inability to do the necessary calculations.
From: Mark Dickinson on 8 Jul 2010 07:23 On Jul 8, 11:58 am, Adam Skutt <ask...(a)gmail.com> wrote: > accurately. Moreover, in general, it's impossible to even round > operations involving transcendental functions to an arbitrary fixed- > precision, you may need effectively infinite precision in order to the > computation. Impossible? Can you explain what you mean by this? Doesn't the decimal module do exactly that, giving correctly-rounded exp() and log() results to arbitrary precision? Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import decimal >>> decimal.getcontext().prec = 1000 >>> decimal.Decimal(2).ln() Decimal('0.6931471805599453094172321214581765680755001343602552541206800094933936219696947156058633269964186875420014810205706857336855202357581305570326707516350759619307275708283714351903070386238916734711233501153644979552391204751726815749320651555247341395258829504530070953263666426541042391578149520437404303855008019441706416715186447128399681717845469570262716310645461502572074024816377733896385506952606683411372738737229289564935470257626520988596932019650585547647033067936544325476327449512504060694381471046899465062201677204245245296126879465461931651746813926725041038025462596568691441928716082938031727143677826548775664850856740776484514644399404614226031930967354025744460703080960850474866385231381816767514386674766478908814371419854942315199735488037516586127535291661000710535582498794147295092931138971559982056543928717000721808576102523688921324497138932037843935308877482597017155910708823683627589842589185353024363421436706118923678919237231467232172053401649256872747782344535348') -- Mark
From: David Mainzer on 8 Jul 2010 08:16 On 07/07/2010 08:08 PM, Raymond Hettinger wrote: > On Jul 7, 5:55 am, Mark Dickinson <dicki...(a)gmail.com> wrote: >> On Jul 7, 1:05 pm, david mainzer <d...(a)tu-clausthal.de> wrote: >> >> >> >>> Dear Python-User, >> >>> today i create some slides about floating point arithmetic. I used an >>> example from >> >>> http://docs.python.org/tutorial/floatingpoint.html >> >>> so i start the python shell on my linux machine: >> >>> dm(a)maxwell $ python >>> Python 2.6.5 (release26-maint, May 25 2010, 12:37:06) >>> [GCC 4.3.4] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information.>>> >>> sum = 0.0 >>>>>>>>> for i in range(10): >> >>> ... sum += 0.1 >>> ...>>> >>> sum >>> 0.99999999999999989 >> >>> But thats looks a little bit wrong for me ... i must be a number greater >>> then 1.0 because 0.1 = 0.100000000000000005551115123125782702118158340454101562500000000000 >>> in python ... if i print it. > > [Mark Dickinson] >> So you've identified one source of error here, namely that 0.1 isn't >> exactly representable (and you're correct that the value stored >> internally is actually a little greater than 0.1). But you're >> forgetting about the other source of error in your example: when you >> do 'sum += 0.1', the result typically isn't exactly representable, so >> there's another rounding step going on. That rounding step might >> produce a number that's smaller than the actual exact sum, and if >> enough of your 'sum += 0.1' results are rounded down instead of up, >> that would easily explain why the total is still less than 1.0. > > One key for understanding floating point mysteries is to look at the > actual binary sums rather that their approximate representation as a > decimal string. The hex() method can make it easier to visualize > Mark's explanation: > >>>> s = 0.0 >>>> for i in range(10): > ... s += 0.1 > ... print s.hex(), repr(s) > > > 0x1.999999999999ap-4 0.10000000000000001 > 0x1.999999999999ap-3 0.20000000000000001 > 0x1.3333333333334p-2 0.30000000000000004 > 0x1.999999999999ap-2 0.40000000000000002 > 0x1.0000000000000p-1 0.5 > 0x1.3333333333333p-1 0.59999999999999998 > 0x1.6666666666666p-1 0.69999999999999996 > 0x1.9999999999999p-1 0.79999999999999993 > 0x1.cccccccccccccp-1 0.89999999999999991 > 0x1.fffffffffffffp-1 0.99999999999999989 > > Having used hex() to understand representation error (how the binary > partial sums are displayed), you can use the Fractions module to gain > a better understanding of rounding error introduced by each addition: > >>>> s = 0.0 >>>> for i in range(10): > exact = Fraction.from_float(s) + Fraction.from_float(0.1) > s += 0.1 > actual = Fraction.from_float(s) > error = actual - exact > print '%-35s%-35s\t%s' % (actual, exact, error) > > > 3602879701896397/36028797018963968 3602879701896397/36028797018963968 > 0 > 3602879701896397/18014398509481984 3602879701896397/18014398509481984 > 0 > 1351079888211149/4503599627370496 10808639105689191/36028797018963968 > 1/36028797018963968 > 3602879701896397/9007199254740992 > 14411518807585589/36028797018963968 -1/36028797018963968 > 1/2 > 18014398509481985/36028797018963968 -1/36028797018963968 > 5404319552844595/9007199254740992 > 21617278211378381/36028797018963968 -1/36028797018963968 > 3152519739159347/4503599627370496 > 25220157913274777/36028797018963968 -1/36028797018963968 > 7205759403792793/9007199254740992 > 28823037615171173/36028797018963968 -1/36028797018963968 > 2026619832316723/2251799813685248 > 32425917317067569/36028797018963968 -1/36028797018963968 > 9007199254740991/9007199254740992 > 36028797018963965/36028797018963968 -1/36028797018963968 > > Hope this helps your slides, > > > Raymond Thanks to all for your help :) All your comments helped me and now i know how it works in python ! Best David
From: Adam Skutt on 8 Jul 2010 09:00 On Jul 8, 7:23 am, Mark Dickinson <dicki...(a)gmail.com> wrote: > On Jul 8, 11:58 am, Adam Skutt <ask...(a)gmail.com> wrote: > > > accurately. Moreover, in general, it's impossible to even round > > operations involving transcendental functions to an arbitrary fixed- > > precision, you may need effectively infinite precision in order to the > > computation. > > Impossible? Can you explain what you mean by this? Doesn't the > decimal module do exactly that, giving correctly-rounded exp() and > log() results to arbitrary precision? You run into the table-maker's dilemma: there's no way to know in advance how many digits you need in order to have n bits of precision in the result. For some computations, the number of bits required to get the desired precision can quickly overwhelm the finite limitations of your machine (e.g., you run out of RAM first or the time to compute the answer is simply unacceptable). Adam
From: Ethan Furman on 8 Jul 2010 09:10
Wolfram Hinderer wrote: > On 7 Jul., 19:32, Ethan Furman <et...(a)stoneleaf.us> wrote: > >>Nobody wrote: >> >>>On Wed, 07 Jul 2010 15:08:07 +0200, Thomas Jollans wrote: >> >>>>you should never rely on a floating-point number to have exactly a >>>>certain value. >> >>>"Never" is an overstatement. There are situations where you can rely >>>upon a floating-point number having exactly a certain value. >> >>It's not much of an overstatement. How many areas are there where you >>need the number >>0.100000000000000005551115123125782702118158340454101562500000000000? >> >>If I'm looking for 0.1, I will *never* (except by accident ;) say >> >>if var == 0.1: >> >>it'll either be <= or >=. > > > The following is an implementation of a well-known algorithm. > Why would you want to replace the floating point comparisons? With > what? <snip code> Interesting. I knew when I posted my above comment that I was ignoring such situations. I cannot comment on the code itself as I am unaware of the algorithm, and haven't studied numbers extensively (although I do find them very interesting). So while you've made your point, I believe mine still stands -- comparing floats using == to absolute numbers is almost always a mistake. ~Ethan~ |