Prev: Intel documentation mistake for single floating point precision !
Next: 1D to 3D coordinate translation. (No 16 million is right ?!)
From: Skybuck Flying on 2 Feb 2010 10:10
Nope, it's not a mistake... I made a mistake...

8000*8000 is way over 24 bits ! ;) :)

I checked with this calculator:

http://www.h-schmidt.net/FloatApplet/IEEE754.html

And also by doing (2^24) ^ 0.5 = 4096

So:

(4095 * 4095)-1 is still in range ! ;)

Bye,
Skybuck =D


 | 
Pages: 1
Prev: Intel documentation mistake for single floating point precision !
Next: 1D to 3D coordinate translation. (No 16 million is right ?!)