From: Skybuck Flying on
When it comes to programming the GPU/7900 GTX/Shader Model 3.0/VP40/VP40
profile it seems three floating point formats are interesting/available:

1. "Single floating point format" also know as IEE 754 which is probably the
floating point format used by Delphi in the single type as well as CPU
processors (?):

http://en.wikipedia.org/wiki/Single_precision_floating-point_format

2. GPU Profile FP40 has a special "native?" format:

"
The fixed data type corresponds to a native signed fixed-point integers with
the range [-2.0,+2.0), sometimes called fx12. This type provides 10
fractional bits of precision.
"

I am not sure if this format is available in VP40 profile as well for vertex
shaders ???

3. Undetermined:

It seems the CG manual which I have contains typo's, the description in FP40
profile seems focked up:

"
half
The half data type corresponds to a floating-point encoding with a sign bit,
10 mantissa bits, and 5 exponent bits (biased by 16), sometimes called
s10e5.
float

The half data type corresponds to a standard IEEE 754 single-precision
floating-point encoding with a sign bit, 23 mantissa bits, and 8 exponent
bits (biased by 128), sometimes called s10e5.
"

This is a pretty serious documentation mistake since now I cannot figure out
what was ment... tomorrow I will have to check if updated documentation is
available...

I am kinda curious what the "half" type is all about it seems to be
different...

I also wonder which type is the fastest on the GPU: is it fx12 or is it half
?

Maybe half and fx12 is the same thing it both mentions "10" ? Hmm...

Ultimately the question is how to extract these seperated bit fields from
the floating point format/field so that it can be mixed and
stuffed/extracted/shuffled whatever into other integer fields or floating
point fields for processing...

So as to construct an "extract bytes from floating point field" routine...

The newer profiles for G80 gpu's have some kind of functions/routines for
that unfortunately I can't use those because I don't have a G80 but a
Pre-G80 ;)

I still want the same functionality on the Pre-G80... So hopefully that is
somehow possible ?! ;)

Tomorrow I will try to figure it out... by first trying to find some more
documentations, maybe some formula's will pop up... if nothing pops up then
I might have to give it a thought and try myself ;) :)

Fingers crossed for now...

Bye,
Skybuck.


From: Skybuck Flying on
Another link describing what is ment with mantissa for cross reference with
the CG manual which mentions it:

http://en.wikipedia.org/wiki/Significand

Delphi and CG both seem to have a function called: FREXP

This function extracts and returns the "Exponent" and the "Fraction/Integer"
(=Mantissa if I understand correctly)

Thus this function could be used for the "float" types in the CG shaders to
extract those bits.

The only other function which might be necessary is the "sign" function
which returns either 1, 0 or -1 depending on the sign...

The sign bit can be anything since it's just a data bit... it needs to be
extracted and appended as well to the extract values...

If I recall correctly two's complement integers have the highest bit set if
negative for signed integers, or if they value is really large. Which is
more or less the same thing...

So if the sign bit is set this means nothing... it could be a small integer
and would be misleading...

I think special treatment might be necessary to prepare the textures for
this...

One trick which comes to mind if the following:

The texture/texel setter checks the two complement value to see if it's 32th
bit is set. If it is set then the floating point value is made negative.
Otherwise it's left/made positive.

Now in the shader it should be pretty easy to decode the sign bit possibly
with a small lookup table to prevent an expensive branch....

[-1] must become 1
[0] must become 0
[1] must become 0

So lookup tables looks like:
[sign(-1)+1=0] = 1
[sign(0)+1=1] = 0
[sign(1)+1=2] = 0

I think this could work... kinda shamefull that 3 temporarely registers have
to be used for this ?! ;)
But maybe it can somehow be stored in the x/y/z/w components to only use one
vector register ?

So far so good, this method might be workable... might be fast...

Maybe there is a better and faster way to extract all bits from a floating
point value.

I have seen some routines/functions but they kinda look complex and hard to
understand which makes them kinda doubtfull... they also didn't seem to
extract all bits ?

I definetly want to extra all 32 bits from a floating point value ! ;)

Bye,
Skybuck.


From: Skybuck Flying on
However I might have overlooked something when it comes to "sign"...

It could also return "NaN"... that would be a nasty bug that could suddenly
blow up in ones face far down the road ;)

I saw somebody else mention this and took care of it somehow with some
complex formula... etc...

Hmm... I wonder if NaN will truely be problematic... it might be since the
textures don't care with kind of floating point values it are... for them
it's just a bunch of bits ;)

Bye,
Skybuck.


From: Skybuck Flying on
No, this link mentions NaN is part of the exponential... and not the sign:

http://en.wikipedia.org/wiki/NaN

The comment (4) in the CG manual was probably referring to something else:

CG Manual:

"sign" description:

"
DESCRIPTION
Returns positive one, zero, or negative one for each of the components of x
based on the component's sign.
1) Returns -1 component if the respective component of x is negative.
2) Returns 0 component if the respective component of x is zero.
3) Returns 1 component if the respective component of x is positive.

4) Ideally, NaN returns NaN.
"

Bye,
Skybuck.