Prev: Basic custom layout manager question
Next: Sending SMS
From: Mikee on 9 Jun 2010 03:17
Hi

I having trouble reading a binary file. The problem seems to occur
with reading some float values so I've made a small test file (from
SQL server using native file output) that contains just 5 floats.

-13.73482 , 20.689 , -99.99999 , 20.157 ,1 9.454

I read the file in and write out the values using

DataInputStream in = new DataInputStream(new
BufferedInputStream(new FileInputStream(filename)));
System.out.print(ByteSwapper.swap(in.readFloat())+" ");
System.out.print(ByteSwapper.swap(in.readFloat())+" ");
System.out.print(ByteSwapper.swap(in.readFloat())+" ");
System.out.print(ByteSwapper.swap(in.readFloat())+" ");
System.out.println(ByteSwapper.swap(in.readFloat())+" ");

The output is -13.734822 20.689 6.9055E-41 20.157 19.454

ie -99.99999 is now 6.9055E-41.

I'm trying to read in a much larger SQL outgest of mixed data types
integers, longs, doubles, floats and bytes and the only problem I've
spotted is floats of -99.9999 (which is a deafult value used for some
columns)

It was necessary to byte swap and ByteSwapper.swap(float) is something
I find on the web an dis below.
I tries other byte swap code with the same result.

Any ideas what I'm doing wrong with the 3rd value and presumably in
general.

Python code of

binaryFile = file('test.bin', 'rb')
while True:
rowBinary = binaryFile.read(20)
if not rowBinary:
break
print struct.unpack('<5f', rowBinary)
binaryFile.close()

correctly gives -13.734822273254395, 20.688999176025391,
-99.999992370605469, 20.156999588012695, 19.454000473022461

java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_12-b04

Thanks
Mike

public static float swap (float value)
{
int intValue = Float.floatToIntBits (value);
intValue = swap (intValue);
return Float.intBitsToFloat (intValue);
}

and

public static int swap (int value)
{
int b1 = (value >> 0) & 0xff;
int b2 = (value >> 8) & 0xff;
int b3 = (value >> 16) & 0xff;
int b4 = (value >> 24) & 0xff;

return b1 << 24 | b2 << 16 | b3 << 8 | b4 << 0;
}
From: Thomas Pornin on 9 Jun 2010 06:14
According to Mikee <mikee.read(a)googlemail.com>:
> DataInputStream in = new DataInputStream(new
> BufferedInputStream(new FileInputStream(filename)));
> System.out.print(ByteSwapper.swap(in.readFloat())+" ");
> System.out.print(ByteSwapper.swap(in.readFloat())+" ");
> System.out.print(ByteSwapper.swap(in.readFloat())+" ");
> System.out.print(ByteSwapper.swap(in.readFloat())+" ");
> System.out.println(ByteSwapper.swap(in.readFloat())+" ");
>
[...]
> public static float swap (float value)
> {
> int intValue = Float.floatToIntBits (value);
> intValue = swap (intValue);
> return Float.intBitsToFloat (intValue);
> }

This is conceptually flawed. What you do, here, is the following:
1. read four bytes
2. interpret them as a float (in.readFloat())
3. convert them back to four bytes, as an "int" value
(Float.floatToIntBits())
4. byte-swap the bytes
5. finally reinterpret the bytes as a float (Float.intBitsToFloat())

Steps 2 and 3 basically cancel each other: step 3 undoes what step 2
did. The trouble is that this roundtrip (steps 2 and 3) may lose some
information, because some bit patterns encode a "NaN" (a special value
for floats) and Float.floatToIntBits() normalizes NaN values. This is
probably what happens to you: when read "byte-swapped", your -99.99999
becomes a pattern for a NaN, which gets normalized to another NaN,
thus changing the bit pattern.

You _might_ correct your code by using Float.float.toRawIntBits()
instead of Float.floatToIntBits(), but even if it works, it is
non-robust (it assumes that DataInputStream also does things that way).
A more correct way of doing things is to read the bytes as an "int"
directly, which basically removes steps 2 and 3 altogether. This would
look like this:

System.out.print(Float.intBitsToFloat(swap(in.readInt())) + " ");


--Thomas Pornin
From: Paul Cager on 9 Jun 2010 06:53
On Jun 9, 11:14 am, Thomas Pornin <por...(a)bolet.org> wrote:
> According to Mikee  <mikee.r...(a)googlemail.com>:
>
> >            DataInputStream in = new DataInputStream(new
> > BufferedInputStream(new FileInputStream(filename)));
> >            System.out.print(ByteSwapper.swap(in.readFloat())+" ");
[...]
> You _might_ correct your code by using Float.float.toRawIntBits()
> instead of Float.floatToIntBits(), but even if it works, it is
> non-robust (it assumes that DataInputStream also does things that way).
> A more correct way of doing things is to read the bytes as an "int"
> directly, which basically removes steps 2 and 3 altogether. This would
> look like this:
>
>     System.out.print(Float.intBitsToFloat(swap(in.readInt())) + " ");

I haven't checked the API (http://java.sun.com/javase/6/docs/api/
index.html?java/nio/package-summary.html) but I seem to remember that
NIO allows you to read floats etc in the byte order of your choice.
That might avoid the need to do all the bit twiddling yourself.
From: Mikee on 9 Jun 2010 09:03
Hi Thomas

Thanks for poitning out where I was going wrong.

>     System.out.print(Float.intBitsToFloat(swap(in.readInt())) + " ");

Yes that worked,

Thanks again
Mike
From: Mikee on 9 Jun 2010 09:05
Thanks Paul

Currently part of the environment I'm working in is constrained to 1.5
but
I'lll check 1.6 out for future ref.

> I haven't checked the API (http://java.sun.com/javase/6/docs/api/
> index.html?java/nio/package-summary.html) but I seem to remember that
> NIO allows you to read floats etc in the byte order of your choice.
> That might avoid the need to do all the bit twiddling yourself.

Cheers
Mike
 |  Next  |  Last
Pages: 1 2 3 4
Prev: Basic custom layout manager question
Next: Sending SMS