Prev: a wiki entry for gfortran
Next: BIND(C) functions in a module error
From: dpb on 6 Aug 2010 14:02
Gideon wrote:
....

> ... I should
> have prefaced this by saying that I'm mostly writing arrays of double
> precision numbers and then reading them into MATLAB.

--

I would recommend switching to "stream" files instead of Fortran
unformatted for the purpose.

This has been standardized in F2003+ and many F95 (I'd almost risk "all"
now; at least all those in the Polyhedron comparison feature/extension
list do) compilers do either by a vendor extension or implementation of
the later features if not all of F2003+.

This doesn't help for previously existing files, of course, nor for
files created out of your control but would certainly ease things otherwise.

--
From: Nick Maclaren on 6 Aug 2010 14:24
In article <4C5C4C0C.6E09(a)nospom.com>, Dave Allured <nospom(a)nospom.com> wrote:
>
>Among the few *modern* unix-like systems and fortran compilers that I
>have used, the unformatted squential record structure is always the
>same:
>
> [length] [data block] [length]
>
> ...
>
>So in systems in modern use, how common are the variations from this
>structure? And what are their structure details, if anyone has the time
>to show a few?

The most recent one I saw differ was the Hitachi SR2201, which was
like that but with a header. As you said, dead easy to convert once
you worked that out. That was in 1995-1998, but Hitachi may still
use the format - and so may some of the other Japanese companies.
Or they may have converted.

My guess is that is damn-near ubiquitous nowadays, in this Year Of
Our Lord MMX.

I no longer have access to an interesting enough range of systems
to be worth posting - sorry.


Regards,
Nick Maclaren.
From: glen herrmannsfeldt on 6 Aug 2010 16:50
Dave Allured <nospom(a)nospom.com> wrote:
(snip)

> So in systems in modern use, how common are the variations from this
> structure? And what are their structure details, if anyone has the time
> to show a few?

I believe that IBM's z/OS still uses the VBS record format that
was used with OS/360 in the early 1960's, before anyone even
thought about unix.

In z/OS files are not considered as an unstructured sequence of
bytes, but as a sequence of records. The OS knows about records,
and so do programs reading and writing them. There are three
record formats (RECFM), F, V, and U, for Fixed length, Variable
length, and Undefined. For F

I believe that IBM's z/OS still uses the VBS record format that
was used with OS/360 in the early 1960's, before anyone even
thought about unix.

In z/OS files are not considered as an unstructured sequence of
bytes, but as a sequence of records. The OS knows about records,
and so do programs reading and writing them. There are three
record formats (RECFM), F, V, and U, for Fixed length, Variable
length, and Undefined. For F and V there is also the choice of
unblocked (one record per physical block) or blocked (more than
one). For U there is only the physical block, no structure known
to the system. V and VB have a four byte block descriptor for
each block, and four byte record descriptor for each record,
but the system keeps track of that, more than on unix-like systems.

For one difference, seeking is done by records, not by bytes,
complicating the implementation of fseek/ftell. Fortran
unformatted uses a special form of VB called VBS (variable,
blocked, spanned) which allows for records longer than a
disk track.

I don't believe that it has been mentioned yet why go to all
the trouble to put these length fields in. Fortran UNFORMATTED
is record oriented. A single READ will read one record, even
with no I/O list, or with fewer list items than on the WRITE.
Also, one can use the BACKSPACE statement to position the file
to an earlier record. Those features would not work without
some kind of length indication.

-- glen

From: Ian Harvey on 7 Aug 2010 01:22
On 2010-08-07 01:28, Gideon wrote:
....
> I should have prefaced this by saying that I'm mostly writing arrays of double
> precision numbers and then reading them into MATLAB.

It faily simple to write a little bit of fortran to read the data
directly into a matlab array. If the compiler that is used to compile
the writing code is also the compiler that compiles the reading code,
then you avoid any uncertainty.

For example (untested - apologies for any errors):


! Compile and link to a shared library (or DLL) such that ReadMyData is
! exported. Link against the various matlab libraries.

MODULE go_read_em
USE, INTRINSIC :: ISO_C_BINDING, ONLY: C_PTR

IMPLICIT NONE
PRIVATE

INTEGER, PARAMETER :: please_implement_newunit = 67
CHARACTER(*), PARAMETER :: my_previously_written_file = 'hello.dat'

! For documentation
PUBLIC :: ReadMyData

!> Integer kind to use for mwSize. We'll assume that we run 32 bit
!! matlab on a 32 bit system or 64 on 64, etc.
INTEGER, PARAMETER, PUBLIC :: mw_size = C_INTPTR_T

!> mxComplexity enum.
ENUM, BIND(C)
!> Identifies an mxArray with no imaginary components.
ENUMERATOR :: mxREAL = 0
!> Identifies an mxArray with imaginary components.
ENUMERATOR :: mxCOMPLEX
END ENUM

INTERFACE
!> Create 2-D, double-precision, floating-point mxArray initialized
to 0.
FUNCTION mxCreateDoubleMatrix(m, n, complexflag) RESULT(mxarray) &
BIND(C, NAME='mxCreateDoubleMatrix')
IMPORT
IMPLICIT NONE
INTEGER(mw_size), INTENT(IN), VALUE :: m
INTEGER(mw_size), INTENT(IN), VALUE :: n
INTEGER(KIND(mxCOMPLEX)), INTENT(IN), VALUE :: complexflag
TYPE(C_PTR) :: mxarray
END FUNCTION mxCreateDoubleMatrix

!> Get a pointer to the real data in an mxArray.
FUNCTION mxGetPr(pm) RESULT(ptr) BIND(C, NAME='mxGetPr')
IMPORT
IMPLICIT NONE
TYPE(C_PTR), INTENT(IN), VALUE :: pm
TYPE(C_PTR) :: ptr
END FUNCTION mxGetPr
END INTERFACE
CONTAINS
FUNCTION ReadMyData() RESULT(pm) &
BIND(C, NAME='ReadMyData')
USE, INTRINSIC :: ISO_C_BINDING, ONLY: C_DOUBLE, &
C_ASSOCIATED, C_F_POINTER
!----
TYPE(C_PTR) :: pm
!----
INTEGER :: m, n
INTEGER(mw_size) :: m_s, n_s
REAL(C_DOUBLE), POINTER :: f_ptr(:,:)
!****
OPEN( UNIT=please_implement_newunit, &
FILE=my_previously_written_file, &
STATUS='OLD', &
ACTION='READ', &
FORM='UNFORMATTED' )
! This bit is robust to a difference in integer kind
READ (please_implement_newunit) m, n
m_s = m
n_s = n
pm = mxCreateDoubleMatrix(m_s, n_s, mxREAL)
IF (.NOT. C_ASSOCIATED(pm)) THEN
CLOSE(please_implement_newunit)
RETURN ! out of memory
END IF
CALL C_F_POINTER(mxGetPr(pm), f_ptr, [m_s, n_s])
! here we assume that you wrote real(c_double) to the file. If not,
! you will need to read into a temporary and then convert.
READ (please_implement_newunit) f_ptr
CLOSE(please_implement_newunit)
END FUNCTION ReadMyData
END MODULE go_read_em


% then in a matlab file read_my_data.m
function d = read_my_data()
loadlibrary( 'my_shared_library_name', @prototypes, ...
'alias', 'a_shorter_alias' );
d = calllib('a_shorter_alias', 'ReadMyData');
unloadlibrary('a_shorter_alias');
end

function [methodinfo,structs,enuminfo,ThunkLibName]=prototypes
ival = {cell(1,1)};
structs = [];
enuminfo = [];
fcnNum = 1;

fcns=struct('name',ival,'calltype',ival,'LHS',ival,'RHS',ival,'alias',ival);
ThunkLibName=[];

fcns.name{fcnNum}='ReadMyData';
fcns.calltype{fcnNum}='cdecl';
fcns.LHS{fcnNum}='MATLAB array';
fcns.RHS{fcnNum}={};

fcnNum = fcnNum + 1;
end
From: Richard Maine on 8 Aug 2010 16:21
dpb <none(a)non.net> wrote:

> Gideon wrote:

> > ... I should
> > have prefaced this by saying that I'm mostly writing arrays of double
> > precision numbers and then reading them into MATLAB.
>
> I would recommend switching to "stream" files instead of Fortran
> unformatted for the purpose.

Good point. I was originally thinking he was talking about handling
existing files from unknown sources, in which case, that doesn't help a
lot, as you say.

For new files meant to be interoperable with non-Fortran environments,
I'd definitely go with stream. Heck, interoperability was the "excuse" I
used to propose stream for the f2003 standard after new proposals were
supposed to be out of order. We already had an approved task of working
on intertoperability and this seemed to fit under that umbrella. I was a
little worried that this "excuse" might not fly, so I tried to do a very
minimal version, hoping that simplicity would help. Somewhat to my
surpise, the only complaints about my proposal were that it didn't go
far enough (so the messier formatted stream got added - I'm still not
sure I like that addition, but that's what a clear majority wanted).

Even for existing unformatted sequential files, if you are trying to
read them using a Fortran compiler that might have different internal
structure conventions, stream is the way to go to "safely" read such a
file of unknown form in order to try to deduce what the form must be.

--
Richard Maine | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle | -- Mark Twain
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: a wiki entry for gfortran
Next: BIND(C) functions in a module error