mxArray serialization
in [Matlab]
|
From: Ron Klein on 22 Apr 2010 11:23 I'm using Matlab API in C++. I open a mat file (using matOpen function), and I read the first (and only variable) in it to an mxArray pointer (mxArray*). I'd like to copy the memory representation of this mxArray in order to reuse it later on, without the overhead of opening the matfile again. ============================ relevant code: MATFile *pmat; const char* name=NULL; mxArray *pa; pmat = matOpen("myfile.mat", "r"); pa = matGetNextVariable(pmat, &name); // how do I serialize "pa"? ============================ I saw this post: http://www.mathworks.co.uk/matlabcentral/newsreader/view_thread/141797 The post describes a mothod of serialization, and then deserialization, from an mxArray to "regular bytes" (unsigned chars in C/C++). However, none of the links there are available. Is there any other post/documentation for serialization and deserialization? Thanks!
From: James Tursa on 22 Apr 2010 11:39 "Ron Klein" <ron(a)xsights.com> wrote in message <hqppkq$fct$1(a)fred.mathworks.com>... > I'm using Matlab API in C++. > I open a mat file (using matOpen function), and I read the first (and only variable) in it to an mxArray pointer (mxArray*). > I'd like to copy the memory representation of this mxArray in order to reuse it later on, without the overhead of opening the matfile again. > ============================ > relevant code: > MATFile *pmat; > const char* name=NULL; > mxArray *pa; > > pmat = matOpen("myfile.mat", "r"); > pa = matGetNextVariable(pmat, &name); > > // how do I serialize "pa"? > ============================ > > I saw this post: > http://www.mathworks.co.uk/matlabcentral/newsreader/view_thread/141797 > The post describes a mothod of serialization, and then deserialization, from an mxArray to "regular bytes" (unsigned chars in C/C++). However, none of the links there are available. > Is there any other post/documentation for serialization and deserialization? > Thanks! Are you simply looking for code that puts all of the parts of the mxArray (type, number of dimensions, dimensions, real data, imag data, etc.) into one contiguous block of memory? Do you need to handle cell arrays and struct arrays or just numeric arrays? Is this ultimately intended for writing to a binary file and then reading it in later on? Or what? James Tursa
From: Ron Klein on 22 Apr 2010 15:49 "James Tursa" <aclassyguy_with_a_k_not_a_c(a)hotmail.com> wrote in message <hqpqjb$42n$1(a)fred.mathworks.com>... > "Ron Klein" <ron(a)xsights.com> wrote in message <hqppkq$fct$1(a)fred.mathworks.com>... > > I'm using Matlab API in C++. > > I open a mat file (using matOpen function), and I read the first (and only variable) in it to an mxArray pointer (mxArray*). > > I'd like to copy the memory representation of this mxArray in order to reuse it later on, without the overhead of opening the matfile again. > > ============================ > > relevant code: > > MATFile *pmat; > > const char* name=NULL; > > mxArray *pa; > > > > pmat = matOpen("myfile.mat", "r"); > > pa = matGetNextVariable(pmat, &name); > > > > // how do I serialize "pa"? > > ============================ > > > > I saw this post: > > http://www.mathworks.co.uk/matlabcentral/newsreader/view_thread/141797 > > The post describes a mothod of serialization, and then deserialization, from an mxArray to "regular bytes" (unsigned chars in C/C++). However, none of the links there are available. > > Is there any other post/documentation for serialization and deserialization? > > Thanks! > > Are you simply looking for code that puts all of the parts of the mxArray (type, number of dimensions, dimensions, real data, imag data, etc.) into one contiguous block of memory? Do you need to handle cell arrays and struct arrays or just numeric arrays? Is this ultimately intended for writing to a binary file and then reading it in later on? Or what? > > James Tursa Bottom line: yes, I am looking for code (if such exists) in C/C++ that copies my mxArray (which is, in fact, a non-trivial data structure) into one contiguous memory block. Then I need the deserializing function, that gets a memory block and returns an mxArray (pointer). As for the data structure itself, I'm not sure about its inner structure. All I know, so far, is that it's not trivial, and it has a few fields. The detailed context: this memory block will be cached in the computer's memory. If it is stored in the memory, there's no need to access files stored in the disk, so we get a performance boost. Thanks for the help so far!
From: James Tursa on 22 Apr 2010 16:07 "Ron Klein" <ron(a)xsights.com> wrote in message <hqq981$k7k$1(a)fred.mathworks.com>... > > Bottom line: yes, I am looking for code (if such exists) in C/C++ that copies my mxArray (which is, in fact, a non-trivial data structure) into one contiguous memory block. Then I need the deserializing function, that gets a memory block and returns an mxArray (pointer). > > As for the data structure itself, I'm not sure about its inner structure. All I know, so far, is that it's not trivial, and it has a few fields. > > The detailed context: this memory block will be cached in the computer's memory. If it is stored in the memory, there's no need to access files stored in the disk, so we get a performance boost. > > Thanks for the help so far! Rather than invent your own format, why not just use the mat file format and write the output to memory instead of a file? The mat file format is public. I don't know of any code out there to do this generically, however. A web search might be in order. One example of this for writing a double array and a cell array to a file can be found here: http://www.mathworks.com/matlabcentral/fileexchange/26731-portable-matfile-exporter-in-c I imagine this technique could be modified to write to a memory block instead of a file. The mat file format can be found here: http://www.mathworks.com/access/helpdesk/help/pdf_doc/matlab/matfile_format.pdf For your purposes you wouldn't want to do any of the compressed stuff, so that will make it a bit easier. James Tursa
From: Ron Klein on 23 Apr 2010 08:53 "James Tursa" <aclassyguy_with_a_k_not_a_c(a)hotmail.com> wrote in message <hqqa9n$7j0$1(a)fred.mathworks.com>... > "Ron Klein" <ron(a)xsights.com> wrote in message <hqq981$k7k$1(a)fred.mathworks.com>... > > > > Bottom line: yes, I am looking for code (if such exists) in C/C++ that copies my mxArray (which is, in fact, a non-trivial data structure) into one contiguous memory block. Then I need the deserializing function, that gets a memory block and returns an mxArray (pointer). > > > > As for the data structure itself, I'm not sure about its inner structure. All I know, so far, is that it's not trivial, and it has a few fields. > > > > The detailed context: this memory block will be cached in the computer's memory. If it is stored in the memory, there's no need to access files stored in the disk, so we get a performance boost. > > > > Thanks for the help so far! > > Rather than invent your own format, why not just use the mat file format and write the output to memory instead of a file? The mat file format is public. I don't know of any code out there to do this generically, however. A web search might be in order. One example of this for writing a double array and a cell array to a file can be found here: > > http://www.mathworks.com/matlabcentral/fileexchange/26731-portable-matfile-exporter-in-c > > I imagine this technique could be modified to write to a memory block instead of a file. The mat file format can be found here: > > http://www.mathworks.com/access/helpdesk/help/pdf_doc/matlab/matfile_format.pdf > > For your purposes you wouldn't want to do any of the compressed stuff, so that will make it a bit easier. > > James Tursa Hi again, FOA, thanks for the help so far. To the issue itself: The main problem I'm facing is that I cannot just perform memcpy (memory copy) from the mxArray pointer I have to a new memory block. That's because there's no documentation about the *memory* representation of the mxArray. For *simple* data types, such as a matrix of doubles, it seems that there is a solution, using mxGetPr function. Other data types are also supported, with the equivalent functions (for instance, mxGetPi for integers). Like this snippet: memcpy((void *)mxGetPr(mtlbDataArray),(void *)data,sizeof(double) * N); However, if the mxArray pointer holds something more complex than a matrix, like a structure that has a class in one of its fields, and the class has a double matrix and an integer matrix... I might be wrong here about the terms, but here's the point: I know *nothing* about the memory representation of a non trivial mxArray. I found no documentation about it. All I can say for sure, is that if I use matGetNextVariable I have a proper mxArray pointer. But matGetNextVariable must use a MATFile pointer as input, which is there only if I use matOpen function. And this matOpen function *must* get a string (a char*) to a file name, which should exist on the disk. So basically, I know how to read a simple matrix from a mat file, and store it in the memory for later usage as an mxArray. This is because I know that memcpy works in such scenarios. But I don't know how to clone/duplicate complex data structures. I don't know if the memory allocated by these structures is in one contiguous block or not, for instance. I found no documentation about it so far. As for the the post you mentioned ( http://www.mathworks.com/matlabcentral/fileexchange/26731-portable-matfile-exporter-in-c ), it describes simple data types only. And again, thanks for the help so far. Ron
|
Next
|
Last
Pages: 1 2 3 4 Prev: combine two matrices Next: Timing of the inversion of a sparse matrix |