Prev: How can I find the mean between to points of normal distribution?
Next: Merging of plots / figures
From: Joseph Cohen on 2 Aug 2010 14:16
Hello,

I have a 500x100 matrix containing doubles (called 'matrix') which I reshape to a 1X50,000 matrix (called 'reshaped_matrix') I then typecast it to uint8 ('uint_matrix') and finally convert it to a hex string with dec2hex and store the output in hex_matrix.

In addition I save the same 500x100 matrix as a mat file. I then read that mat file with fread and store the output in 'mat_data'. Finally, using dec2hex I transform that output into a hex string and store it in 'hex_mat_data'.

For some reason hex_mat_data takes up less space than hex_matrix even though mat_data takes up more space that uint_matrix. Why is this? Also, Why does dec2hex make the variable take up more space when performed on uint_matrix but shrinks the variable when performed on the output from fread.

Finally, can I get my final hex string to be smaller without saving it as a matfile first? I want to avoid the time it takes to save it as a mat and also I don't need the extra information stored in a matfile in hex form.

Thank you!

*******************************************************************
Below is the result of the whos function for all variables mentioned above:


Name Size Bytes Class Attributes
matrix 500x100 400000 double
reshaped_matrix 50000x1 400000 double
uint_matrix 400000x1 400000 uint8
hex_matrix 400000x2 1600000 char



mat_data 96177x1 769416 double
hex_mat_data 96177x2 384708 char

Also here is the first few values of 'matrix', to give you an idea of what I'm working with. The rest of 'matrix' is also double values between 0 and 1:
0.4858 0.5640 0.9546 0.7568 0.6616 0.4761 0.7373 0.6689
From: us on 2 Aug 2010 14:20
On Aug 2, 8:16 pm, "Joseph Cohen" <hypergra...(a)hotmail.com> wrote:
> Hello,
>
> I have a 500x100 matrix containing doubles (called 'matrix') which I reshape to a 1X50,000 matrix (called 'reshaped_matrix') I then typecast it to uint8 ('uint_matrix') and finally convert it to a hex string with dec2hex and store the output in hex_matrix.
>
> In addition I save the same 500x100 matrix as a mat file. I then read that mat file with fread and store the output in 'mat_data'. Finally, using dec2hex I transform that output into a hex string and store it in 'hex_mat_data'.
>
> For some reason hex_mat_data takes up less space than hex_matrix even though mat_data takes up more space that uint_matrix. Why is this? Also, Why does dec2hex make the variable take up more space when performed on uint_matrix but shrinks the variable when performed on the output from fread.
>
> Finally, can I get my final hex string to be smaller without saving it as a matfile first? I want to avoid the time it takes to save it as a mat and also I don't need the extra information stored in a matfile in hex form.
>
> Thank you!
>
> *******************************************************************
> Below is the result of the whos function for all variables mentioned above:
>
>  Name                      Size               Bytes  Class     Attributes
>   matrix                  500x100             400000  double              
>   reshaped_matrix       50000x1               400000  double              
>   uint_matrix          400000x1               400000  uint8    
>   hex_matrix           400000x2              1600000  char                
>
> mat_data              96177x1               769416  double              
>   hex_mat_data          96177x2               384708  char                
>
> Also here is the first few values of 'matrix', to give you an idea of what I'm working with. The rest of 'matrix' is also double values between 0 and 1:
>          0.4858    0.5640    0.9546    0.7568    0.6616    0.4761    0.7373    0.6689

huh(?!)...
FREAD a MAT-file(?)...
did you compare the results(?)...

us
From: Joseph Cohen on 2 Aug 2010 17:54
us <us(a)neurol.unizh.ch> wrote in message <3b6d0608-e44a-4d8d-b457-3c99a3e8b894(a)t5g2000prd.googlegroups.com>...
> On Aug 2, 8:16 pm, "Joseph Cohen" <hypergra...(a)hotmail.com> wrote:
> > Hello,
> >
> > I have a 500x100 matrix containing doubles (called 'matrix') which I reshape to a 1X50,000 matrix (called 'reshaped_matrix') I then typecast it to uint8 ('uint_matrix') and finally convert it to a hex string with dec2hex and store the output in hex_matrix.
> >
> > In addition I save the same 500x100 matrix as a mat file. I then read that mat file with fread and store the output in 'mat_data'. Finally, using dec2hex I transform that output into a hex string and store it in 'hex_mat_data'.
> >
> > For some reason hex_mat_data takes up less space than hex_matrix even though mat_data takes up more space that uint_matrix. Why is this? Also, Why does dec2hex make the variable take up more space when performed on uint_matrix but shrinks the variable when performed on the output from fread.
> >
> > Finally, can I get my final hex string to be smaller without saving it as a matfile first? I want to avoid the time it takes to save it as a mat and also I don't need the extra information stored in a matfile in hex form.
> >
> > Thank you!
> >
> > *******************************************************************
> > Below is the result of the whos function for all variables mentioned above:
> >
> >  Name                      Size               Bytes  Class     Attributes
> >   matrix                  500x100             400000  double              
> >   reshaped_matrix       50000x1               400000  double              
> >   uint_matrix          400000x1               400000  uint8    
> >   hex_matrix           400000x2              1600000  char                
> >
> > mat_data              96177x1               769416  double              
> >   hex_mat_data          96177x2               384708  char                
> >
> > Also here is the first few values of 'matrix', to give you an idea of what I'm working with. The rest of 'matrix' is also double values between 0 and 1:
> >          0.4858    0.5640    0.9546    0.7568    0.6616    0.4761    0.7373    0.6689
>
> huh(?!)...
> FREAD a MAT-file(?)...
> did you compare the results(?)...
>

I'm doing FREAD on a Mat file because I need to store hex strings in a database. Thus, I need either the raw matrix or the mat file containing it in hex string form.

I'm not sure what you mean by "compare the results", but I was able to retrieve the original form from the final form of each of the two methods I used. I also used whos and compared all the variable properties which I included in my last post.

Any ideas on why they've got different sizes?
From: Jan Simon on 2 Aug 2010 18:08
Dear Joseph,

please show us, how you create the MAT file and how you read it with FREAD. As Urs has mentioned: Usually the output of FREAD differs from the input of SAVE.

Kind regards, Jan
From: James Tursa on 2 Aug 2010 18:40
"Joseph Cohen" <hypergrapes(a)hotmail.com> wrote in message <i3721n$pa8$1(a)fred.mathworks.com>...
> Hello,
>
> I have a 500x100 matrix containing doubles (called 'matrix') which I reshape to a 1X50,000 matrix (called 'reshaped_matrix') I then typecast it to uint8 ('uint_matrix') and finally convert it to a hex string with dec2hex and store the output in hex_matrix.
>
> In addition I save the same 500x100 matrix as a mat file. I then read that mat file with fread and store the output in 'mat_data'. Finally, using dec2hex I transform that output into a hex string and store it in 'hex_mat_data'.
>
> For some reason hex_mat_data takes up less space than hex_matrix even though mat_data takes up more space that uint_matrix. Why is this? Also, Why does dec2hex make the variable take up more space when performed on uint_matrix but shrinks the variable when performed on the output from fread.
>
> Finally, can I get my final hex string to be smaller without saving it as a matfile first? I want to avoid the time it takes to save it as a mat and also I don't need the extra information stored in a matfile in hex form.
>
> Thank you!
>
> *******************************************************************
> Below is the result of the whos function for all variables mentioned above:
>
>
> Name Size Bytes Class Attributes
> matrix 500x100 400000 double
> reshaped_matrix 50000x1 400000 double
> uint_matrix 400000x1 400000 uint8
> hex_matrix 400000x2 1600000 char
>
>
>
> mat_data 96177x1 769416 double
> hex_mat_data 96177x2 384708 char
>
> Also here is the first few values of 'matrix', to give you an idea of what I'm working with. The rest of 'matrix' is also double values between 0 and 1:
> 0.4858 0.5640 0.9546 0.7568 0.6616 0.4761 0.7373 0.6689

As others have already requested, I would stress that we need to see your code (how exactly are you doing the conversions & reading & writing etc) in order to answer your questions. But keep in mind the following:

- A char in MATLAB takes 2 bytes.
- A single byte of 8 bits typically takes two hex characters to represent.
- The MATLAB MAT file format can use compression, which can mislead you into how much memory the data will take when loaded if you are just looking at file size.
- fread and fwrite can do conversions on the data during the process, which can change the memory footprint of the data.

James Tursa
 |  Next  |  Last
Pages: 1 2
Prev: How can I find the mean between to points of normal distribution?
Next: Merging of plots / figures