Prev: Discrete Wavelet Frame
Next: Speed change of SUM
From: Srimal Jayawardena on 22 Feb 2010 03:38
Hi

I need to pass a few large matrices (50,000x3 elements each) across my matlab functions.

1) I dont want to pass copies of them as variables as my functions are called about 10000 times and I dont want to fragment memory

2) I dont want to read from disk within each function call as it will slow down my code

Can I

1) Use a pointer to this matrix and send the pointer as an argument to my functions? If so whats the command please?

2) I could think of using a global variable for these big matrices. Not sure if its a good idea though. Is there a better way?

Thanks

Srimal.
From: Rune Allnor on 22 Feb 2010 04:10
On 22 Feb, 09:38, "Srimal Jayawardena" <srim...(a)gmail.com> wrote:
> Hi
>
> I need to pass a few large matrices (50,000x3 elements each) across my matlab functions.

Assuming those elements are real numbers on double precision
binary format, that's less than 1.5 MBytes of data. Not large
at all.

> 1) I dont want to pass copies of them as variables as my functions are called about 10000 times and I dont want to fragment memory

Do you modify the matrices inside our functions?
If 'no', chances are that they will not be copied
to a new memory location.

> 2) I dont want to read from disk within each function call

It depends on what you do to your data. If you have
10000 different matrices and call the function once
on each matrix, then the total data set is some 15 GBytes.
In that case, you *will* have to flush your data back
and forth to disk.

But you say you have 'a few' matrices. If you call the
function 10000 times on a small number of matrices, then
there is no need to consider disk IO.

> as it will slow down my code

Forget about that. The first priority is to get something
up and running that actually works. Only when you have
a working program does it make sense to start talking
about speed.

> Can I
>
> 1) Use a pointer to this matrix and send the pointer as an argument to my functions? If so whats the command please?

Can't be done in matlab. Matlab attempts to be smart,
though, not copying the variables if they are not modified
inside the function.

> 2) I could think of using a global variable for these big matrices. Not sure if its a good idea though.

It isn't. Global variables is a mess, so don't go there.

As for the matrices, if there are 10000 unique matrices then
you will not have sufficient memory space for them anyway.

> Is there a better way?

Again, assuming you are dealing with numbers, the matrices
are less than 1.5 MBytes each, so there is no real problem
in this particular case.

Code up something that works. When you are happy that it
works, start the thing, and let the computer work over night.
No need for you to babysit it while it crunches the numbers.

Apart from that, these kinds of considerations is a key argument
why people stick with compiled languages like Fortran, C, and
C++ that offer far better control of resources, and far better
efficiency.

At the cost of requiring trained designers and programmers.

Rune
 | 
Pages: 1
Prev: Discrete Wavelet Frame
Next: Speed change of SUM