Prev: Command to display what Matlab is doing?
Next: Muscle Wrapping over an cylinder - Shortest path
From: Thibault Daoulas on 3 Jun 2010 08:44
Hi all,
I have been reading and trying several ways suggested here to handle growing matrices over loops. My concern starts the same way :
- Unkown size of a matrix, and at each step, a new row is concatenated

What is different now is that I need to look up into this matrix at each step, and update its values.

What I could understand from the previous solutions, including growdata, and growdata2, is that they're store them as cells, and unpack them once the job is finished, but in the meantime their value is not reachable, neither changeable.

Is there a way to work around that ?

An simple example of my goal, which say here, outputs at each step the mean of 2 columns :

while(~finished)
M(step, 1)=[M(1:step-1) 1];
M(step, 2)=[M(1:step-1) -1];
step=step+1;
mean1 = mean(M(:, 1));
mean2 = mean(M(:, 2));
end

Cheers !

Thibault.
From: dpb on 3 Jun 2010 09:36
Thibault Daoulas wrote:
> Hi all,
> I have been reading and trying several ways suggested here to handle
> growing matrices over loops. My concern starts the same way :
> - Unkown size of a matrix, and at each step, a new row is concatenated
>
> What is different now is that I need to look up into this matrix at each
> step, and update its values.
> What I could understand from the previous solutions, including growdata,
> and growdata2, is that they're store them as cells, and unpack them once
> the job is finished, but in the meantime their value is not reachable,
> neither changeable.

You can always access cell values and/or change them as well; it simply
takes dereferencing them.

> Is there a way to work around that ?
>
> An simple example of my goal, which say here, outputs at each step the
> mean of 2 columns :
>
> while(~finished)
> M(step, 1)=[M(1:step-1) 1];
> M(step, 2)=[M(1:step-1) -1];
> step=step+1;
> mean1 = mean(M(:, 1));
> mean2 = mean(M(:, 2));
> end
....

Well, other than the fact that w/o preallocating this is going to get
slower and slower as it grows so if it is more than the small example it
may be an issue, the basics of the above would work...

Trivial keyboard example...

>> x=[];
>> idx=0;
>> while idx<5,x=[x; rand(1,1)];mean(x),idx=idx+1;end
ans =
0.0579
ans =
0.2054
ans =
0.4080
ans =
0.3084
ans =
0.2745
>> x
x =
0.0579
0.3529
0.8132
0.0099
0.1389
>>

--
From: Thibault Daoulas on 3 Jun 2010 11:05
dpb <none(a)non.net> wrote in message <hu8b8i$6uj$1(a)news.eternal-september.org>...
> Thibault Daoulas wrote:
> > Hi all,
> > I have been reading and trying several ways suggested here to handle
> > growing matrices over loops. My concern starts the same way :
> > - Unkown size of a matrix, and at each step, a new row is concatenated
> >
> > What is different now is that I need to look up into this matrix at each
> > step, and update its values.
> > What I could understand from the previous solutions, including growdata,
> > and growdata2, is that they're store them as cells, and unpack them once
> > the job is finished, but in the meantime their value is not reachable,
> > neither changeable.
>
> You can always access cell values and/or change them as well; it simply
> takes dereferencing them.
>
> > Is there a way to work around that ?
> >
> > An simple example of my goal, which say here, outputs at each step the
> > mean of 2 columns :
> >
> > while(~finished)
> > M(step, 1)=[M(1:step-1) 1];
> > M(step, 2)=[M(1:step-1) -1];
> > step=step+1;
> > mean1 = mean(M(:, 1));
> > mean2 = mean(M(:, 2));
> > end
> ...
>
> Well, other than the fact that w/o preallocating this is going to get
> slower and slower as it grows so if it is more than the small example it
> may be an issue, the basics of the above would work...
>
> Trivial keyboard example...
>
> >> x=[];
> >> idx=0;
> >> while idx<5,x=[x; rand(1,1)];mean(x),idx=idx+1;end
> ans =
> 0.0579
> ans =
> 0.2054
> ans =
> 0.4080
> ans =
> 0.3084
> ans =
> 0.2745
> >> x
> x =
> 0.0579
> 0.3529
> 0.8132
> 0.0099
> 0.1389
> >>
>
> --

Well, that's precisely the point. I'd do that if I had not tens of thousands of loops. I'm considering using progressive preallocation, that would extend the memory when I'm reaching an initial predetermined size limit. Another idea is to use other data structures, but I'm fearing it'll just slow down the thing even more, as they're actually based on th array data structure in Matlab... Correct me if I'm wrong !
From: dpb on 3 Jun 2010 11:19
Thibault Daoulas wrote:
> dpb <none(a)non.net> wrote in message
> <hu8b8i$6uj$1(a)news.eternal-september.org>...
....

>> Well, other than the fact that w/o preallocating this is going to get
>> slower and slower as it grows so if it is more than the small example
>> it may be an issue, the basics of the above would work...
....

> Well, that's precisely the point. I'd do that if I had not tens of
> thousands of loops. I'm considering using progressive preallocation,
....

"You can allocate me now or you can allocate me later..."

One's gotta' pay the piper one way or another at one time or in many
small(er) installments.

Certainly preallocation is far faster in general and it's probably
better to over-allocate then truncate/release unneeded rather than try
to be too fine and be force to extend multiple times during the process.
The latter will likely cause temporaries and copies behind the scenes.

--
From: Alan B on 3 Jun 2010 12:31
"Thibault Daoulas" <thibault.daoulas(a)gmail.com> wrote in message <hu8gbg$jq0$1(a)fred.mathworks.com>...
> Well, that's precisely the point. I'd do that if I had not tens of thousands of loops. I'm considering using progressive preallocation, that would extend the memory when I'm reaching an initial predetermined size limit. Another idea is to use other data structures, but I'm fearing it'll just slow down the thing even more, as they're actually based on th array data structure in Matlab... Correct me if I'm wrong !

For progressive reallocation, you can double the array length each time you reach the end of it, rather than using a single preset length. Example:

F=[1 1 0 0];
n=3;
while F(n-1)<1000,
F(n) = F(n-1)+F(n-2); % calculate
n=n+1;
if length(F)==n, F(2*n)=0; end % allocate
end
F = F(1:n-1); % truncate

I don't know much about the Matlab internals, so I'm not sure if this is a great way to do things in Matlab specifically.

Alternatively, growdata doesn't look too complicated, you could modify that to allow you to access the cell data as you go.
 |  Next  |  Last
Pages: 1 2 3 4
Prev: Command to display what Matlab is doing?
Next: Muscle Wrapping over an cylinder - Shortest path