Prev: Proper GRNN use in Matlab
Next: Filter design/implementation question
From: Sean Douglas on 1 Jun 2010 16:48
"John D'Errico" <woodchips(a)rochester.rr.com> wrote in message <hu3nbc$o7m$1(a)fred.mathworks.com>...
> "Sean Douglas" <seanjdouglas(a)hotmail.com> wrote in message <hu3la9$4hp$1(a)fred.mathworks.com>...
> > I just wrote a moving average program, but it is taking about 10 minutes to run and this is too long. Here is the code I am using on a small example.
> >
> > a= 20041126 7.11
> > 20041127 2.33
> > 20041128 5.88
> > 20041129 4.67
> > 20041130 4.66
> > 20041201 3.01
> > 20041203 2.58
> > 20041204 1.35
> >
> >
> > MA=3
> > n=length(a)
> > k=n - MA
> > m=1
> >
> > for q=1:n+1 - MA
> > j=m:MA+m - 1
> > Mavg(q)=sum(a(j,2))/MA
> > m=m+1
> > end
> >
> > so i think this for statement is taking way way too long, expecially when using 3000 columns of data and figuring a Moving Average of 150 rather then the MA=3 I am using in this example to show you guys what I am up to.
> >
>
> I would point out that the correct way to do this is with
> conv or filter. These solutions are far better choices than
> the cumsum trick, which will have precision issues.
>
> However, nobody has pointed out why your code is so
> miserably slow. Learn to preallocate that array! It is
> because you have not preallocated it to the final size
> that your loop is slow.
>
> John

thank you John and everyone else I am looking up "conv "and "preallocate array" so I can get this program working faster.

thanks
From: Sean Douglas on 1 Jun 2010 16:55
"John D'Errico" <woodchips(a)rochester.rr.com> wrote in message <hu3nbc$o7m$1(a)fred.mathworks.com>...
> "Sean Douglas" <seanjdouglas(a)hotmail.com> wrote in message <hu3la9$4hp$1(a)fred.mathworks.com>...
> > I just wrote a moving average program, but it is taking about 10 minutes to run and this is too long. Here is the code I am using on a small example.
> >
> > a= 20041126 7.11
> > 20041127 2.33
> > 20041128 5.88
> > 20041129 4.67
> > 20041130 4.66
> > 20041201 3.01
> > 20041203 2.58
> > 20041204 1.35
> >
> >
> > MA=3
> > n=length(a)
> > k=n - MA
> > m=1
> >
> > for q=1:n+1 - MA
> > j=m:MA+m - 1
> > Mavg(q)=sum(a(j,2))/MA
> > m=m+1
> > end
> >
> > so i think this for statement is taking way way too long, expecially when using 3000 columns of data and figuring a Moving Average of 150 rather then the MA=3 I am using in this example to show you guys what I am up to.
> >
>
> I would point out that the correct way to do this is with
> conv or filter. These solutions are far better choices than
> the cumsum trick, which will have precision issues.
>
> However, nobody has pointed out why your code is so
> miserably slow. Learn to preallocate that array! It is
> because you have not preallocated it to the final size
> that your loop is slow.
>
> John

Hey John, Should I be using both conv and preallocating the arry, or would I just do one or the other?

thanks
From: Walter Roberson on 1 Jun 2010 17:05
Sean Douglas wrote:

> Hey John, Should I be using both conv and preallocating the arry, or
> would I just do one or the other?

If you use conv, you will likely produce the entire answer in one command and
so will not need preallocation for this task (but might need it for whatever
you are going to do with the averages.)
From: John D'Errico on 1 Jun 2010 17:36
"Sean Douglas" <seanjdouglas(a)hotmail.com> wrote in message <hu3s3p$igi$1(a)fred.mathworks.com>...
> "John D'Errico" <woodchips(a)rochester.rr.com> wrote in message <hu3nbc$o7m$1(a)fred.mathworks.com>...
> > "Sean Douglas" <seanjdouglas(a)hotmail.com> wrote in message <hu3la9$4hp$1(a)fred.mathworks.com>...
> > > I just wrote a moving average program, but it is taking about 10 minutes to run and this is too long. Here is the code I am using on a small example.
> > >
> > > a= 20041126 7.11
> > > 20041127 2.33
> > > 20041128 5.88
> > > 20041129 4.67
> > > 20041130 4.66
> > > 20041201 3.01
> > > 20041203 2.58
> > > 20041204 1.35
> > >
> > >
> > > MA=3
> > > n=length(a)
> > > k=n - MA
> > > m=1
> > >
> > > for q=1:n+1 - MA
> > > j=m:MA+m - 1
> > > Mavg(q)=sum(a(j,2))/MA
> > > m=m+1
> > > end
> > >
> > > so i think this for statement is taking way way too long, expecially when using 3000 columns of data and figuring a Moving Average of 150 rather then the MA=3 I am using in this example to show you guys what I am up to.
> > >
> >
> > I would point out that the correct way to do this is with
> > conv or filter. These solutions are far better choices than
> > the cumsum trick, which will have precision issues.
> >
> > However, nobody has pointed out why your code is so
> > miserably slow. Learn to preallocate that array! It is
> > because you have not preallocated it to the final size
> > that your loop is slow.
> >
> > John
>
> Hey John, Should I be using both conv and preallocating the arry, or would I just do one or the other?

When you append new elements to an array in a loop,
increasing the size of the array in every pass, matlab
must reallocate new (more) memory for that array every
time. This is what takes time.

When you create the result using conv, there is no
need to preallocate, since conv creates the entire array
in one step.

So loops that grow arrays should preallocate.

John
From: Steve Amphlett on 1 Jun 2010 17:52
"John D'Errico" <woodchips(a)rochester.rr.com> wrote in message <hu3ug4$2e9$1(a)fred.mathworks.com>...
> "Sean Douglas" <seanjdouglas(a)hotmail.com> wrote in message <hu3s3p$igi$1(a)fred.mathworks.com>...
> > "John D'Errico" <woodchips(a)rochester.rr.com> wrote in message <hu3nbc$o7m$1(a)fred.mathworks.com>...
> > > "Sean Douglas" <seanjdouglas(a)hotmail.com> wrote in message <hu3la9$4hp$1(a)fred.mathworks.com>...
> > > > I just wrote a moving average program, but it is taking about 10 minutes to run and this is too long. Here is the code I am using on a small example.
> > > >
> > > > a= 20041126 7.11
> > > > 20041127 2.33
> > > > 20041128 5.88
> > > > 20041129 4.67
> > > > 20041130 4.66
> > > > 20041201 3.01
> > > > 20041203 2.58
> > > > 20041204 1.35
> > > >
> > > >
> > > > MA=3
> > > > n=length(a)
> > > > k=n - MA
> > > > m=1
> > > >
> > > > for q=1:n+1 - MA
> > > > j=m:MA+m - 1
> > > > Mavg(q)=sum(a(j,2))/MA
> > > > m=m+1
> > > > end
> > > >
> > > > so i think this for statement is taking way way too long, expecially when using 3000 columns of data and figuring a Moving Average of 150 rather then the MA=3 I am using in this example to show you guys what I am up to.
> > > >
> > >
> > > I would point out that the correct way to do this is with
> > > conv or filter. These solutions are far better choices than
> > > the cumsum trick, which will have precision issues.
> > >
> > > However, nobody has pointed out why your code is so
> > > miserably slow. Learn to preallocate that array! It is
> > > because you have not preallocated it to the final size
> > > that your loop is slow.
> > >
> > > John
> >
> > Hey John, Should I be using both conv and preallocating the arry, or would I just do one or the other?
>
> When you append new elements to an array in a loop,
> increasing the size of the array in every pass, matlab
> must reallocate new (more) memory for that array every
> time. This is what takes time.
>
> When you create the result using conv, there is no
> need to preallocate, since conv creates the entire array
> in one step.
>
> So loops that grow arrays should preallocate.
>
> John

If the filter has equal coefficients, consider writing a MEX function that uses a moving buffer. The full convolution will do many more floating point ops than required. It's a good learning exercise for newcomers to MEX.
First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: Proper GRNN use in Matlab
Next: Filter design/implementation question