Using xcorr on a 10512 x 10512 with 6 lags
in [Matlab]
|
From: Wayne King on 9 Jul 2010 13:01 "James " <jfaghm(a)googlemail.com> wrote in message <i17hbg$ktl$1(a)fred.mathworks.com>... > Thank you Godzilla for your response. I'm still getting acquainted with time-series. > Are you referring to the y = decimate(x,r) method? > Thanks. > James > > "Godzilla " <godzilla(a)tokyo.edu> wrote in message <i164oj$og8$1(a)fred.mathworks.com>... > > "Steven Lord" <slord(a)mathworks.com> wrote in message <i153t5$qnf$1(a)fred.mathworks.com>... > > > > > > "James " <jfaghm(a)googlemail.com> wrote in message > > > news:i14stm$k24$1(a)fred.mathworks.com... > > > > Hi I'm trying to get auto-correlation of a 10512 x 120 matrix with 6 > > > > shifts. The result is 13 x 10512^2 matrix which it seems kills my > > > > computer. > > > > > > A 13-by-110502144 (= 10512^2) real full double precision matrix requires a > > > contiguous block of memory of size approximately 10.7 GB. Since you can't > > > work with that large a contiguous block of memory on a 32-bit system I'm > > > assuming your machine is a 64-bit system. Even on such a system, allocating > > > that much memory is going to take TIME. > > > > > > > I'm using a MAC with 32GB of RAM and it's still crashing! > > > > > > If it's crashing reproducibly, please send the crash report (and whatever > > > commands/data are necessary to reproduce the crash) to Technical Support. > > > > > > If it's throwing an error indicating that MATLAB is out of memory, that's > > > different. > > > > > > -- > > > Steve Lord > > > slord(a)mathworks.com > > > comp.soft-sys.matlab (CSSM) FAQ: http://matlabwiki.mathworks.com/MATLAB_FAQ > > > To contact Technical Support use the Contact Us link on > > > http://www.mathworks.com > > > > > > > have you considered low-pass filtering the data and then decimation? Hi James, yes decimate low pass filters the input data and then downsamples. Just be aware that lowpass filtering smooths your data, and the higher the decimation factor, the lower the cutoff frequency for the lowpass filter must be to prevent aliasing. Wayne Wayne
From: James on 16 Jul 2010 19:10
Hi Wayne, Thank you for your reply. Actually, the issue with my data isn't with the frequency but w/ the number of data points. Each time series is only 744 points long, but I have 10512 time series and I want to compute the cross-correlation with 6 month lags. So when I do a a cross-correlation of two time series everything is fine, it's when I try to use all of them that the computer just dies. James "Wayne King" <wmkingty(a)gmail.com> wrote in message <i17kkg$aaq$1(a)fred.mathworks.com>... > "James " <jfaghm(a)googlemail.com> wrote in message <i17hbg$ktl$1(a)fred.mathworks.com>... > > Thank you Godzilla for your response. I'm still getting acquainted with time-series. > > Are you referring to the y = decimate(x,r) method? > > Thanks. > > James > > > > "Godzilla " <godzilla(a)tokyo.edu> wrote in message <i164oj$og8$1(a)fred.mathworks.com>... > > > "Steven Lord" <slord(a)mathworks.com> wrote in message <i153t5$qnf$1(a)fred.mathworks.com>... > > > > > > > > "James " <jfaghm(a)googlemail.com> wrote in message > > > > news:i14stm$k24$1(a)fred.mathworks.com... > > > > > Hi I'm trying to get auto-correlation of a 10512 x 120 matrix with 6 > > > > > shifts. The result is 13 x 10512^2 matrix which it seems kills my > > > > > computer. > > > > > > > > A 13-by-110502144 (= 10512^2) real full double precision matrix requires a > > > > contiguous block of memory of size approximately 10.7 GB. Since you can't > > > > work with that large a contiguous block of memory on a 32-bit system I'm > > > > assuming your machine is a 64-bit system. Even on such a system, allocating > > > > that much memory is going to take TIME. > > > > > > > > > I'm using a MAC with 32GB of RAM and it's still crashing! > > > > > > > > If it's crashing reproducibly, please send the crash report (and whatever > > > > commands/data are necessary to reproduce the crash) to Technical Support. > > > > > > > > If it's throwing an error indicating that MATLAB is out of memory, that's > > > > different. > > > > > > > > -- > > > > Steve Lord > > > > slord(a)mathworks.com > > > > comp.soft-sys.matlab (CSSM) FAQ: http://matlabwiki.mathworks.com/MATLAB_FAQ > > > > To contact Technical Support use the Contact Us link on > > > > http://www.mathworks.com > > > > > > > > > > have you considered low-pass filtering the data and then decimation? > > Hi James, yes decimate low pass filters the input data and then downsamples. Just be aware that lowpass filtering smooths your data, and the higher the decimation factor, the lower the cutoff frequency for the lowpass filter must be to prevent aliasing. > > Wayne > > > > Wayne |