MATLAB Out-of-core Solution
in [Matlab]
|
Prev: The Precision of SVD in matlab
Next: How to use optimset for fminsearch to run Levenberg-Marquardt instead of Nelder-Mead?
From: Neal Martin on 28 Oct 2009 17:22 Can someone explain how MATLAB handles the situation in which a user runs a large job that exceeds the memory capacity of the system (for example, 1M X 1M FFT)? In particular, if the system is configured with a large swap space, is it used? Is the implementation of the FFT done as an out-of-core solution? If so, does it fail under certain system configuration conditions and what are these conditions? Thanks, Neal
From: Ashish Uthama on 29 Oct 2009 13:32
On Wed, 28 Oct 2009 17:22:05 -0400, Neal Martin <nmartin(a)bellsouth.net> wrote: > Can someone explain how MATLAB handles the situation in which a user > runs a large job that exceeds the memory capacity of the system (for > example, 1M X 1M FFT)? In particular, if the system is configured with > a large swap space, is it used? Is the implementation of the FFT done > as an out-of-core solution? If so, does it fail under certain system > configuration conditions and what are these conditions? > > Thanks, Neal Some background info: http://www.mathworks.com/support/tech-notes/1100/1106.html The first section ought to answer your questions on swap space. It might also be worthwhile to have a quick look at 'distributed arrays' supported by the parallel computing toolbox (PCT). (A distributed array does support FFT operations) http://www.mathworks.com/products/parallel-computing/parallel/distarrays.html AFAIK no specific out-of-core solutions are implemented in core MATLAB. You could try harnessing MEMMAPFILE to page your data. Or, if you are using a newer version and your problem statement fits the usecase, use BLOCKPROC (Image processing toolbox) to process your data in discrete blocks. Of course, it will be easier to implement if your operation is easily reducible to processing on blocks and aggregating the results later. |