Prev: Volcanic Ash Cloud over Europe in 14 lines of Mathematica
Next: Easy way to label graphics grids?
From: Vince Virgilio on 20 Apr 2010 05:50
On Apr 19, 5:59 am, ValeX <rjovale(a)gmail.com> wrote:
> i'm running a Do loop over a grid of points, say 100x100.
> the problem is that for every grid point the ram used increases by 1.4
> MB and so i cant actually finish the evaluation
> [100x100x1.4 is roughly 14GB]
>
> of all the outputs i need only 3 numbers per grid point and so i tried
> the following:
>
> ClearAll[all the variables i used and i dont need anymore];
> ClearSystemCache[];
> Unprotect[In, Out]; Clear[In, Out]; Protect[In, Out];
>
> but the memory does not go down...
> even if i do Remove["Global`*"] nothing happens.
>
> in the Do loop i do a bunch of things like FindRoot, Interpolation,
> NDSolve, NIntegrate, ..
> is it that maybe i've got to clear some specific place other than the
> history and Global?

Some oblique advice . . .

Mathematica has internal caches that cannot be cleared by the user in
a single kernel session. And they grow, and grow. Not to the casual
observer or user, but yet they grow.

They work in such a way that repeated evaluations of essentially the
same expression (say, a largish simulation) irreversibly consume
significant memory. For my problems, this drove me to partition the
"grid", run each partition, save to disk, and restart the kernel,
between partitions.

Fare well,

Vince Virgilio

From: Zach Bjornson on 20 Apr 2010 05:51
Unfortunately there's no way to get the kernel to deallocate memory.
I've asked for this functionality to be included in the next version,
but I don't think it has. If you use version 7, there's a trick I use:

Evaluate your memory-munching loop on one "parallel kernel."
results = ParallelEvaluate[(*your do loop*), First(a)Kernels[]]

At this point the slave kernel will be the one using all the memory, and
you can use
CloseKernels[]
to free up all of the memory it had allocated.

Since this will still use too much memory in one shot, you can enclose
the above pair of operations in, say, Table or another loop, and use
that outer loop to adjust the inner loop's bounds, making sure that only
those three data points per grid point are saved in the outer loop.

Setting
$HistoryLength = 0
may help some as well, but will not solve your problem.

-Zach

On 4/19/2010 5:59 AM, ValeX wrote:
> i'm running a Do loop over a grid of points, say 100x100.
> the problem is that for every grid point the ram used increases by 1.4
> MB and so i cant actually finish the evaluation
> [100x100x1.4 is roughly 14GB]
>
> of all the outputs i need only 3 numbers per grid point and so i tried
> the following:
>
> ClearAll[all the variables i used and i dont need anymore];
> ClearSystemCache[];
> Unprotect[In, Out]; Clear[In, Out]; Protect[In, Out];
>
> but the memory does not go down...
> even if i do Remove["Global`*"] nothing happens.
>
> in the Do loop i do a bunch of things like FindRoot, Interpolation,
> NDSolve, NIntegrate, ..
> is it that maybe i've got to clear some specific place other than the
> history and Global?
>
>

From: dh on 20 Apr 2010 05:52
On 19.04.2010 11:59, ValeX wrote:
> i'm running a Do loop over a grid of points, say 100x100.
> the problem is that for every grid point the ram used increases by 1.4
> MB and so i cant actually finish the evaluation
> [100x100x1.4 is roughly 14GB]
>
> of all the outputs i need only 3 numbers per grid point and so i tried
> the following:
>
> ClearAll[all the variables i used and i dont need anymore];
> ClearSystemCache[];
> Unprotect[In, Out]; Clear[In, Out]; Protect[In, Out];
>
> but the memory does not go down...
> even if i do Remove["Global`*"] nothing happens.
>
> in the Do loop i do a bunch of things like FindRoot, Interpolation,
> NDSolve, NIntegrate, ..
> is it that maybe i've got to clear some specific place other than the
> history and Global?
>
Mathematica version 7.0.1
Hi,
ClearAll does not seem to work with a pattern. Consider:
d = RandomReal[{-1, 1}, 10^7];
MemoryInUse[]
ClearAll["Global`*"]
MemoryInUse[]
this does not clear the memory. However, If you use
ClearAll[d]
it works.
Daniel



--

Daniel Huber
Metrohm Ltd.
Oberdorfstr. 68
CH-9100 Herisau
Tel. +41 71 353 8585, Fax +41 71 353 8907
E-Mail:<mailto:dh(a)metrohm.com>
Internet:<http://www.metrohm.com>


First  |  Prev  | 
Pages: 1 2
Prev: Volcanic Ash Cloud over Europe in 14 lines of Mathematica
Next: Easy way to label graphics grids?