Prev: Ann: Gnocl gets own URL/
Next: Compiling
From: blacksqr on 28 Feb 2010 15:51
> The feedback you've gotten in 2960042, from Donal, explicitly says
> that it is not a memory leak IF running it twice doesn't extends the
> memory claim, which is what Donal (and everybody else) observe(s)
> normally.

I'm aware that my initial attempt at a bug report was inadequate. The
feedback made clear to me that my understanding of the issue is not
sufficient to frame the problem much better in another bug report.
Rather than waste more of the maintainers' time with additional
flailing in the bug database, I was hoping to get a bit of education
on the topic here.
>
> Since your initial report didn't mention this extra, worrying piece of
> evidence you've just described, Donal set the bug status to Invalid
> +Pending, meaning you have two weeks to provide counter evidence
> before it gets automatically closed (which is reversible too).

As I (inadequately) stated the issue there, Donal's action was
perfectly appropriate. Before I take another stab in the dark and
waste more of his time, I'm hoping to get a little additional insight
from charitable, knowledgeable souls here.
>
> So instead of switching to another communication channel
> (comp.lang.tcl), just deposit this evidence as a comment to the bug.
> We'll take it seriously there.
>
The evidence is difficult to condense and parametrize to bug report
dimensions, since manifestation of the behavior depends strongly on
the configuration of the individual computer. I will try however; but
it would make it a bit easier if I could get an answer to a general
question: if a Tcl procedure takes up almost all of a computer's
available RAM in variables within the procedure's scope, and after all
variables are explicitly unset and the procedure returns Tcl still
holds on to all the previously used RAM so that another invocation of
the procedure causes the interpreter to crash due to a memory
allocation error, would that be considered by the maintainers to be a
bug?

From: Alexandre Ferrieux on 28 Feb 2010 16:36
On Feb 28, 9:51 pm, blacksqr <stephen.hunt...(a)alum.mit.edu> wrote:
>
> The evidence is difficult to condense and parametrize to bug report
> dimensions, since manifestation of the behavior depends strongly on
> the configuration of the individual computer.  I will try however; but

Yes, it means some hard work on your side, for example to tweak your
setup so that it still exhibits the bug with a [string repeat A
250000000] instead of a 250-meg file. But don't that work is wasted:
it is crucial for either your understanding or where the leak lies in
your app, or our understanding of a real case of Tcl-core leak.

> it would make it a bit easier if I could get an answer to a general
> question: if a Tcl procedure takes up almost all of a computer's
> available RAM in variables within the procedure's scope, and after all
> variables are explicitly unset and the procedure returns Tcl still
> holds on to all the previously used RAM so that another invocation of
> the procedure causes the interpreter to crash due to a memory
> allocation error, would that be considered by the maintainers to be a
> bug?

Definitely yes. The high-water mark method means that a repeated,
identical allocation should reuse the same resources without a single
byte of overhead. So again, when you manage to circumvent the need for
a large specific input file, we'll have no rest until Tcl's honor is
washed :)

-Alex


From: blacksqr on 1 Mar 2010 00:23
Thanks for the feedback. I have attached a short demo script to the
original bug ticket which crashes my interpreter and, I hope,
illustrates the problem adequately.
From: Glenn Jackman on 1 Mar 2010 11:30
At 2010-02-28 12:05AM, "blacksqr" wrote:
[...]
> My problem is that I have written a procedure that reads a very large
> file into RAM, processes it, and writes the results to an output
> file. The net RAM usage should be zero after the procedure returns.
> I would like to call this procedure several times, but after the first
> time my computer's RAM is almost depleted, and during the second
> procedure call there's not enough RAM to read the second file into
> memory. Tcl, instead of releasing un-needed memory to allow the file
> read, crashes with a memory allocation error.
[...]

I don't want to derail your issue. Have you considered not reading the
whole file into memory at once? Is it a text file you're processing?
Can you read the file line-by-line, or by chunks?


--
Glenn Jackman
Write a wise saying and your name will live forever. -- Anonymous
From: tomk on 1 Mar 2010 11:48
As you have found out, it is sometime more expedient to recast the
solution to a problem then to solve it. Databases were invented
because computers have limited working memory. If you're processing
the data serially (multi-pass is still serial) then do your own block
management. If the data is random then put it in a database prior to
processing. If you use a database like SqlLite then it uses mapped
file access (I think) which is what you really want and will save you
the work of writing the code.

If you're interested in trying a different solution post a description
of your problem and ask for suggestions.

tomk
First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: Ann: Gnocl gets own URL/
Next: Compiling