Prev: On a somewhat disappointing correspondence
Next: [HACKERS] Partitioning/inherited tables vs FKs
From: Nikhil Sontakke on 6 May 2010 03:47
Hi,

Continuing on this:

Can someone please explain why we do not reset the expression context
if an SRF is involved during execution? Once the current result from
the SRF has been consumed, I would think that the
ecxt_per_tuple_memory context should be reset. As its name suggests,
it is supposed to a per tuple context and is not meant to be
long-lived. To test this out I shifted the call to ResetExprContext to
just before returning from the SRF inside ExecResult and I do not see
the memleak at all. Patch attached with this mail.

The SRF has its own long-lived "SRF multi-call context" anyways. And
AIUI, SRFs return tuples one-by-one or do we materialize the same into
a tuplestore in some cases?

Regards,
Nikhils

On Wed, May 5, 2010 at 7:23 PM, Nikhil Sontakke
<nikhil.sontakke(a)enterprisedb.com> wrote:
> Hi,
>
> I saw this behavior with latest GIT head:
>
> create table xlarge(val numeric(19,0));
> insert into xlarge values(generate_series(1,5));
>
> The above generate series will return an int8 which will then be
> casted to numeric (via int8_to_numericvar) before being inserted into
> the table. I observed that the ExprContext memory associated with
> econtext->ecxt_per_tuple_memory is slowly bloating up till the end of
> the insert operation.
>
> This becomes significant the moment we try to insert a significant
> number of entries using this SRF. I can see the memory being consumed
> by the PG backend slowly grow to a large percentage.
>
> I see that the executor (take ExecResult as an example) does not reset
> the expression context early if an SRF is churning out tuples. What
> could be a good way to fix this?
>
> Regards,
> Nikhils
> --
> http://www.enterprisedb.com
>



--
http://www.enterprisedb.com
From: Nikhil Sontakke on 6 May 2010 08:13
> Patch attached with this mail.
>

The previous patch was WIP. Please take a look at this one instead. I
hope this handles the cases where results are not just base datums but
palloc'ed datums like string types too.

Regards,
Nikhils

> The SRF has its own long-lived "SRF multi-call context" anyways. And
> AIUI, SRFs return tuples one-by-one or do we materialize the same into
> a tuplestore in some cases?
>
> Regards,
> Nikhils
>
> On Wed, May 5, 2010 at 7:23 PM, Nikhil Sontakke
> <nikhil.sontakke(a)enterprisedb.com> wrote:
>> Hi,
>>
>> I saw this behavior with latest GIT head:
>>
>> create table xlarge(val numeric(19,0));
>> insert into xlarge values(generate_series(1,5));
>>
>> The above generate series will return an int8 which will then be
>> casted to numeric (via int8_to_numericvar) before being inserted into
>> the table. I observed that the ExprContext memory associated with
>> econtext->ecxt_per_tuple_memory is slowly bloating up till the end of
>> the insert operation.
>>
>> This becomes significant the moment we try to insert a significant
>> number of entries using this SRF. I can see the memory being consumed
>> by the PG backend slowly grow to a large percentage.
>>
>> I see that the executor (take ExecResult as an example) does not reset
>> the expression context early if an SRF is churning out tuples. What
>> could be a good way to fix this?
>>
>> Regards,
>> Nikhils
>> --
>> http://www.enterprisedb.com
>>
>
>
>
> --
> http://www.enterprisedb.com
>



--
http://www.enterprisedb.com
From: Tom Lane on 6 May 2010 10:27
Nikhil Sontakke <nikhil.sontakke(a)enterprisedb.com> writes:
> Can someone please explain why we do not reset the expression context
> if an SRF is involved during execution?

Consider
srf(foo(col))
where foo returns a pass-by-reference datatype. Your proposed patch
would cut the knees out from under argument values that the SRF could
reasonably expect to still be there on subsequent calls.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Nikhil Sontakke on 7 May 2010 02:14
Hi,

>> Can someone please explain why we do not reset the expression context
>> if an SRF is involved during execution?
>
> Consider
>        srf(foo(col))
> where foo returns a pass-by-reference datatype.  Your proposed patch
> would cut the knees out from under argument values that the SRF could
> reasonably expect to still be there on subsequent calls.
>

Yeah this is my basic confusion. But wouldn't the arguments be
evaluated afresh on the subsequent call for this SRF? In that case
freeing up the context of the *last* call should not be an issue I
would think.

And if this is indeed the case we should be using a different longer
lived context and not the ecxt_per_tuple_memory context..

Regards,
Nikhils
--
http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Tom Lane on 7 May 2010 11:09
Nikhil Sontakke <nikhil.sontakke(a)enterprisedb.com> writes:
>> Consider
>> � � � �srf(foo(col))
>> where foo returns a pass-by-reference datatype.

> Yeah this is my basic confusion. But wouldn't the arguments be
> evaluated afresh on the subsequent call for this SRF?

No, see ExecMakeFunctionResult(). If we did that we'd have serious
problems with volatile functions, ie srf(random()).

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

 |  Next  |  Last
Pages: 1 2 3
Prev: On a somewhat disappointing correspondence
Next: [HACKERS] Partitioning/inherited tables vs FKs