libpq, PQexecPrepared, data size sent to FE vs. FETCH_COUNT
in [PgSql]
|
Prev: [HACKERS] Hot Standby performance and deadlocking
Next: [HACKERS] [PATCH] Add XMLEXISTS function from the SQL/XML standard (was: Addxpath_exists Function)
From: Alex Goncharov on 25 May 2010 07:35 ,--- I/Alex (Mon, 24 May 2010 12:25:18 -0400) ----* | No equivalent of FETCH_COUNT is available at the libpq level, so I | assume that the interface I am using is smart enough not to send | gigabytes of data to FE. | | Where does the result set (GBs of data) reside after I call | PQexecPrepared? On BE, I hope? Sorry for asking again... No sarcasm meant: is there no straightforward answer here? Or nobody is certain? Or a wrong list? Thanks, -- Alex -- alex-goncharov(a)comcast.net -- -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Yeb Havinga on 25 May 2010 08:08 Alex Goncharov wrote: > ,--- I/Alex (Mon, 24 May 2010 12:25:18 -0400) ----* > | No equivalent of FETCH_COUNT is available at the libpq level, so I > | assume that the interface I am using is smart enough not to send > | gigabytes of data to FE. > | > | Where does the result set (GBs of data) reside after I call > | PQexecPrepared? On BE, I hope? > > Sorry for asking again... > > No sarcasm meant: is there no straightforward answer here? Or nobody > is certain? Or a wrong list? > The straighforward answer is that the libpq frontend c-library does not support something like the JDBC client's setFetchSize. The GBs of data are gathered at the site of the libpq client (pgresult object gathered/allocated while consuming result input from backend). regards, Yeb Havinga -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Abhijit Menon-Sen on 25 May 2010 07:56 At 2010-05-25 07:35:34 -0400, alex-goncharov(a)comcast.net wrote: > > | Where does the result set (GBs of data) reside after I call > | PQexecPrepared? On BE, I hope? Unless you explicitly declare and fetch from an SQL-level cursor, your many GBs of data are going to be transmitted to libpq, which will eat lots of memory. (The wire protocol does have something like cursors, but libpq does not use them, it retrieves the entire result set.) -- ams -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Alex Goncharov on 25 May 2010 09:17 ,--- Abhijit Menon-Sen (Tue, 25 May 2010 17:26:18 +0530) ----* | Unless you explicitly declare and fetch from an SQL-level cursor, your | many GBs of data are going to be transmitted to libpq, which will eat | lots of memory. (The wire protocol does have something like cursors, | but libpq does not use them, it retrieves the entire result set.) ,--- Yeb Havinga (Tue, 25 May 2010 14:08:51 +0200) ----* | The GBs of data are gathered at the site of the libpq client (pgresult | object gathered/allocated while consuming result input from backend). `------------------------------------------------------* Thank you very much! -- Alex -- alex-goncharov(a)comcast.net -- -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Andrew Chernow on 25 May 2010 09:20
On 05/25/2010 07:35 AM, Alex Goncharov wrote: > ,--- I/Alex (Mon, 24 May 2010 12:25:18 -0400) ----* > | No equivalent of FETCH_COUNT is available at the libpq level, so I > | assume that the interface I am using is smart enough not to send > | gigabytes of data to FE. > | > | Where does the result set (GBs of data) reside after I call > | PQexecPrepared? On BE, I hope? > > Sorry for asking again... > > No sarcasm meant: is there no straightforward answer here? Or nobody > is certain? Or a wrong list? > Issue multiple queries and make use of LIMIT/OFFSET. You'll have to go manual on this one. -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |