Exposing the Xact commit order to the user
in [PgSql]
|
Prev: [HACKERS] Exposing the Xact commit order to the user
Next: [HACKERS] Synchronization levels in SR
From: "Kevin Grittner" on 3 Jun 2010 18:24 "Kevin Grittner" <Kevin.Grittner(a)wicourts.gov> wrote: > Greg Stark <gsstark(a)mit.edu> wrote: > >> what kind of interface you need > > For the potential uses I can see, it would be great to have a SRF > which took two parameters: xid of last known commit and a limit > how many commits past that to return. Jan's very first post had it right; my idea was flawed: | Exposing the data will be done via a set returning function. The | SRF takes two arguments. The maximum number of rows to return and | the last serial number processed by the reader. The advantage of | such SRF is that the result can be used in a query that right away | delivers audit or replication log information in transaction | commit order. The SRF can return an empty set if no further | transactions have committed since, or an error if data segments | needed to answer the request have already been purged. | | Purging of the data will be possible in several different ways. | Autovacuum will call a function that drops segments of the data | that are outside the postgresql.conf configuration with respect to | maximum age or data volume. There will also be a function reserved | for superusers to explicitly purge the data up to a certain serial | number. Apologies for not looking back to the start of the thread before that last post. It was all laid out right at the start. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Bruce Momjian on 3 Jun 2010 18:29 Jan Wieck wrote: > On 6/3/2010 4:04 PM, Bruce Momjian wrote: > > If you want to fork Postgres and add it, go ahead, but if the community > > has to maintain the code and document it, we care. > > That comment was rather unprofessional. I think the rest of us still try > to find the best solution for the problem, not kill the discussion. You > may want to rejoin that effort. > > I care about an efficient, low overhead way to get a certain > information, that is otherwise extremely difficult, expensive and > version dependent to get. > > I care about cleaning up more of the mistakes, made in the original > development of Slony. Namely using hacks and kluges to implement > details, not supported by a current version of PostgreSQL. Londiste and > Slony made a good leap on that with the txid data type. Slony made > another step like that with 2.0, switching to the (for that very purpose > developed and contributed) native trigger configuration instead of > hacking system catalogs. This would be another step in that direction > and we would be able to unify Londiste's and Slony's transport mechanism > and eliminating the tick/sync kluge. > > Care to explain what exactly you care about? Here is what I was replying to: > >> I actually have a hard time understanding why people are so opposed t$ > > >> feature that has zero impact at all unless a DBA actually turns in ON. > >> What is the problem with exposing the commit order of transactions? Jan's comment is why should others care what he wants because it has zero impact? I am saying the community cares because we have to maintain the code. I stand by my comment. I remember a dismissive comment by Jan when 'session_replication_role' was added, and a similar strong comment from me at that time as well. It seems we are doing this again. -- Bruce Momjian <bruce(a)momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Bruce Momjian on 3 Jun 2010 18:31 Bruce Momjian wrote: > Jan Wieck wrote: > > On 6/3/2010 4:04 PM, Bruce Momjian wrote: > > > If you want to fork Postgres and add it, go ahead, but if the community > > > has to maintain the code and document it, we care. > > > > That comment was rather unprofessional. I think the rest of us still try > > to find the best solution for the problem, not kill the discussion. You > > may want to rejoin that effort. > > > > I care about an efficient, low overhead way to get a certain > > information, that is otherwise extremely difficult, expensive and > > version dependent to get. > > > > I care about cleaning up more of the mistakes, made in the original > > development of Slony. Namely using hacks and kluges to implement > > details, not supported by a current version of PostgreSQL. Londiste and > > Slony made a good leap on that with the txid data type. Slony made > > another step like that with 2.0, switching to the (for that very purpose > > developed and contributed) native trigger configuration instead of > > hacking system catalogs. This would be another step in that direction > > and we would be able to unify Londiste's and Slony's transport mechanism > > and eliminating the tick/sync kluge. > > > > Care to explain what exactly you care about? > > Here is what I was replying to: > > > >> I actually have a hard time understanding why people are so opposed t$ > > > >> feature that has zero impact at all unless a DBA actually turns in ON. > > >> What is the problem with exposing the commit order of transactions? > > Jan's comment is why should others care what he wants because it has > zero impact? I am saying the community cares because we have to > maintain the code. I stand by my comment. > > I remember a dismissive comment by Jan when 'session_replication_role' > was added, and a similar strong comment from me at that time as well. > It seems we are doing this again. Of course, if I am misintepreting what Jan said, please let me know. -- Bruce Momjian <bruce(a)momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Alvaro Herrera on 3 Jun 2010 19:11 Excerpts from Chris Browne's message of jue jun 03 16:21:35 -0400 2010: > What this offers is *SOME* idea of how much updating work a particular > transaction did. It's a bit worse than you suggest: > > - If replication triggers have captured tuples, those would get > counted. > > - TOAST updates might lead to extra updates being counted. > > But back to where you started, I'd anticipate 7 inserts, 7 deletes, > and 7 updates being counted as something around 21 updates. > > And if that included 5 TOAST changes, it might bump up to 26. > > If there were replication triggers in place, that might bump the count > up to 45 (which I chose arbitrarily). Why not send separate numbers of tuple inserts/updates/deletes, which we already have from pgstats? -- Álvaro Herrera <alvherre(a)commandprompt.com> The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jan Wieck on 3 Jun 2010 19:48
On 6/3/2010 5:58 PM, Greg Stark wrote: > On Thu, Jun 3, 2010 at 8:50 PM, Jan Wieck <JanWieck(a)yahoo.com> wrote: >>> I'm puzzled how you would define this value. How do you add 7 inserts, >>> 7 deletes, and 7 updates? Is that 21 rows modified? >> >> I actually have a hard time understanding why people are so opposed to a >> feature that has zero impact at all unless a DBA actually turns in ON. What >> is the problem with exposing the commit order of transactions? > > The post you were responding to was regarding the meaninglessness of > the "number of records" attribute you wanted. Your response is a non > sequitor. I never proposed a "number of records" attribute. I proposed a sum of the row counts in the statistics collector. That row count would be a mix of insert, update, delete and toast operations. It's not an exact indicator of anything, but a good enough hint of how much data may come down the pipe if I were to select all replication data belonging to that transaction. > > I think the commit order of transactions would be a good thing to > expose though I've asked repeatedly what kind of interface you need > and never gotten answers to all the questions. In the original email that started this whole thread I wrote: > Exposing the data will be done via a set returning function. The SRF > takes two arguments. The maximum number of rows to return and the last > serial number processed by the reader. The advantage of such SRF is that > the result can be used in a query that right away delivers audit or > replication log information in transaction commit order. The SRF can > return an empty set if no further transactions have committed since, or > an error if data segments needed to answer the request have already been > purged. Did that not answer your question? Jan -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |