Exposing the Xact commit order to the user
in [PgSql]
|
Prev: [HACKERS] Exposing the Xact commit order to the user
Next: [HACKERS] Synchronization levels in SR
From: Jan Wieck on 26 May 2010 19:48 On 5/26/2010 4:34 PM, Kevin Grittner wrote: > Jan Wieck <JanWieck(a)Yahoo.com> wrote: > >> Without this logic, the replication system could not combine >> multiple origin sessions into one replication session without >> risking to never find a state, in which it can commit. > > My latest idea for handling this in WAL-based replication involves > WAL-logging information about the transaction through which a the > committing transaction makes it safe to view. There are a few > options here at the detail level that I'm still thinking through. > The idea would be that the xmin from read-only queries on the slaves > might be somewhere behind where you would expect based on > transactions committed. (The details involve such things as where > non-serializable transactions fall into the plan on both sides, and > whether it's worth the effort to special-case read-only transactions > on the master.) > > I can't say that I'm 100% sure that some lurking detail won't shoot > this technique down for HS, but it seems good to me at a conceptual > level. Without simulating multiple simultaneous transactions during playback, how are you going to manage that the tuples, already inserted on behalf of the ongoing master transactions, disappear when they abort on the master? Jan -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: "Kevin Grittner" on 27 May 2010 09:50 Jan Wieck <JanWieck(a)Yahoo.com> wrote: > On 5/26/2010 4:34 PM, Kevin Grittner wrote: >> My latest idea for handling this in WAL-based replication >> involves WAL-logging information about the transaction through >> which a the committing transaction makes it safe to view. There >> are a few options here at the detail level that I'm still >> thinking through. The idea would be that the xmin from read-only >> queries on the slaves might be somewhere behind where you would >> expect based on transactions committed. (The details involve >> such things as where non-serializable transactions fall into the >> plan on both sides, and whether it's worth the effort to >> special-case read-only transactions on the master.) >> >> I can't say that I'm 100% sure that some lurking detail won't >> shoot this technique down for HS, but it seems good to me at a >> conceptual level. > > Without simulating multiple simultaneous transactions during > playback, how are you going to manage that the tuples, already > inserted on behalf of the ongoing master transactions, disappear > when they abort on the master? When do writes ever become visible to a snapshot without having been committed? I'm not talking about changing that in any way. I'm talking about deferring visibility of committed transactions until they can be viewed without risking serialization anomalies. This requires, at a minimum, that any concurrent serializable transactions which are not read-only have completed. (Perhaps I'm not understanding your question....) -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jan Wieck on 27 May 2010 12:01 On 5/27/2010 9:59 AM, Greg Stark wrote: > On Wed, May 26, 2010 at 5:38 PM, Greg Stark <gsstark(a)mit.edu> wrote: >> How about just API generalities? Like, where do you need this data, on >> the master or on the slave? Would PGXC like it on the transaction >> coordinator? >> >> What question do you need to answer, do you need to pull out sets of >> commits in certain ranges or look up specific transaction ids and find >> out when they committed? Or do you only need to answer which of two >> transaction ids committed first? >> > > This thread has been hard to follow for me. Were any of these > questions answered? > Yes. On 5/26/2010 4:49 PM, Jan Wieck wrote: > On 5/26/2010 12:38 PM, Greg Stark wrote: >> > On Wed, May 26, 2010 at 5:10 PM, Jan Wieck <JanWieck(a)yahoo.com> wrote: >>> >> ... but to answer that request, actually I don't even think we should be >>> >> discussing API specifics. >>> >> >> > >> > How about just API generalities? Like, where do you need this data, on >> > the master or on the slave? Would PGXC like it on the transaction >> > coordinator? >> > >> > What question do you need to answer, do you need to pull out sets of >> > commits in certain ranges or look up specific transaction ids and find >> > out when they committed? Or do you only need to answer which of two >> > transaction ids committed first? > > The question I want answered is > > "what was the order and xid of the next 0..n transactions, that > committed after transaction X?" > > Preferably I would avoid scanning the entire available WAL just to get > the next n xid's to process. > > The proposal assigned a unique serial number (file segment and position > driven) to each xid and used that for the ordering as well as > identification of the last known transaction. That is certainly a > premature implementation detail. > > In this implementation it wouldn't even matter if a transaction that was > recorded actually never made it because it crashed before the WAL flush. > It would be reported by this "commit order" feature, but there would be > no traces of whatever it did to be found inside the DB, so that anomaly > is harmless. > > > Jan > > -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin > -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Greg Stark on 27 May 2010 09:59 On Wed, May 26, 2010 at 5:38 PM, Greg Stark <gsstark(a)mit.edu> wrote: > How about just API generalities? Like, where do you need this data, on > the master or on the slave? Would PGXC like it on the transaction > coordinator? > > What question do you need to answer, do you need to pull out sets of > commits in certain ranges or look up specific transaction ids and find > out when they committed? Or do you only need to answer which of two > transaction ids committed first? > This thread has been hard to follow for me. Were any of these questions answered? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jan Wieck on 27 May 2010 17:02
On 5/27/2010 12:01 PM, Jan Wieck wrote: > On 5/27/2010 9:59 AM, Greg Stark wrote: >> This thread has been hard to follow for me. Were any of these >> questions answered? >> > > Yes. The thing missing is any sort of answer to that problem description. Jan -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |