Listen / Notify - what to do when the queue is full
in [PgSql]
|
From: Joachim Wieland on 20 Jan 2010 17:39 On Wed, Jan 20, 2010 at 11:08 PM, Jeff Davis <pgsql(a)j-davis.com> wrote: >> Yes. That is the case with the existing implementation as well, no? >> We don't consider sending notifies until transaction end, so anything >> that commits during the xact in which you UNLISTEN will get dropped. > > Only if the transaction containing UNLISTEN commits. Are you saying it > would also be OK to drop NOTIFYs if a backend's UNLISTEN transaction > aborts? If the backend's UNLISTEN transaction aborts, then it has never executed UNLISTEN... So it will continue to get notifications (if it has executed a LISTEN before). Joachim -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 20 Jan 2010 17:40 Jeff Davis <pgsql(a)j-davis.com> writes: > On Wed, 2010-01-20 at 15:54 -0500, Tom Lane wrote: >> Yes. That is the case with the existing implementation as well, no? >> We don't consider sending notifies until transaction end, so anything >> that commits during the xact in which you UNLISTEN will get dropped. > Only if the transaction containing UNLISTEN commits. Are you saying it > would also be OK to drop NOTIFYs if a backend's UNLISTEN transaction > aborts? No, I would say not, but that wasn't being proposed was it? The decisions about what to do are only made at/after commit. > Thinking out loud: If we're taking this approach, I wonder if it might > be a good idea to PreventTransactionChain for LISTEN and UNLISTEN? That shouldn't be necessary IMO. There's never been such a restriction before. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jeff Davis on 20 Jan 2010 21:06 On Tue, 2010-01-19 at 19:24 -0500, Tom Lane wrote: > (I'm still > wondering if we couldn't do without the lock altogether though.) Here's the problem as I see it: If we insert the notifications into the queue before actually recording the commit, there's a window in between where another backend could perform the expected sequence as you wrote: 1. LISTEN foo; (and commit the listen) 2. examine current database state 3. assume that we'll get a NOTIFY for any change that commits subsequently to what we saw in step 2 and miss the NOTIFYs, and not see the updated database state. But I don't think that the NOTIFYs will actually be missed. Once put into the queue, the notification will only be removed from the queue after all backends have read it. But no backend will advance past it as long as the notification is from an uncommitted transaction. By the time the notifying transaction is committed, the listening transaction will also be committed, and therefore subscribed to the queue. The newly-listening backend will be awakened properly as well, because that's done after the notifying transaction commits, and therefore will wake up any listening transactions that committed earlier. However, there's still a problem inserting into the queue when no backends are listening. Perhaps that can be solved right before we wake up the listening backends after the notifying transaction commits: if there are no listening backends, clear the queue. We still might get spurious notifications if they were committed before the LISTEN transaction was committed. And we also might get spurios notifications if the UNLISTEN doesn't take effect quite quickly enough. Those are both acceptable. If the above scheme is too complex, we can always use a heavyweight lock. However, there's no pg_listener so it's not obvious what LOCKTAG to use. We can just pick something arbitrary, like the Oid of the new pg_listening() function, I suppose. Is there any precedent for that? Thoughts? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Joachim Wieland on 21 Jan 2010 04:14 On Thu, Jan 21, 2010 at 3:06 AM, Jeff Davis <pgsql(a)j-davis.com> wrote: > Here's the problem as I see it: You are writing a lot of true facts but I miss to find a real problem... What exactly do you see as a problem? The only time you are writing "problem" is in this paragraph: > However, there's still a problem inserting into the queue when no > backends are listening. Perhaps that can be solved right before we wake > up the listening backends after the notifying transaction commits: if > there are no listening backends, clear the queue. This gets already done, in SignalBackends(), a notifying transactions counts the number of listening backends. If no other backend is listening, then it signals itself so that the queue gets cleaned (i.e. the global pointer gets forwarded and the slru pages will be truncated if possible). I have been working on simplifying the patch yesterday, I still need to adapt comments and review it again but I am planning to post the new version tonight. Then we have a common base again to discuss further :-) Thanks for your review, Joachim -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jeff Davis on 21 Jan 2010 12:21
On Thu, 2010-01-21 at 10:14 +0100, Joachim Wieland wrote: > On Thu, Jan 21, 2010 at 3:06 AM, Jeff Davis <pgsql(a)j-davis.com> wrote: > > Here's the problem as I see it: > > You are writing a lot of true facts but I miss to find a real > problem... What exactly do you see as a problem? I worded that in a confusing way, I apologize. My point was that I don't think we need a lock, because I don't see any situation in which the notifications would be lost. > I have been working on simplifying the patch yesterday, I still need > to adapt comments and review it again but I am planning to post the > new version tonight. Then we have a common base again to discuss > further :-) Sounds good. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |