Listen / Notify - what to do when the queue is full
in [PgSql]
|
From: Jeff Davis on 19 Jan 2010 18:41 On Wed, 2009-12-09 at 11:43 +0100, Joachim Wieland wrote: > Examples: > > Backend 1: Backend 2: > > transaction starts > NOTIFY foo; > commit starts > transaction starts > LISTEN foo; > commit starts > commit to clog > commit to clog > > => Backend 2 will receive Backend 1's notification. How does the existing notification mechanism solve this problem? Is it really a problem? Why would Backend2 expect to receive the notification? > > Backend 1: Backend 2: > > transaction starts > NOTIFY foo; > commit starts > transaction starts > UNLISTEN foo; > commit starts > commit to clog > commit to clog > > => Backend 2 will not receive Backend 1's notification. This is the same problem, except that it doesn't matter. A spurious notification is not a bug, right? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 19 Jan 2010 19:05 Jeff Davis <pgsql(a)j-davis.com> writes: > How does the existing notification mechanism solve this problem? Is it > really a problem? Why would Backend2 expect to receive the notification? The intended way to use LISTEN/NOTIFY for status tracking is 1. LISTEN foo; (and commit the listen) 2. examine current database state 3. assume that we'll get a NOTIFY for any change that commits subsequently to what we saw in step 2 In the current implementation, a transaction that is in process of commit during step 1 might possibly not see your pg_listener record as committed, and so it might not send you a NOTIFY for whatever it did. If it still hasn't committed when you perform step 2, then you'd fail to see its changes as part of your initial state, *and* you'd not get a NOTIFY when the changes did become visible. The way we prevent this race condition is that a listener takes exclusive lock on pg_listener before entering its record, and doesn't release the lock until after committing. Notifiers likewise take exclusive lock. This serializes things so that either the modifying transaction commits before the listener completes step 1 (and hence the listener will see its updates in step 2), or the listener is guaranteed to have a committed record in pg_listener when the modifying process determines whom to notify. I guess Joachim is trying to provide a similar guarantee for the new implementation, but I'm not clear on why it would require locking. The new implementation is broadcast and ISTM it shouldn't require the modifying transaction to know which processes are listening. I haven't read the patch but I agree that the description you give is pretty scary from a performance standpoint. More locks around transaction commit doesn't seem like a good idea. If they're only taken when an actual LISTEN or NOTIFY has happened in the current transaction, that'd be okay (certainly no worse than what happens now) but the naming suggested that this'd happen unconditionally. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jeff Davis on 19 Jan 2010 19:19 On Tue, 2010-01-19 at 19:05 -0500, Tom Lane wrote: > I guess Joachim is trying to provide a similar guarantee for the new > implementation, but I'm not clear on why it would require locking. > The new implementation is broadcast and ISTM it shouldn't require the > modifying transaction to know which processes are listening. I think there is a better way. I'll dig into it a little more. > I haven't read the patch but I agree that the description you give is > pretty scary from a performance standpoint. More locks around > transaction commit doesn't seem like a good idea. I was also worried about holding multiple LWLocks at once -- is such practice generally avoided in the rest of the code? > If they're only taken > when an actual LISTEN or NOTIFY has happened in the current transaction, > that'd be okay (certainly no worse than what happens now) but the naming > suggested that this'd happen unconditionally. It appears that the locks are only taken when LISTEN or NOTIFY is involved. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 19 Jan 2010 19:24 Jeff Davis <pgsql(a)j-davis.com> writes: > I was also worried about holding multiple LWLocks at once -- is such > practice generally avoided in the rest of the code? It's allowed but remember that there is no deadlock detection in lwlock.c. You must be very certain that there is only one possible order in which such locks could be taken. Interactions with heavyweight locks would be bad news as well. > It appears that the locks are only taken when LISTEN or NOTIFY is > involved. On the whole it might be better if a heavyweight lock were used, such that it'll automatically clean up after commit. (I'm still wondering if we couldn't do without the lock altogether though.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jeff Davis on 19 Jan 2010 19:29
On Tue, 2010-01-19 at 19:24 -0500, Tom Lane wrote: > Jeff Davis <pgsql(a)j-davis.com> writes: > > I was also worried about holding multiple LWLocks at once -- is such > > practice generally avoided in the rest of the code? > > It's allowed but remember that there is no deadlock detection in lwlock.c. > You must be very certain that there is only one possible order in which > such locks could be taken. Interactions with heavyweight locks would be > bad news as well. That was my worry initially. > On the whole it might be better if a heavyweight lock were used, > such that it'll automatically clean up after commit. (I'm still > wondering if we couldn't do without the lock altogether though.) Yes, I think there's a better way as well. I'll look into it. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |