Listen / Notify - what to do when the queue is full
in [PgSql]
|
From: Jeff Davis on 9 Feb 2010 01:13 In this version of the patch, there is a compiler warning: async.c: In function 'AtPrepare_Notify': async.c:593: warning: unused variable 'p' and also two trivial merge conflicts: an OID conflict for the functions you added, and a trivial code conflict. On Sun, 2010-02-07 at 17:32 +0100, Joachim Wieland wrote: > On Wed, Feb 3, 2010 at 2:05 AM, Jeff Davis <pgsql(a)j-davis.com> wrote: > > The original comment was a part of the NotifyStmt case, and I don't > > think we can support NOTIFY issued on a standby system -- surely there's > > no way for the standby to communicate the notification to the master. > > Anyway, this is getting a little sidetracked; I don't think we need to > > worry about HS right now. > > True but I was not talking about moving any notifications to different > servers. Clients listening on one server should receive the > notifications from NOTIFYs executed on this server, no matter if it is > a standby or the master server. I'm not sure I agree with that philosophy. If the driving use-case for LISTEN/NOTIFY is cache invalidation, then a NOTIFY on the master should make it to all listening backends on the slaves. > This is still kind of an open item but it's an slru issue and should > also be true for other functionality that uses slru queues. I haven't found out anything new here. > This I don't understand... If power goes out and we restart, we'd > first put all notifications from the prepared transactions into the > queue. We know that they fit because they have fit earlier as well (we > wouldn't allow user connections until we have worked through all 2PC > state files). > Ok, it appears you've thought the 2PC interaction through more than I have. Even if we don't include it this time, I'm glad to hear that there is a plan to do so. Feel free to include support if you have it ready, but it's late in the CF so I don't want you to get sidetracked on that issue. > There was another problem that the slru files did not all get deleted > at server restart, which is fixed now. Good catch. > Regarding the famous ASCII-restriction open item I have now realized > what I haven't thought of previously: notifications are not > transferred between databases, they always stay in one database. Since > libpq does the conversion between server and client encoding, it is > questionable if we really need to restrict this at all... But I am not > an encoding expert so whoever feels like he can confirm or refute > this, please speak up. I would like to support encoded text, but I think there are other problems. For instance, what if one server has a client_encoding that doesn't support some of the glyphs being sent by the notifying backend? Then we have a mess, because we can't deliver it. I think we just have to say "ASCII only" here, because there's no reasonable way to handle this, regardless of implementation. If the user issues SELECT and the client_encoding can't support some of the glyphs, we can throw an error. But for a notification? We just have no mechanism for that. Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jeff Davis on 9 Feb 2010 16:46 On Mon, 2010-02-08 at 22:13 -0800, Jeff Davis wrote: > I would like to support encoded text, but I think there are other > problems. For instance, what if one server has a client_encoding that > doesn't support some of the glyphs being sent by the notifying backend? > Then we have a mess, because we can't deliver it. I was thinking more about this. It seems clear that we want the backend that issues the notify to only put 7-bit ASCII in the payload. But if the client sends the letters 'string' as a payload, and the representation in the server encoding is something other than the normal 7-bit ASCII representation of 'string', it will be incorrect, right? Looking at the documentation, it appears that all of the server encodings represent 7-bit ascii characters using the same 7-bit ascii representation. Is that true? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 9 Feb 2010 16:51 Jeff Davis <pgsql(a)j-davis.com> writes: > Looking at the documentation, it appears that all of the server > encodings represent 7-bit ascii characters using the same 7-bit ascii > representation. Is that true? Correct. We only support ASCII-superset encodings, both for frontend and backend. Limiting NOTIFY payloads to 7-bit would definitely avoid the issue. The question is if that's more of a pain than a benefit. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 9 Feb 2010 17:30 On Tue, Feb 9, 2010 at 4:51 PM, Tom Lane <tgl(a)sss.pgh.pa.us> wrote: > Jeff Davis <pgsql(a)j-davis.com> writes: >> Looking at the documentation, it appears that all of the server >> encodings represent 7-bit ascii characters using the same 7-bit ascii >> representation. Is that true? > > Correct. We only support ASCII-superset encodings, both for frontend > and backend. > > Limiting NOTIFY payloads to 7-bit would definitely avoid the issue. > The question is if that's more of a pain than a benefit. I think it's a reasonable restriction for now. We have limited time remaining here; and we can always consider relaxing the restriction in the future when we have more time to think through the issues. It'll still be a big improvement over what we have now. ....Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jeff Davis on 9 Feb 2010 18:01
On Tue, 2010-02-09 at 16:51 -0500, Tom Lane wrote: > Limiting NOTIFY payloads to 7-bit would definitely avoid the issue. > The question is if that's more of a pain than a benefit. I don't see any alternative. If one backend sends a NOTIFY payload that contains a non-ASCII character, there's a risk that we won't be able to deliver it to another backend with a client_encoding that can't represent that character. Also, just the fact that client_encoding can be changed at pretty much any time is a potential problem, because it's difficult to know whether a particular notification was sent using the old client_encoding or the new one (because it's asynchronous). Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |