Keepalive for max_standby_delay
in [PgSql]
|
Prev: [HACKERS] Keepalive for max_standby_delay
Next: Unexpected page allocation behavior on insert-only tables
From: Tom Lane on 3 Jun 2010 12:58 Simon Riggs <simon(a)2ndQuadrant.com> writes: > On Wed, 2010-06-02 at 16:00 -0400, Tom Lane wrote: >> the current situation that query grace periods go to zero > Possibly a better way to handle this concern is to make the second > parameter: min_standby_grace_period - the minimum time a query will be > given in which to execute, even if max_standby_delay has been reached or > exceeded. > Would that more directly address you concerns? > min_standby_grace_period (ms) SIGHUP A minimum grace period seems like a good idea to me, but I think it's somewhat orthogonal to the core problem here. I think we all intuitively feel that there should be a way to dial back the grace period when a slave is "far behind" on applying WAL. The problem is first how to figure out what "far behind" means, and second how to adjust the grace period in a way that doesn't have surprising misbehaviors. A minimum grace period would prevent some of the very worst misbehaviors but it's not really addressing the core problem. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 3 Jun 2010 13:18 On Thu, 2010-06-03 at 12:47 -0400, Tom Lane wrote: > Simon Riggs <simon(a)2ndQuadrant.com> writes: > > On Wed, 2010-06-02 at 13:14 -0400, Tom Lane wrote: > >> This patch seems to me to be going in fundamentally the wrong direction. > >> It's adding complexity and overhead (far more than is needed), and it's > >> failing utterly to resolve the objections that I raised to start with. > > > Having read your proposal, it seems changing from time-on-sender to > > time-on-receiver is a one line change to the patch. > > > What else are you thinking of removing, if anything? > > Basically, we need to get rid of everything that feeds timestamps from > the WAL content into the kill-delay logic. I understand your wish to do this, though it isn't always accurate to do that in the case where there is a backlog. The patch already does that *mostly* for the streaming case. The only time it does use the WAL content timestamp is when the WAL content timestamp is later than the oldest receive time, so in that case it is used as a correction. (see code comments and comments below also) > >> In particular, Simon seems to be basically refusing to do anything about > >> the complaint that the code fails unless master and standby clocks are > >> in close sync. I do not believe that this is acceptable, and since he > >> won't fix it, I guess I'll have to. > > > Syncing two servers in replication is common practice, as has been > > explained here; I'm still surprised people think otherwise. > > Doesn't affect the complaint in the least: I do not find it acceptable > to have that be *mandated* in order for our code to work sensibly. OK, accepted. > I would be OK with having something approaching what you want as a > non-default optional behavior (with a clearly-documented dependency > on having synchronized clocks). Yes, that's what I'd been thinking also. So that gives us a way forwards. standby_delay_base = apply (default) | send determines whether the standby_delay is calculated solely with reference to the standby server (apply) or whether times from the sending server are used (send). Use of send implies that the clocks on primary and standby are synchronised to within a useful accuracy, in which case it is usual to enforce that with time synchronisation software such as ntp. > But in any case the current behavior is > still quite broken as regards reading stale timestamps from WAL. Agreed. That wasn't the objective of this patch or a priority. If you're reading from an archive, you need to set max_standby_delay appropriately, current recommendation is -1. We can address that concern once the main streaming case is covered, or we can add that now. Are you planning to work on these things now as you said? How about I apply my patch now, then you do another set of changes afterwards to add the other items you mentioned, since that is now 2 additional parameters and related code? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 3 Jun 2010 13:32 Simon Riggs <simon(a)2ndQuadrant.com> writes: > On Thu, 2010-06-03 at 12:47 -0400, Tom Lane wrote: >> But in any case the current behavior is >> still quite broken as regards reading stale timestamps from WAL. > Agreed. That wasn't the objective of this patch or a priority. > If you're reading from an archive, you need to set max_standby_delay > appropriately, current recommendation is -1. That's not a fix; the problem is that there *is no* appropriate setting for max_standby_delay when you're reading from archive. It is unlikely that the values of the timestamps you're reading should be considered to have any bearing on what grace period to allow; but nonetheless it's desirable to be able to have a non-infinite grace time. Basically what I think is that we want what you called "apply" semantics always for reading from archive (and I still think the DBA should be able to set that grace period separately from the one that applies in SR operation). Paying attention to the master's timestamps is only reasonable in the SR context. And yes, I want the dependency on WAL timestamps to be gone completely, not just mostly. I think that driving the delay logic off of SR receipt time (and/or the timestamp we agreed to add to the SR protocol) is the appropriate and sufficient way to deal with the problem. Looking at the embedded timestamps just confuses the feedback loop behavior. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 3 Jun 2010 13:46 On Thu, 2010-06-03 at 13:32 -0400, Tom Lane wrote: > Simon Riggs <simon(a)2ndQuadrant.com> writes: > > On Thu, 2010-06-03 at 12:47 -0400, Tom Lane wrote: > >> But in any case the current behavior is > >> still quite broken as regards reading stale timestamps from WAL. > > > Agreed. That wasn't the objective of this patch or a priority. > > > If you're reading from an archive, you need to set max_standby_delay > > appropriately, current recommendation is -1. > > That's not a fix; the problem is that there *is no* appropriate setting > for max_standby_delay when you're reading from archive. It is unlikely > that the values of the timestamps you're reading should be considered to > have any bearing on what grace period to allow; but nonetheless it's > desirable to be able to have a non-infinite grace time. I'm not saying that's a fix; I agree we should change that also. > Basically what I think is that we want what you called "apply" semantics > always for reading from archive (and I still think the DBA should be > able to set that grace period separately from the one that applies in > SR operation). Paying attention to the master's timestamps is only > reasonable in the SR context. Agreed. > And yes, I want the dependency on WAL timestamps to be gone completely, > not just mostly. I think that driving the delay logic off of SR receipt > time (and/or the timestamp we agreed to add to the SR protocol) is the > appropriate and sufficient way to deal with the problem. Looking at the > embedded timestamps just confuses the feedback loop behavior. I disagree with "sufficient", with good reason. Please look at the code comments and see what will happen if we don't make the correction suggested. If after that you still disagree, then do it your way and we'll wait for comments in the beta; I think there will be some, but I am tired and prone to agreement to allow this to go through. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 3 Jun 2010 18:48
On Thu, 2010-06-03 at 18:18 +0100, Simon Riggs wrote: > Are you planning to work on these things now as you said? Are you? Or do you want me to? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |