max_standby_delay considered harmful
in [PgSql]
|
Prev: Further Hot Standby documentation required
Next: [HACKERS] Streaming replication - unable to stop the standby
From: Heikki Linnakangas on 5 May 2010 10:56 Simon Riggs wrote: > I am mostly unavailable for next few days. (Repairing bikeshed.) Hey, you're supposed to do the bikeshedding on-list! ;-) -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 5 May 2010 11:10 Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> writes: > To accomplish what you're trying to accomplish, you would need to label > each received WAL record with the timestamp when it was received, and > compare the reception timestamp of the record you're applying against > current timestamp. Yeah, this is why I thought that closed-loop lag control was a research project. In practice, we don't have to track it at the individual record level. The real behavior of walsender is that we get a "gob" of WAL each activity cycle, and so tracking the WAL start location and receipt time for each gob ought to be sufficient. (In fact trying to ascribe any finer-grain receipt time than that to individual WAL records is probably bogus anyway.) It might be enough to remember the start location and time for the latest gob, depending on exactly what control algorithm you want to use. But the whole thing requires significant thought and testing, which we really haven't got time for now. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 5 May 2010 12:18 On Wed, May 5, 2010 at 9:58 AM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > Tom Lane wrote: >> Comments? > > There's currently three ways to set max_standby_delay: > > max_standby_delay = -1 # Query wins > max_standby_delay = 0 # Recovery wins > max_standby_delay > X # Query wins until lag > X. > > As Tom points out, the 3rd option has all sorts of problems. I very much > like the behavior that max_standby_delay tries to accomplish, but I have > to agree that it's not very reliable as it is. I don't like Tom's > proposal either; the standby can fall behind indefinitely, and queries > get a varying grace period. > > Let's rip out the concept of a delay altogether, and make it a boolean. > If you really want your query to finish, set it to -1 (using the current > max_standby_delay nomenclature). If recovery is important to you, set it > to 0. Does my proposal (upthread) to limit this by quantity of WAL rather than time have any legs, or is that impractical and/or otherwise poor? ....Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 5 May 2010 12:30 Robert Haas wrote: > Does my proposal (upthread) to limit this by quantity of WAL rather > than time have any legs, or is that impractical and/or otherwise poor? That would certainly be easier to implement sanely than a time-based quantity. One problem is that we don't know how much unapplied WAL there is, when you're not using streaming replication. And I'm not sure how useful that is to users - it's very hard to estimate what to set it to. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 5 May 2010 14:33
On Wed, May 5, 2010 at 12:30 PM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > Robert Haas wrote: >> Does my proposal (upthread) to limit this by quantity of WAL rather >> than time have any legs, or is that impractical and/or otherwise poor? > > That would certainly be easier to implement sanely than a time-based > quantity. One problem is that we don't know how much unapplied WAL there > is, when you're not using streaming replication. Hmm, that's a problem, likely fatally so. > And I'm not sure how > useful that is to users - it's very hard to estimate what to set it to. I'm not sure whether that's really an issue or not. I mean, if we say that the standby is allowed to be, say, 16MB behind the master, we know that recovery time is bounded by how long it takes to replay 16MB. Which is in some ways more defined than saying we're behind the primary by 30 min, which could take a long time to replay or not much at all. But, I guess it's moot. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |