max_standby_delay considered harmful
in [PgSql]
|
Prev: Further Hot Standby documentation required
Next: [HACKERS] Streaming replication - unable to stop the standby
From: Simon Riggs on 6 May 2010 08:13 On Thu, 2010-05-06 at 13:46 +0200, Florian Pflug wrote: > On May 6, 2010, at 12:48 , Simon Riggs wrote: > > On Thu, 2010-05-06 at 11:36 +0200, Florian Pflug wrote: > >> If there was an additional SQL-callable function that returned the backends the recovery process is currently waiting for, plus one that reported that last timestamp seen in the WAL, than all those different cancellation policies could be implemented as daemons that monitor recovery and kill backends as needed, no? > >> > >> That would allow people to experiment with different cancellation policies, and maybe shed some light on what the useful policies are in practice. > > > > It would be easier to implement a conflict resolution plugin that is > > called when a conflict occurs, allowing users to have a customisable > > mechanism. Again, I have no objection to that proposal. > > True, providing a plugin API would be even better, since no SQL callable API would have to be devised, and possible algorithms wouldn't be constrained by such an API's limitations. > > The existing max_standby_delay logic could be moved to such a plugin, living in contrib. Since it was already established (I believe) that the existing max_standby_delay logic is sufficiently fragile to require significant knowledge on the user's side about potential pitfalls, asking those users to install the plugin from contrib shouldn't be too much to ask for. > > This way, users who really need something more sophisticated than recovery wins always or standby wins always are given the tools they need *if* they're willing to put in the extra effort. For those who don't, offering max_standby_delay probably does more harm than good anyway, so nothing is lost by not offering it in the first place. No problem from me with that approach. As long as 9.0 ships with the current capability to enforce max_standby_delay, I have no problem. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Greg Smith on 6 May 2010 08:35 Heikki Linnakangas wrote: > Robert Haas wrote: > >> I am not convinced it will be unpredictable. The only caveats that >> I've seen so far are: >> >> - You need to run ntpd. >> - Queries will get cancelled like crazy if you're not using steaming >> replication. >> > > And also in situations where the master is idle for a while and then > starts doing stuff. That's the most significant source of confusion, > IMHO, I wouldn't mind the requirement of ntpd so much. > I consider it mandatory to include an documentation update here that says "if you set max_standby_delay > 0, and do not run something that regularly generates activity to the master like [example], you will get unnecessary query cancellation on the standby". As well as something like what Josh was suggesting, adding warnings that this is "for advanced users only", to borrow his wording. This is why my name has been on the open items list for a while now--to make sure I follow through on that. I haven't written it yet because there were still changes to the underlying code being made up until moments before beta started, then this discussion started without a break between. There are a clear set of user land things that can be done to make up the deficiencies in the state of the server code, but we won't even get to see how they work out in the field (feedback needed to improve the 9.1 design) if this capability goes away altogether. Is it not clear that there are some people who consider the occasional bit of cancellation OK, because they can correct for at the application layer and they're willing to factor it in to their design if it allows using the otherwise idle HA standby? I'm fine with expanding that section of the documentation too, to make it more obvious that's the only situation this aspect of HS is aimed at and suitable for. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support greg(a)2ndQuadrant.com www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Greg Smith on 6 May 2010 08:42 Yeb Havinga wrote: > Rob Wultsch wrote: >> I can not imagine setting this value to anything other than a bool and >> most of the time that bool would be -1. > That's funny because when I was reading this thread, I was thinking > the exact opposite: having max_standby_delay always set to 0 so I know > the standby server is as up-to-date as possible. If you ask one person about this, you'll discover they only consider one behavior here sane, and any other setting is crazy. Ask five people, and you'll likely find someone who believes the complete opposite. Ask ten and carefully work out the trade-offs they're willing to make given the fundamental limitations of replication, and you'll arrive at the range of behaviors available right now, plus some more that haven't been built yet. There are a lot of different types of database applications out there, each with their own reliability and speed requirements to balance. -- Greg Smith 2ndQuadrant US Baltimore, MD PostgreSQL Training, Services and Support greg(a)2ndQuadrant.com www.2ndQuadrant.us -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Dimitri Fontaine on 6 May 2010 10:09 Simon Riggs <simon(a)2ndQuadrant.com> writes: > It would be easier to implement a conflict resolution plugin that is > called when a conflict occurs, allowing users to have a customisable > mechanism. Again, I have no objection to that proposal. To implement, if you say so, no doubt. To use, that means you need to install a contrib module after validation that the trade offs there are the one you're interested into, or you have to code it yourself. In C. I don't see that as an improvement over what we have now. Our main problem seems to be the documentation of the max_standby_delay, where we give the impression it's doing things the code can not do. IIUC. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 6 May 2010 10:24
On Thu, 2010-05-06 at 16:09 +0200, Dimitri Fontaine wrote: > Simon Riggs <simon(a)2ndQuadrant.com> writes: > > It would be easier to implement a conflict resolution plugin that is > > called when a conflict occurs, allowing users to have a customisable > > mechanism. Again, I have no objection to that proposal. > > To implement, if you say so, no doubt. To use, that means you need to > install a contrib module after validation that the trade offs there are > the one you're interested into, or you have to code it yourself. In C. > > I don't see that as an improvement over what we have now. Our main > problem seems to be the documentation of the max_standby_delay, where we > give the impression it's doing things the code can not do. IIUC. I meant "easier to implement than what Florian suggested". The plugin would also allow you to have the pause/resume capability. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |