Keepalive for max_standby_delay
in [PgSql]
|
Prev: [HACKERS] Keepalive for max_standby_delay
Next: Unexpected page allocation behavior on insert-only tables
From: Simon Riggs on 17 May 2010 03:07 On Mon, 2010-05-17 at 11:51 +0900, Fujii Masao wrote: > Is it OK that this keepalive message cannot be used by HS in file-based > log-shipping? Even in SR, the startup process cannot use the keepalive > until walreceiver has been started up. Yes, I see those things. We already have archive_timeout to handle the keepalive case in file-based. When starting up the delay is high anyway, so doesn't really matter about accuracy - though we do use latestXLogTime in that cases. > WalSndKeepAlive() always calls initStringInfo(), which seems to cause > memory-leak. Thanks. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 17 May 2010 04:40 On Sun, 2010-05-16 at 16:53 +0100, Simon Riggs wrote: > > > > Attached patch rearranges the walsender loops slightly to fix the above. > > XLogSend() now only sends up to MAX_SEND_SIZE bytes (== XLOG_SEG_SIZE / > > 2) in one round and returns to the main loop after that even if there's > > unsent WAL, and the main loop no longer sleeps if there's unsent WAL. > > That way the main loop gets to respond to signals quickly, and we also > > get to update the shared memory status and PS display more often when > > there's a lot of catching up to do. > > > > Comments > > 8MB at a time still seems like a large batch to me. > > libpq is going to send it in smaller chunks anyway, so I don't see the > importance of trying to keep the batch too large. It just introduces > delay into the sending process. We should be sending chunks that matches > libpq better. More to the point the logic will fail if XLOG_BLCKSZ > PQ_BUFFER_SIZE because it will send partial pages. Having MAX_SEND_SIZE > PQ_BUFFER_SIZE is pointless, as libpq currently stands. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Jim Nasby on 17 May 2010 12:36 On May 15, 2010, at 12:05 PM, Heikki Linnakangas wrote: > What exactly is the user trying to monitor? If it's "how far behind is > the standby", the difference between pg_current_xlog_insert_location() > in the master and pg_last_xlog_replay_location() in the standby seems > more robust and well-defined to me. It's a measure of XLOG location (ie. > bytes) instead of time, but time is a complicated concept. I can tell you that end users *will* want a time-based indication of how far behind we are. DBAs will understand "we're this many transactions behind", but managers and end users won't. Unless it's unreasonable to provide that info, we should do so. -- Jim C. Nasby, Database Architect jim(a)nasby.net 512.569.9461 (cell) http://jim.nasby.net -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 18 May 2010 17:06 On 17/05/10 04:40, Simon Riggs wrote: > On Sun, 2010-05-16 at 16:53 +0100, Simon Riggs wrote: >>> >>> Attached patch rearranges the walsender loops slightly to fix the above. >>> XLogSend() now only sends up to MAX_SEND_SIZE bytes (== XLOG_SEG_SIZE / >>> 2) in one round and returns to the main loop after that even if there's >>> unsent WAL, and the main loop no longer sleeps if there's unsent WAL. >>> That way the main loop gets to respond to signals quickly, and we also >>> get to update the shared memory status and PS display more often when >>> there's a lot of catching up to do. >>> >>> Comments >> >> 8MB at a time still seems like a large batch to me. >> >> libpq is going to send it in smaller chunks anyway, so I don't see the >> importance of trying to keep the batch too large. It just introduces >> delay into the sending process. We should be sending chunks that matches >> libpq better. > > More to the point the logic will fail if XLOG_BLCKSZ> PQ_BUFFER_SIZE > because it will send partial pages. I don't see a failure. We rely on not splitting WAL records across messages, but we're talking about libpq-level CopyData messages, not TCP messages. > Having MAX_SEND_SIZE> PQ_BUFFER_SIZE is pointless, as libpq currently > stands. Well, it does affect the size of the read() in walsender, and I'm sure there's some overhead in setting the ps display and the other small stuff we do once per message. But you're probably right that we could easily make MAX_SEND_SIZE much smaller with no noticeable affect on performance, while making walsender more responsive to signals. I'll decrease it to, say, 512 kB. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 18 May 2010 17:08
On 17/05/10 12:36, Jim Nasby wrote: > On May 15, 2010, at 12:05 PM, Heikki Linnakangas wrote: >> What exactly is the user trying to monitor? If it's "how far behind is >> the standby", the difference between pg_current_xlog_insert_location() >> in the master and pg_last_xlog_replay_location() in the standby seems >> more robust and well-defined to me. It's a measure of XLOG location (ie. >> bytes) instead of time, but time is a complicated concept. > > I can tell you that end users *will* want a time-based indication of how far behind we are. DBAs will understand "we're this many transactions behind", but managers and end users won't. Unless it's unreasonable to provide that info, we should do so. No doubt about that, the problem is that it's hard to provide a reliable time-based indication. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |