Stefan's bug (was: max_standby_delay considered harmful)
in [PgSql]
|
From: Fujii Masao on 18 May 2010 02:09 On Tue, May 18, 2010 at 2:26 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: > On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote: >> On Mon, May 17, 2010 at 9:01 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: >> >> (1) >> >> Smart or fast shutdown requested in PM_STARTUP state always removes >> >> the backup_label file if it exists. But it might be still required >> >> for subsequent recovery. I changed your patch so that additionally >> >> the postmaster skips deleting the backup_label in that case. >> > >> > Don't like the name NeedBackupLabel seems too specific. That really >> > corresponds to "we were in recovery". We should have a couple of >> > super-states that correspond to am in recovery/am not in recovery so we >> > can drive it from that. >> >> ISTM that we can use XLogCtl->SharedRecoveryInProgress for that. >> Is this OK? > > That can change state at any time. Would that work? Yes. XLogCtl->SharedRecoveryInProgress is set to TRUE only when XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's set to FALSE only at the end of recovery. Here is the updated patch (fix_smart_shutdown_in_recovery_v3_fujii.patch). I used XLogCtl->SharedRecoveryInProgress instead of NeedBackupLabel, which made the patch very simple. And I prevented the postmaster from invoking walreceiver after shutdown or recovery. >> (2) >> pg_ctl -ms stop emits the following warning whenever there is the >> backup_label file in $PGDATA. >> >> WARNING: online backup mode is active >> Shutdown will not complete until pg_stop_backup() is called. >> >> This warning doesn't fit in with the shutdown during recovery case. >> Since smart shutdown might be requested by other than pg_ctl, the >> warning should be emitted in server side rather than client, I think. >> How about moving the warning to the server side? Though I'm not sure if this should be fixed for 9.0, I attached the patch (move_bkp_cancel_warning_v1.patch). Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
From: Simon Riggs on 18 May 2010 02:24 On Tue, 2010-05-18 at 15:09 +0900, Fujii Masao wrote: > On Tue, May 18, 2010 at 2:26 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: > > On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote: > >> On Mon, May 17, 2010 at 9:01 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: > >> >> (1) > >> >> Smart or fast shutdown requested in PM_STARTUP state always removes > >> >> the backup_label file if it exists. But it might be still required > >> >> for subsequent recovery. I changed your patch so that additionally > >> >> the postmaster skips deleting the backup_label in that case. > >> > > >> > Don't like the name NeedBackupLabel seems too specific. That really > >> > corresponds to "we were in recovery". We should have a couple of > >> > super-states that correspond to am in recovery/am not in recovery so we > >> > can drive it from that. > >> > >> ISTM that we can use XLogCtl->SharedRecoveryInProgress for that. > >> Is this OK? > > > > That can change state at any time. Would that work? > > Yes. XLogCtl->SharedRecoveryInProgress is set to TRUE only when > XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's > set to FALSE only at the end of recovery. You should be using RecoveryInProgress() > Here is the updated patch (fix_smart_shutdown_in_recovery_v3_fujii.patch). > I used XLogCtl->SharedRecoveryInProgress instead of NeedBackupLabel, > which made the patch very simple. And I prevented the postmaster > from invoking walreceiver after shutdown or recovery. > > >> (2) > >> pg_ctl -ms stop emits the following warning whenever there is the > >> backup_label file in $PGDATA. > >> > >> WARNING: online backup mode is active > >> Shutdown will not complete until pg_stop_backup() is called. > >> > >> This warning doesn't fit in with the shutdown during recovery case. > >> Since smart shutdown might be requested by other than pg_ctl, the > >> warning should be emitted in server side rather than client, I think. > >> How about moving the warning to the server side? > > Though I'm not sure if this should be fixed for 9.0, I attached the > patch (move_bkp_cancel_warning_v1.patch). > > Regards, > -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Fujii Masao on 18 May 2010 02:45 On Tue, May 18, 2010 at 3:24 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: > On Tue, 2010-05-18 at 15:09 +0900, Fujii Masao wrote: >> On Tue, May 18, 2010 at 2:26 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: >> > On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote: >> >> On Mon, May 17, 2010 at 9:01 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: >> >> >> (1) >> >> >> Smart or fast shutdown requested in PM_STARTUP state always removes >> >> >> the backup_label file if it exists. But it might be still required >> >> >> for subsequent recovery. I changed your patch so that additionally >> >> >> the postmaster skips deleting the backup_label in that case. >> >> > >> >> > Don't like the name NeedBackupLabel seems too specific. That really >> >> > corresponds to "we were in recovery". We should have a couple of >> >> > super-states that correspond to am in recovery/am not in recovery so we >> >> > can drive it from that. >> >> >> >> ISTM that we can use XLogCtl->SharedRecoveryInProgress for that. >> >> Is this OK? >> > >> > That can change state at any time. Would that work? >> >> Yes. XLogCtl->SharedRecoveryInProgress is set to TRUE only when >> XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's >> set to FALSE only at the end of recovery. > > You should be using RecoveryInProgress() Isn't access to a bool variable atomic? And I think that postmaster should not take any locks since its deadlock would cause a fatal situation. No? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Fujii Masao on 18 May 2010 03:05 On Tue, May 18, 2010 at 3:45 PM, Fujii Masao <masao.fujii(a)gmail.com> wrote: > On Tue, May 18, 2010 at 3:24 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: >> On Tue, 2010-05-18 at 15:09 +0900, Fujii Masao wrote: >>> On Tue, May 18, 2010 at 2:26 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: >>> > On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote: >>> >> On Mon, May 17, 2010 at 9:01 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: >>> >> >> (1) >>> >> >> Smart or fast shutdown requested in PM_STARTUP state always removes >>> >> >> the backup_label file if it exists. But it might be still required >>> >> >> for subsequent recovery. I changed your patch so that additionally >>> >> >> the postmaster skips deleting the backup_label in that case. >>> >> > >>> >> > Don't like the name NeedBackupLabel seems too specific. That really >>> >> > corresponds to "we were in recovery". We should have a couple of >>> >> > super-states that correspond to am in recovery/am not in recovery so we >>> >> > can drive it from that. >>> >> >>> >> ISTM that we can use XLogCtl->SharedRecoveryInProgress for that. >>> >> Is this OK? >>> > >>> > That can change state at any time. Would that work? >>> >>> Yes. XLogCtl->SharedRecoveryInProgress is set to TRUE only when >>> XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's >>> set to FALSE only at the end of recovery. >> >> You should be using RecoveryInProgress() > > Isn't access to a bool variable atomic? If it's not atomic, I'll add the following comment into CancelBackup(): Don't bother with lock to access XLogCtl->SharedRecoveryInProgress, because there should be no other processes running when this code is reached. Thought? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 18 May 2010 04:10
On Tue, 2010-05-18 at 16:05 +0900, Fujii Masao wrote: > >>> >> ISTM that we can use XLogCtl->SharedRecoveryInProgress for that. > >>> >> Is this OK? > >>> > > >>> > That can change state at any time. Would that work? > >>> > >>> Yes. XLogCtl->SharedRecoveryInProgress is set to TRUE only when > >>> XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's > >>> set to FALSE only at the end of recovery. > >> > >> You should be using RecoveryInProgress() > > > > Isn't access to a bool variable atomic? > > If it's not atomic, I'll add the following comment into CancelBackup(): > > Don't bother with lock to access XLogCtl->SharedRecoveryInProgress, > because there should be no other processes running when this code > is reached. Call it via a function. There is no need for postmaster to know the innards of xlog.c, which could change in future. Modularity. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |