Prev: pgsql: Make standby server continuously retry restoringthe next WAL
Next: pg_restore --single-transaction and --clean
From: Fujii Masao on 5 Apr 2010 05:03
On Mon, Feb 15, 2010 at 3:45 PM, Fujii Masao <masao.fujii(a)gmail.com> wrote:
> On Fri, Feb 12, 2010 at 11:46 PM, Tom Lane <tgl(a)sss.pgh.pa.us> wrote:
>> Even more to the point is that some of them, like PGPORT, are highly
>> likely to be set in a server's environment to point to the server
>> itself.  It would be extremely dangerous to automatically try to start
>> replication just because we find those set.  In fact, I would argue that
>> we should fix things so that any such variables inherited from the
>> server environment are intentionally *NOT* used for making SR
>> connections.

This Tom's complaint is listed as a TODO item. How should we treat this?

I'm leaning toward postponing the item to v9.1 or later. Currently the
server during recovery doesn't accept the replication connection. So
it's not so dangerous for walreceiver to use the environment variables
which might point to the server itself, I think. That connection is
always refused.

Let us revisit this issue when we allow the standby server to accept the
replication connection from another standby? And I think that we should
prevent the standby from accepting the connection from its walreceiver,
rather than prevent the standby from using the environment variables.

Regards,

--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Robert Haas on 5 Apr 2010 07:11
On Mon, Apr 5, 2010 at 4:46 AM, Simon Riggs <simon(a)2ndquadrant.com> wrote:
> On Wed, 2010-03-31 at 23:54 +0300, Heikki Linnakangas wrote:
>> Robert Haas wrote:
>> > Agreed.  I think if the server starts up in standby mode and it is an
>> > inconsistent state with no source of WAL, then the startup process
>> > should exit with a suitable error message, which AIUI will result in
>> > the whole server shutting down.  However if there is no source of WAL
>> > but the server is in a consistent state, then I think we should allow
>> > it to start up as a read-only standby.
>> >
>> > Now, an interesting question is - if the server is in this state, and
>> > somebody manually drops more WAL into pg_xlog, what happens? And what
>> > happens in the similar case where primary_conninfo is set but we can't
>> > connect to the master at the moment, and someone drops a pile of WAL
>> > on us?
>>
>> With the recent changes to the retry logic
>> (http://archives.postgresql.org/pgsql-committers/2010-03/msg00356.php),
>> they will be replayed. Even if neither primary_conninfo or
>> restore_command is given, the server will still keep polling pg_xlog,
>> and if you copy a WAL file to standby's pg_xlog directory, it will be
>> replayed and recovery will make progress.
>>
>> I wouldn't recommend setting up a standby server like that, but it's not
>> totally unreasonable. So the standby always has a potential source of
>> WAL, pg_xlog.
>
> I have inadvertently made it impossible to specify
>   standby_mode && (!primary_conninfo && !restore_command)
>
> I did that because Robert had separately to this thread reported a hang,
> caused by this specification. I have verified this.

I don't remember reporting this (or maybe you meant the other Robert);
but there are so many threads on this topic that it's hard to keep
track of them all. Can you refresh my memory?

> pg_xlog is a *potential* source of WAL, but if the files requested are
> not present then the server just sits and waits with *no* messages. That
> is unacceptable, IMHO.
>
> What should we do now?

Well, actually, what it does for me is sits there and prints the last
xlog location over and over again every 2s. I'd actually like to get
to "sits and waits with no messages", but it's not clear how to do
that.

....Robert

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Simon Riggs on 5 Apr 2010 07:34
On Mon, 2010-04-05 at 07:11 -0400, Robert Haas wrote:
> On Mon, Apr 5, 2010 at 4:46 AM, Simon Riggs <simon(a)2ndquadrant.com> wrote:
> > On Wed, 2010-03-31 at 23:54 +0300, Heikki Linnakangas wrote:
> >> Robert Haas wrote:
> >> > Agreed. I think if the server starts up in standby mode and it is an
> >> > inconsistent state with no source of WAL, then the startup process
> >> > should exit with a suitable error message, which AIUI will result in
> >> > the whole server shutting down. However if there is no source of WAL
> >> > but the server is in a consistent state, then I think we should allow
> >> > it to start up as a read-only standby.
> >> >
> >> > Now, an interesting question is - if the server is in this state, and
> >> > somebody manually drops more WAL into pg_xlog, what happens? And what
> >> > happens in the similar case where primary_conninfo is set but we can't
> >> > connect to the master at the moment, and someone drops a pile of WAL
> >> > on us?
> >>
> >> With the recent changes to the retry logic
> >> (http://archives.postgresql.org/pgsql-committers/2010-03/msg00356.php),
> >> they will be replayed. Even if neither primary_conninfo or
> >> restore_command is given, the server will still keep polling pg_xlog,
> >> and if you copy a WAL file to standby's pg_xlog directory, it will be
> >> replayed and recovery will make progress.
> >>
> >> I wouldn't recommend setting up a standby server like that, but it's not
> >> totally unreasonable. So the standby always has a potential source of
> >> WAL, pg_xlog.
> >
> > I have inadvertently made it impossible to specify
> > standby_mode && (!primary_conninfo && !restore_command)
> >
> > I did that because Robert had separately to this thread reported a hang,
> > caused by this specification. I have verified this.
>
> I don't remember reporting this (or maybe you meant the other Robert);
> but there are so many threads on this topic that it's hard to keep
> track of them all. Can you refresh my memory?
>
> > pg_xlog is a *potential* source of WAL, but if the files requested are
> > not present then the server just sits and waits with *no* messages. That
> > is unacceptable, IMHO.
> >
> > What should we do now?
>
> Well, actually, what it does for me is sits there and prints the last
> xlog location over and over again every 2s. I'd actually like to get
> to "sits and waits with no messages", but it's not clear how to do
> that.

That's exactly the opposite of your report. Thread you started, on
hackers, in last week or so.

It's not clear to me *why* you would want it to sit there doing nothing,
and even if that has a purpose, saying nothing at all is not useful.
(Note that it cannot enter Hot Standby mode even in that state).

--
Simon Riggs www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Robert Haas on 5 Apr 2010 07:42
On Mon, Apr 5, 2010 at 7:34 AM, Simon Riggs <simon(a)2ndquadrant.com> wrote:
> On Mon, 2010-04-05 at 07:11 -0400, Robert Haas wrote:
>> On Mon, Apr 5, 2010 at 4:46 AM, Simon Riggs <simon(a)2ndquadrant.com> wrote:
>> > On Wed, 2010-03-31 at 23:54 +0300, Heikki Linnakangas wrote:
>> >> Robert Haas wrote:
>> >> > Agreed.  I think if the server starts up in standby mode and it is an
>> >> > inconsistent state with no source of WAL, then the startup process
>> >> > should exit with a suitable error message, which AIUI will result in
>> >> > the whole server shutting down.  However if there is no source of WAL
>> >> > but the server is in a consistent state, then I think we should allow
>> >> > it to start up as a read-only standby.
>> >> >
>> >> > Now, an interesting question is - if the server is in this state, and
>> >> > somebody manually drops more WAL into pg_xlog, what happens? And what
>> >> > happens in the similar case where primary_conninfo is set but we can't
>> >> > connect to the master at the moment, and someone drops a pile of WAL
>> >> > on us?
>> >>
>> >> With the recent changes to the retry logic
>> >> (http://archives.postgresql.org/pgsql-committers/2010-03/msg00356.php),
>> >> they will be replayed. Even if neither primary_conninfo or
>> >> restore_command is given, the server will still keep polling pg_xlog,
>> >> and if you copy a WAL file to standby's pg_xlog directory, it will be
>> >> replayed and recovery will make progress.
>> >>
>> >> I wouldn't recommend setting up a standby server like that, but it's not
>> >> totally unreasonable. So the standby always has a potential source of
>> >> WAL, pg_xlog.
>> >
>> > I have inadvertently made it impossible to specify
>> >   standby_mode && (!primary_conninfo && !restore_command)
>> >
>> > I did that because Robert had separately to this thread reported a hang,
>> > caused by this specification. I have verified this.
>>
>> I don't remember reporting this (or maybe you meant the other Robert);
>> but there are so many threads on this topic that it's hard to keep
>> track of them all.  Can you refresh my memory?
>>
>> > pg_xlog is a *potential* source of WAL, but if the files requested are
>> > not present then the server just sits and waits with *no* messages. That
>> > is unacceptable, IMHO.
>> >
>> > What should we do now?
>>
>> Well, actually, what it does for me is sits there and prints the last
>> xlog location over and over again every 2s.  I'd actually like to get
>> to "sits and waits with no messages", but it's not clear how to do
>> that.
>
> That's exactly the opposite of your report. Thread you started, on
> hackers, in last week or so.

Which thread? What was the subject line? The only thing I remember
saying about this was:

http://archives.postgresql.org/pgsql-hackers/2010-03/msg01247.php

> It's not clear to me *why* you would want it to sit there doing nothing,
> and even if that has a purpose, saying nothing at all is not useful.
> (Note that it cannot enter Hot Standby mode even in that state).

Actually it can, if the database state is consistent. Anyway, this
was already discussed upthread... feel free to put in your $0.02.

....Robert

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Simon Riggs on 5 Apr 2010 14:45
On Mon, 2010-04-05 at 18:03 +0900, Fujii Masao wrote:

> I'm leaning toward postponing the item to v9.1 or later.

If you want to defer anything, then I'd like to get a summary of what
you are thinking of deferring and why that is acceptable. Right now
there are lots of unfinished items and no movement on them. Yes, I'm
unhappy about that.

My feeling is that we should only default on "port" as part of the
primary_conninfo. All other settings are required or we should reject
streaming. Replication connections should be explicit.

--
Simon Riggs www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

First  |  Prev  | 
Pages: 1 2 3 4 5 6
Prev: pgsql: Make standby server continuously retry restoringthe next WAL
Next: pg_restore --single-transaction and --clean