pg_start_backup and pg_stop_backup Re: [HACKERS] Re:[COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend uponcorrect
in [PgSql]
|
Prev: explicit (void *) casts
Next: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS]pgsql: Make CheckRequiredParameterValues() depend upon correct
From: Aidan Van Dyk on 28 Apr 2010 15:04 * Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> [100428 14:49]: > You'd need a stand-alone tool to do the streaming in that case, and no > such tool exists yet, but I would be surprised if one doesn't appear on > pgfoundry sooner or later :-). And this tool is something I will eventually be interested in working on or collaborating on... I'm hoping to be able to build a tool that: 1) Connects to PG walsender (a la walreceiver) 2) Streams WAL from pg master 3) Saves WAL into "files" (a la archive)... i.e. I'm looking to keep a more-up-to-date PITR archive than waiting for traditional WAL file archiving... And eventually (9.1+) I'm hoping that walsender will have grown enough to allow me to configure PG to wait on the commit until the master has both sync'ed the WAL file, and received a "sync ack" from my wal-stream-save-to-file tool... Because then I'll have a situation where I can easily have a synchronous, separate machine copy of all my WAL without having to jump through hoops with stuff like drbd or MD+nbd, etc as my WAL disk... And yes, I don't personally care about streaming replication replaying WAL as it comes, or running queries in recovery... I'm looking towards PG not saying my transaction is committed unless it's safely on that machines disks (or BBcache) *and* another machine... That's the type of replication a paranoid guy like me waits for... Yes, that's possible now with exotic os/net/fs configuration, but imagine how nice it will be when it can all be done in userspace with just PG (and pg-compatible) tool, etc... -- Aidan Van Dyk Create like a god, aidan(a)highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
From: Aidan Van Dyk on 28 Apr 2010 16:38 * Kevin Grittner <Kevin.Grittner(a)wicourts.gov> [100428 15:51]: > > I don't personally care about streaming replication replaying WAL > > as it comes, or running queries in recovery... > > I'm with you that far, but I wouldn't want the sender to wait for > remote persistence. I remember a presentation at pgcon a while ago, it was probaly Fujii (from NTT?) about their log streaming, and at that time, they talked about different "sync" options... So I'ld love to be able to have comits be: async (like current option) local wal sync (like current) local wal sync + walsender sent local wal sync + walsender confirmed And ideally, the "walsender sent/confirmed" would even allow making sure it was sent/confirmed to $X connections... I want to be able to guarantee it's on 2 machines, not that if my slave was connected it would be on there, but something happened and my "slave" has disconnected, so it's only got local WAL... And then on whatever "tool" is receiving the log streaming, it can be set to confirm when either: received buffer write buffer to file write buffer to file + sync write buffer to file + sync + replay That should give you all the sync levels they talked about in their presentation... -- Aidan Van Dyk Create like a god, aidan(a)highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
From: Bruce Momjian on 30 Apr 2010 12:22 Tom Lane wrote: > Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> writes: > > Tom Lane wrote: > >> If you aren't archiving then there's no guarantee that you'll still have > >> a continuous WAL series starting from the start of the backup. > > > I wasn't really thinking of this use case, but you could set > > wal_keep_segments "high enough". > > Ah. Okay, that seems like a workable approach, at least for people with > reasonably predictable WAL loads. We could certainly improve on it > later to make it more bulletproof, but it's usable now --- if we relax > the error checks. > > (wal_keep_segments can be changed without restarting, right?) Should we allow -1 to mean "keep all segments"? -- Bruce Momjian <bruce(a)momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Bruce Momjian on 30 Apr 2010 13:39 Robert Haas wrote: > On Fri, Apr 30, 2010 at 12:22 PM, Bruce Momjian <bruce(a)momjian.us> wrote: > > Tom Lane wrote: > >> Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> writes: > >> > Tom Lane wrote: > >> >> If you aren't archiving then there's no guarantee that you'll still have > >> >> a continuous WAL series starting from the start of the backup. > >> > >> > I wasn't really thinking of this use case, but you could set > >> > wal_keep_segments "high enough". > >> > >> Ah. ?Okay, that seems like a workable approach, at least for people with > >> reasonably predictable WAL loads. ?We could certainly improve on it > >> later to make it more bulletproof, but it's usable now --- if we relax > >> the error checks. > >> > >> (wal_keep_segments can be changed without restarting, right?) > > > > Should we allow -1 to mean "keep all segments"? > > If that's what you want to do, use archive_mode. Uh, I assume that will require me to store the WAL files somewhere else, rather than keeping them in /pg_xlog, which I thought was the goal. Am I missing something? -- Bruce Momjian <bruce(a)momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Bruce Momjian on 30 Apr 2010 13:49
Simon Riggs wrote: > On Fri, 2010-04-30 at 12:22 -0400, Bruce Momjian wrote: > > > > > > (wal_keep_segments can be changed without restarting, right?) > > > > Should we allow -1 to mean "keep all segments"? > > Why is that not called "max_wal_segments"? wal_keep_segments sounds like > its been through Google translate. LOL, good one. I assume it was done so it would start with 'wal', but I see 'max_wal_senders', which doesn't start with 'wal' and would match your suggestion exactly. I think we should either rename 'wal_keep_segments' or 'max_wal_senders'. -- Bruce Momjian <bruce(a)momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |