Command to prune archive at restartpoints
in [PgSql]
|
From: Andrew Dunstan on 10 Jun 2010 11:33 Heikki Linnakangas wrote: > On 10/06/10 17:38, Andrew Dunstan wrote: >> I think my logic needs a tiny piece of adjustment, to ignore the >> timeline segment of the file name. > > I'm not sure you should ignore it. Presumably anything in an older > timeline is indeed not required anymore and can be removed, and > anything in a newer timeline... how did it get there? Seems safer not > remove it. > Well, I was just following the logic in pg-standby.c: /* * We ignore the timeline part of the XLOG segment identifiers * in deciding whether a segment is still needed. This * ensures that we won't prematurely remove a segment from a * parent timeline. We could probably be a little more * proactive about removing segments of non-parent timelines, * but that would be a whole lot more complicated. * * We use the alphanumeric sorting property of the filenames * to decide which ones are earlier than the * exclusiveCleanupFileName file. Note that this means files * are not removed in the order they were originally written, * in case this worries you. */ if (strlen(xlde->d_name) == XLOG_DATA_FNAME_LEN && strspn(xlde->d_name, "0123456789ABCDEF") == XLOG_DATA_FNAME_LEN && strcmp(xlde->d_name + 8, exclusiveCleanupFileName + 8) < 0) cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 10 Jun 2010 10:17 On Thu, Jun 10, 2010 at 3:28 AM, Simon Riggs <simon(a)2ndquadrant.com> wrote: > On Thu, 2010-06-10 at 10:18 +0300, Heikki Linnakangas wrote: >> On 09/06/10 10:21, Simon Riggs wrote: >> > On Tue, 2010-06-08 at 18:30 -0400, Andrew Dunstan wrote: >> > >> >> I prefer archive_cleanup_command. We should name things after their >> >> principal function, not an implementation detail, IMNSHO. >> >> >> >> More importantly, we should include an example in the docs. I created >> >> one the other day �when this was actually bothering me a bit (see >> >> <http://people.planetpostgresql.org/andrew/index.php?/archives/85-Keeping-a-hot-standby-log-archive-clean.html>). >> >> That seemed to work ok, but maybe it's too long, and maybe people would >> >> prefer a shell script to perl. >> > >> > I submitted a patch to make the command "pg_standby -a %r" >> > >> > That's a more portable solution, ISTM. >> > >> > I'll commit that and fix the docs. >> >> Huh, wait. There's no -a option in pg_standby, so I presume you're >> planning to add that too. I don't like confusing pg_standby into this, >> the docs are currently quite clear that if you want to use the built-in >> standby mode, you can't use pg_standby, and this would muddy the waters. > > It won't kill us to change that sentence. "pg_standby is only used now > within the cleanup command" etc > > pg_standby already contains the exact logic we need here. Having two > sets of code for the same thing isn't how we do things. > >> Maybe we could add a new pg_cleanuparchive binary, but we'll need some >> discussion... > > Which will go nowhere, as we both already know. I have a feeling that I may be poking my nose into an incipient shouting match, but FWIW I agree with Heikki that it would be preferable to keep this separate from pg_standby. Considering that Andrew wrote this in 24 lines of Perl code (one-third of which are basically just there for logging purposes), I'm not that worried about code duplication, unless what we actually need is significantly more complicated. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Andrew Dunstan on 10 Jun 2010 10:38 Robert Haas wrote: >> It won't kill us to change that sentence. "pg_standby is only used now >> within the cleanup command" etc >> >> pg_standby already contains the exact logic we need here. Having two >> sets of code for the same thing isn't how we do things. >> Well, we could factor out that part of the code so it could be used in two binaries. But ... >>> Maybe we could add a new pg_cleanuparchive binary, but we'll need some >>> discussion... >>> >> Which will go nowhere, as we both already know. >> > > I have a feeling that I may be poking my nose into an incipient > shouting match, but FWIW I agree with Heikki that it would be > preferable to keep this separate from pg_standby. Considering that > Andrew wrote this in 24 lines of Perl code (one-third of which are > basically just there for logging purposes), I'm not that worried about > code duplication, unless what we actually need is significantly more > complicated. > > I think my logic needs a tiny piece of adjustment, to ignore the timeline segment of the file name. But that will hardly involve a great deal of extra code - just chop off the first 8 chars. It's not like the code for this in pg_standby.c is terribly complex. The virtue of a perl script is that it's very easily customizable, e.g. you might only delete files if they are older than a certain age. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 10 Jun 2010 11:19 On 10/06/10 17:38, Andrew Dunstan wrote: > I think my logic needs a tiny piece of adjustment, to ignore the > timeline segment of the file name. I'm not sure you should ignore it. Presumably anything in an older timeline is indeed not required anymore and can be removed, and anything in a newer timeline... how did it get there? Seems safer not remove it. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Dimitri Fontaine on 10 Jun 2010 15:24
Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> writes: > Maybe we could add a new pg_cleanuparchive binary, but we'll need some > discussion... Would this binary ever be used manually, not invoked by PostgreSQL? As it depends on the %r option to be given and to be right, I don't think so. Therefore my take on this problem is to provide internal commands here, that maybe wouldn't need to be explicitly passed any argument. If they're internal they certainly can access to the information they need? As a user, I'd find it so much better to trust PostgreSQL for proposing sane defaults. As a developer, you will certainly find it easier to maintain, document and distribute. While at it, the other internal command we need is pg_archive_bypass for the archive_command so that windows users have the /usr/bin/true option too. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |