Bug-fix and new feature of pg_lesslog is released
in [PgSql]
|
Prev: pg_dump should not try CREATE OR REPLACE LANGUAGE < 9.0
Next: Bug-fix and new feature of pg_lesslogis released
From: Tom Lane on 12 May 2010 17:03 Bruce Momjian <bruce(a)momjian.us> writes: > Yes, I would love to get this into /contrib for PG 9.1! How much are people really going to care about pg_lesslog now that we've got streaming replication? There might be some small use-case still left, but it's hard to believe that it would be worth carrying it in contrib. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Magnus Hagander on 12 May 2010 17:09 On Wed, May 12, 2010 at 11:03 PM, Tom Lane <tgl(a)sss.pgh.pa.us> wrote: > Bruce Momjian <bruce(a)momjian.us> writes: >> Yes, I would love to get this into /contrib for PG 9.1! > > How much are people really going to care about pg_lesslog now that > we've got streaming replication? There might be some small use-case > still left, but it's hard to believe that it would be worth carrying > it in contrib. Until we have a tool to use streaming replication to actually archive backups, there's still a fairly large usecase for it... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Takahiro Itagaki on 13 May 2010 01:23 Tom Lane <tgl(a)sss.pgh.pa.us> wrote: > Bruce Momjian <bruce(a)momjian.us> writes: > > Yes, I would love to get this into /contrib for PG 9.1! > > How much are people really going to care about pg_lesslog now that > we've got streaming replication? There might be some small use-case > still left, but it's hard to believe that it would be worth carrying > it in contrib. I hope pg_lesslog would work as a WAL filter of streaming replication. It might be hard-coded in WAL sender, or be an addon based on a new common filtering infrastructure of WAL streaming. Also, there is a long-standing issue in pg_lesslog; It slows down recovery because we need to read data pages before write in recovery. We're avoiding reading pages for full-page image in 8.3, but pg_lesslog will disable the optimization. Recovery routine in core also needs to be adjusted to use read-ahead, like posix_fadvise(). There was another idea, full-page image logs separated with WAL logging. In theory, full-page images don't have to be written at commit, but only by writing corresponding data pages, I'm not sure whether it is an actually good idea or not, but if we go the direction, we won't need pg_lesslog. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Koichi Suzuki on 13 May 2010 02:13 Thanks a lot for the comment/advice. Yes, full page backup block considerablly shortens the recovery time. As we discussed about two years ago, I have a solution accelerate the recovery even without full page image. I'd like to submit this solution to the community again. When I evaluated this two years ago, recovery speed was as good as those with full page image, depending upon application and tuning, of course. This is a separate tool and can be used in various scenes. Regards; ---------- Koichi Suzuki 2010/5/13 Takahiro Itagaki <itagaki.takahiro(a)oss.ntt.co.jp>: > > Tom Lane <tgl(a)sss.pgh.pa.us> wrote: > >> Bruce Momjian <bruce(a)momjian.us> writes: >> > Yes, I would love to get this into /contrib for PG 9.1! >> >> How much are people really going to care about pg_lesslog now that >> we've got streaming replication? Â There might be some small use-case >> still left, but it's hard to believe that it would be worth carrying >> it in contrib. > > I hope pg_lesslog would work as a WAL filter of streaming replication. > It might be hard-coded in WAL sender, or be an addon based on a new > common filtering infrastructure of WAL streaming. > > Also, there is a long-standing issue in pg_lesslog; It slows down recovery > because we need to read data pages before write in recovery. We're avoiding > reading pages for full-page image in 8.3, but pg_lesslog will disable > the optimization. Recovery routine in core also needs to be adjusted to use > read-ahead, like posix_fadvise(). > > There was another idea, full-page image logs separated with WAL logging. > In theory, full-page images don't have to be written at commit, but only > by writing corresponding data pages, I'm not sure whether it is an actually > good idea or not, but if we go the direction, we won't need pg_lesslog. > > Regards, > --- > Takahiro Itagaki > NTT Open Source Software Center > > > -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Koichi Suzuki on 13 May 2010 02:21
WAL streming filter is an interesting idea. We can leave full page backup for local recovery and decrease the amount of WAL to transfer. If SR writes WAL in record by record basis, implementation will be simple. If SR writes WAL in block by block basis, WAL records may be divided from transfer to transfer and it is not simple to handle. Regards; ---------- Koichi Suzuki 2010/5/13 Takahiro Itagaki <itagaki.takahiro(a)oss.ntt.co.jp>: > > Tom Lane <tgl(a)sss.pgh.pa.us> wrote: > >> Bruce Momjian <bruce(a)momjian.us> writes: >> > Yes, I would love to get this into /contrib for PG 9.1! >> >> How much are people really going to care about pg_lesslog now that >> we've got streaming replication? Â There might be some small use-case >> still left, but it's hard to believe that it would be worth carrying >> it in contrib. > > I hope pg_lesslog would work as a WAL filter of streaming replication. > It might be hard-coded in WAL sender, or be an addon based on a new > common filtering infrastructure of WAL streaming. > > Also, there is a long-standing issue in pg_lesslog; It slows down recovery > because we need to read data pages before write in recovery. We're avoiding > reading pages for full-page image in 8.3, but pg_lesslog will disable > the optimization. Recovery routine in core also needs to be adjusted to use > read-ahead, like posix_fadvise(). > > There was another idea, full-page image logs separated with WAL logging. > In theory, full-page images don't have to be written at commit, but only > by writing corresponding data pages, I'm not sure whether it is an actually > good idea or not, but if we go the direction, we won't need pg_lesslog. > > Regards, > --- > Takahiro Itagaki > NTT Open Source Software Center > > > -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |