Idea for getting rid of VACUUM FREEZE on cold pages
in [PgSql]
|
Prev: [HACKERS] beta testing - planner bug - ERROR: XX000: failed to build any 2-way joins
Next: [HACKERS] beta testing - pg_upgrade bug fix - double free
From: Simon Riggs on 8 Jun 2010 17:08 On Tue, 2010-06-08 at 16:58 -0400, Robert Haas wrote: > On Tue, Jun 8, 2010 at 4:55 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: > > On Fri, 2010-06-04 at 10:18 -0400, Tom Lane wrote: > >> Jan Wieck <JanWieck(a)Yahoo.com> writes: > >> > On 6/2/2010 3:10 PM, Alvaro Herrera wrote: > >> >> I'd prefer a setting that would tell the system to freeze all tuples > >> >> that fall within a safety range whenever any tuple in the page is frozen > >> >> -- weren't you working on a patch to do this? (was it Jeff Davis?) > >> > >> > I just see a lot of cost caused by this "safety range". I yet have to > >> > see its real value, other than "feel good". > >> > >> Jan, you don't know what you're talking about. I have repeatedly had > >> cases where being able to look at xmin was critical to understanding > >> a bug. I *will not* hold still for a solution that effectively reduces > >> min_freeze_age to zero. > > > > Recent history shows Tom's view to be the most useful one: its useful to > > keep seeing the xmin. The last time we altered the way we set hint bits > > we caused multiple data loss bugs doing it. We will need to debug things > > and the WAL is always long gone (great idea though). > > > > Why not just have a separate flag for HEAP_XMIN_FROZEN, that way we can > > keep the xmin but also can see it is frozen? > > We could do that, but I think the point of this exercise is to reduce > I/O - specifically, I/O caused by anti-wraparound vacuums - and I'm > not clear how such a flag would help with that. Hmmm: You suggested a variant of this idea, so whatever reasoning was behind your suggestion would be shared here, surely? Tom has been saying we cannot freeze early because we need to keep xmins. I agree with that. This suggestion shows it is possible to freeze a tuple AND keep its xmin. So that removes the argument that we should freeze more aggressively (whenever we write the block) and can thus reduce longer term I/O costs. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 8 Jun 2010 18:03 On Tue, Jun 8, 2010 at 5:08 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: >> > Why not just have a separate flag for HEAP_XMIN_FROZEN, that way we can >> > keep the xmin but also can see it is frozen? >> >> We could do that, but I think the point of this exercise is to reduce >> I/O - specifically, I/O caused by anti-wraparound vacuums - and I'm >> not clear how such a flag would help with that. > > Hmmm: You suggested a variant of this idea, so whatever reasoning was > behind your suggestion would be shared here, surely? > > Tom has been saying we cannot freeze early because we need to keep > xmins. I agree with that. This suggestion shows it is possible to freeze > a tuple AND keep its xmin. So that removes the argument that we should > freeze more aggressively (whenever we write the block) and can thus > reduce longer term I/O costs. OK, yes, I see what you're getting at now. There are two possible ways to do freeze the tuples and keep the xmin: we can either rely on the PD_ALL_VISIBLE page-level bit (as I previously proposed) or we can additionally have a HEAP_XMIN_FROZEN bit as you propose here. I am not sure which way is better. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 8 Jun 2010 16:58 On Tue, Jun 8, 2010 at 4:55 PM, Simon Riggs <simon(a)2ndquadrant.com> wrote: > On Fri, 2010-06-04 at 10:18 -0400, Tom Lane wrote: >> Jan Wieck <JanWieck(a)Yahoo.com> writes: >> > On 6/2/2010 3:10 PM, Alvaro Herrera wrote: >> >> I'd prefer a setting that would tell the system to freeze all tuples >> >> that fall within a safety range whenever any tuple in the page is frozen >> >> -- weren't you working on a patch to do this? �(was it Jeff Davis?) >> >> > I just see a lot of cost caused by this "safety range". I yet have to >> > see its real value, other than "feel good". >> >> Jan, you don't know what you're talking about. �I have repeatedly had >> cases where being able to look at xmin was critical to understanding >> a bug. �I *will not* hold still for a solution that effectively reduces >> min_freeze_age to zero. > > Recent history shows Tom's view to be the most useful one: its useful to > keep seeing the xmin. The last time we altered the way we set hint bits > we caused multiple data loss bugs doing it. We will need to debug things > and the WAL is always long gone (great idea though). > > Why not just have a separate flag for HEAP_XMIN_FROZEN, that way we can > keep the xmin but also can see it is frozen? We could do that, but I think the point of this exercise is to reduce I/O - specifically, I/O caused by anti-wraparound vacuums - and I'm not clear how such a flag would help with that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 8 Jun 2010 18:18 On Tue, 2010-06-08 at 18:03 -0400, Robert Haas wrote: > OK, yes, I see what you're getting at now. There are two possible > ways to do freeze the tuples and keep the xmin: we can either rely on > the PD_ALL_VISIBLE page-level bit (as I previously proposed) or we can > additionally have a HEAP_XMIN_FROZEN bit as you propose here. I am > not sure which way is better. Doing it at tuple level is more flexible and allows more aggressive freezing. It also works better with existing tuple visibility code. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 8 Jun 2010 16:55
On Fri, 2010-06-04 at 10:18 -0400, Tom Lane wrote: > Jan Wieck <JanWieck(a)Yahoo.com> writes: > > On 6/2/2010 3:10 PM, Alvaro Herrera wrote: > >> I'd prefer a setting that would tell the system to freeze all tuples > >> that fall within a safety range whenever any tuple in the page is frozen > >> -- weren't you working on a patch to do this? (was it Jeff Davis?) > > > I just see a lot of cost caused by this "safety range". I yet have to > > see its real value, other than "feel good". > > Jan, you don't know what you're talking about. I have repeatedly had > cases where being able to look at xmin was critical to understanding > a bug. I *will not* hold still for a solution that effectively reduces > min_freeze_age to zero. Recent history shows Tom's view to be the most useful one: its useful to keep seeing the xmin. The last time we altered the way we set hint bits we caused multiple data loss bugs doing it. We will need to debug things and the WAL is always long gone (great idea though). Why not just have a separate flag for HEAP_XMIN_FROZEN, that way we can keep the xmin but also can see it is frozen? We already WAL-log certain flag settings, so why not this one also? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |