Hot Standby and VACUUM FULL
in [PgSql]
|
Prev: Add on_perl_init and proper destruction to plperl UPDATE v3[PATCH]
Next: [HACKERS] Package namespace and Safe init cleanup for plperl UPDATE 3 [PATCH]
From: Tom Lane on 30 Jan 2010 15:17 Simon Riggs <simon(a)2ndQuadrant.com> writes: > The last item on my list before close is making VACUUM FULL and Hot > Standby play nicely together. > The options to do this were and still are: > (1) Add WAL messages for non-transactional relcache invalidations > (2) Allow system relations to be cluster-ed/vacuum full-ed. > (1) was how we did it originally and I believe it worked without > problem. We saw the opportunity to do (2) and it has been worth > exploring. Refresh my memory on why (1) lets us avoid fixing (2)? It sounds like a kluge at best ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 30 Jan 2010 15:31 On Sat, 2010-01-30 at 15:17 -0500, Tom Lane wrote: > Simon Riggs <simon(a)2ndQuadrant.com> writes: > > The last item on my list before close is making VACUUM FULL and Hot > > Standby play nicely together. > > > The options to do this were and still are: > > > (1) Add WAL messages for non-transactional relcache invalidations > > (2) Allow system relations to be cluster-ed/vacuum full-ed. > > > (1) was how we did it originally and I believe it worked without > > problem. We saw the opportunity to do (2) and it has been worth > > exploring. > > Refresh my memory on why (1) lets us avoid fixing (2)? (1) allows us to retain VACUUM FULL INPLACE for system relations, thus avoiding the need to do (2). Non-transactional invalidations need to be replayed in recovery for the same reason they exist on the primary. > It sounds like a kluge at best ... (2) isn't a necessary change right now. It is the best design going forwards, but its burst radius stretches far beyond Hot Standby. There is enough code in HS for us to support, so adding to it makes little sense for me, in this release, since there is no functional benefit in doing so. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 31 Jan 2010 14:00 Simon Riggs wrote: > On Sat, 2010-01-30 at 15:17 -0500, Tom Lane wrote: >> Simon Riggs <simon(a)2ndQuadrant.com> writes: >>> The last item on my list before close is making VACUUM FULL and Hot >>> Standby play nicely together. >>> The options to do this were and still are: >>> (1) Add WAL messages for non-transactional relcache invalidations >>> (2) Allow system relations to be cluster-ed/vacuum full-ed. >>> (1) was how we did it originally and I believe it worked without >>> problem. We saw the opportunity to do (2) and it has been worth >>> exploring. >> Refresh my memory on why (1) lets us avoid fixing (2)? > > (1) allows us to retain VACUUM FULL INPLACE for system relations, thus > avoiding the need to do (2). Non-transactional invalidations need to be > replayed in recovery for the same reason they exist on the primary. > >> It sounds like a kluge at best ... > > (2) isn't a necessary change right now. It is the best design going > forwards, but its burst radius stretches far beyond Hot Standby. There > is enough code in HS for us to support, so adding to it makes little > sense for me, in this release, since there is no functional benefit in > doing so. Well, it'll avoid having to support the kludges in HS required for VACUUM FULL INPLACE. I'm in favor of (2), unless some unforeseen hard problems come up with implementing the ideas on that discussed earlier. Yeah, that's pretty vague, but basically I think it's worth spending some more time doing (2), than doing (1). For one, if the plan is to do (2) in next release anyway, we might as well do it now and avoid having to support the HS+VACUUM FULL INPLACE combination in only 9.0 stable branch for years to come. I believe we had a pretty well-thought out plan on how to support system catalogs with the new VACUUM FULL. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 31 Jan 2010 14:35 On Sun, 2010-01-31 at 21:00 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > On Sat, 2010-01-30 at 15:17 -0500, Tom Lane wrote: > >> Simon Riggs <simon(a)2ndQuadrant.com> writes: > >>> The last item on my list before close is making VACUUM FULL and Hot > >>> Standby play nicely together. > >>> The options to do this were and still are: > >>> (1) Add WAL messages for non-transactional relcache invalidations > >>> (2) Allow system relations to be cluster-ed/vacuum full-ed. > >>> (1) was how we did it originally and I believe it worked without > >>> problem. We saw the opportunity to do (2) and it has been worth > >>> exploring. > >> Refresh my memory on why (1) lets us avoid fixing (2)? > > > > (1) allows us to retain VACUUM FULL INPLACE for system relations, thus > > avoiding the need to do (2). Non-transactional invalidations need to be > > replayed in recovery for the same reason they exist on the primary. > > > >> It sounds like a kluge at best ... > > > > (2) isn't a necessary change right now. It is the best design going > > forwards, but its burst radius stretches far beyond Hot Standby. There > > is enough code in HS for us to support, so adding to it makes little > > sense for me, in this release, since there is no functional benefit in > > doing so. > > Well, it'll avoid having to support the kludges in HS required for > VACUUM FULL INPLACE. > > I'm in favor of (2), unless some unforeseen hard problems come up with > implementing the ideas on that discussed earlier. Yeah, that's pretty > vague, but basically I think it's worth spending some more time doing > (2), than doing (1). For one, if the plan is to do (2) in next release > anyway, we might as well do it now and avoid having to support the > HS+VACUUM FULL INPLACE combination in only 9.0 stable branch for years > to come. That's a good argument, but with respect, it isn't you who is writing the code, nor will it be you that supports it, AIUI. Right now, HS is isolated in many ways. If we introduce this change it will effect everybody and that means I'll be investigating all manner of bug reports that have zip to do with HS for a long time to come. What I would say is that for 9.0 we can easily remove the INPLACE option as an explicit request. > I believe we had a pretty well-thought out plan on how to support system > catalogs with the new VACUUM FULL. I think calling it a "well thought out plan" is, err, overstating things. We had what looked like a viable sketch of how to proceed and I agreed to investigate that. Having done so, I'm saying I don't like what I see going further and wish to backtrack to my known safe solution. Overall, I don't see any benefit in pursuing that course any further. I just see risk, on balance with (2), whereas (1) seems easier/faster, less risk and more isolated. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 31 Jan 2010 14:35
Simon Riggs <simon(a)2ndQuadrant.com> writes: > On Sat, 2010-01-30 at 15:17 -0500, Tom Lane wrote: >> Simon Riggs <simon(a)2ndQuadrant.com> writes: >>> The options to do this were and still are: >>> (1) Add WAL messages for non-transactional relcache invalidations >>> (2) Allow system relations to be cluster-ed/vacuum full-ed. >> Refresh my memory on why (1) lets us avoid fixing (2)? > (1) allows us to retain VACUUM FULL INPLACE for system relations, thus > avoiding the need to do (2). Non-transactional invalidations need to be > replayed in recovery for the same reason they exist on the primary. Well, I would expect that invalidation events need to be transmitted to hot-standby slaves no matter what --- backends running queries on an HS slave need to hear about inval events just as much as backends on the master do. So my take on it is that all inval events will have to have associated WAL records when in HS mode, independently of what we choose to do about VACUUM. Anyway, it's still not apparent to me exactly what the connection is between VACUUM FULL and Hot Standby. I remember that we said HS didn't work with VACUUM FULL (INPLACE) but I don't recall why that is, and the links on the open-items pages are not leading me to any useful discussion. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |