Prev: [HACKERS] Check constraints on non-immutable keys
Next: Proposal for 9.1: WAL streaming from WALbuffers
From: Robert Haas on 30 Jun 2010 20:52
2010/6/30 Tom Lane <tgl(a)sss.pgh.pa.us>:
> (thinks some more...) �Maybe you don't even need the fencepoint record
> per se. �I think all it's doing for you is making sure you don't process
> commit records on different streams out-of-order. �There might be some
> other, more direct way to do that.
>
> (thinks yet more...) �Actually the weak point in this scheme is that it
> wouldn't serialize transactions that occur in different databases and
> don't touch any shared catalogs. �It'd be entirely possible for T1 in
> DB1 to be reported committed, then T2 in DB2 to be reported committed,
> then a crash occurs after which T2 is seen committed and T1 not. �While
> this would be all right if the clients for T1 and T2 can't communicate,
> that isn't the real world.

Eh? If T1 and T2 are both reported committed, then they'll still be
committed after crash recovery, assuming synchronous_commit is turned
on. If not, our ACID has no D. Still, I suspect you're right that
there are serialization anomalies buried in here somewhere that can't
happen today.

And at any rate, the per-database thing isn't really the design goal,
anyway. It would be much nicer if we could find a way to support N>1
WAL streams without requiring that they be segregated by database.
We'd like to be able to write WAL faster, and commit faster, during
normal operation, and recover more quickly during recovery, especially
archive recovery.

You need to make sure not only that you replay commit records in
order, but also that, for example, you don't replay an
XLOG_HEAP2_CLEAN record too early.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Joe Conway on 30 Jun 2010 22:04
On 06/30/2010 05:52 PM, Robert Haas wrote:
> And at any rate, the per-database thing isn't really the design goal,
> anyway.

FWIW, I've run into more than one client where PITR and/or warm standby
on a per-database level would be a killer feature.

Joe

From: Tom Lane on 30 Jun 2010 22:21
Robert Haas <robertmhaas(a)gmail.com> writes:
> You need to make sure not only that you replay commit records in
> order, but also that, for example, you don't replay an
> XLOG_HEAP2_CLEAN record too early.

Hm, good point. That probably means that you *do* need fencepost
records, and furthermore that you might need an interlock to ensure that
you get the fencepost in early enough on the other stream. Ugh ---
there goes your concurrency.

What about having a single WAL stream for all commit records (thereby
avoiding any possible xact-serialization funnies) and other WAL records
divided up among multiple streams in some fashion or other? A commit
record would bear minimum-LSN pointers for all the streams that its
transaction had written to. Things like HEAP_CLEAN records would bear
minimum-LSN pointers for the commit stream. Workable?

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Robert Haas on 3 Jul 2010 07:02
2010/6/30 Tom Lane <tgl(a)sss.pgh.pa.us>:
> Robert Haas <robertmhaas(a)gmail.com> writes:
>> You need to make sure not only that you replay commit records in
>> order, but also that, for example, you don't replay an
>> XLOG_HEAP2_CLEAN record too early.
>
> Hm, good point. �That probably means that you *do* need fencepost
> records, and furthermore that you might need an interlock to ensure that
> you get the fencepost in early enough on the other stream. �Ugh ---
> there goes your concurrency.
>
> What about having a single WAL stream for all commit records (thereby
> avoiding any possible xact-serialization funnies) and other WAL records
> divided up among multiple streams in some fashion or other? �A commit
> record would bear minimum-LSN pointers for all the streams that its
> transaction had written to. �Things like HEAP_CLEAN records would bear
> minimum-LSN pointers for the commit stream. �Workable?

I don't see why not. Of course, the performance of any of these ideas
is another question altogether...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Simon Riggs on 3 Jul 2010 10:46
On Wed, 2010-06-30 at 22:21 -0400, Tom Lane wrote:

> What about having a single WAL stream for all commit records (thereby
> avoiding any possible xact-serialization funnies) and other WAL
> records
> divided up among multiple streams in some fashion or other? A commit
> record would bear minimum-LSN pointers for all the streams that its
> transaction had written to. Things like HEAP_CLEAN records would bear
> minimum-LSN pointers for the commit stream. Workable?

I'm interested in the idea of putting full page writes into one stream
and all other WAL records into another.

That would allow us to stream less data for log shipping.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Training and Services


--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

First  |  Prev  |  Next  |  Last
Pages: 1 2 3
Prev: [HACKERS] Check constraints on non-immutable keys
Next: Proposal for 9.1: WAL streaming from WALbuffers