Streaming replication and non-blocking I/O
in [PgSql]
|
Prev: Sought after architectures for the PostgreSQL buildfarm?
Next: [HACKERS] 答复: questions about concurrency control in Postgresql
From: Heikki Linnakangas on 22 Dec 2009 06:49 Fujii Masao wrote: > On Tue, Dec 22, 2009 at 3:30 PM, Heikki Linnakangas > <heikki.linnakangas(a)enterprisedb.com> wrote: >> I think we can just use load_external_function() to load the library and >> call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the >> library name. Walreceiver is quite tightly coupled with the rest of the >> backend anyway, so I don't think we need to come up with a pluggable API >> at the moment. >> >> That's the way I did it yesterday, see 'replication' branch in my git >> repository, but it looks like I fumbled the commit so that some of the >> changes were committed as part of the merge commit with origin/master >> (=CVS HEAD). Sorry about that. > > Umm.., I still cannot find the place where the walreceiver module is > loaded by using load_external_function() in your 'replication' branch. > Also the compilation of that branch fails. Is the 'pushed' branch the > latest? Sorry if I'm missing something. Ah, I see. The changes were not included in the merge commit after all, but I had simple forgot to "git add" them. Sorry about that, should be there now. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 22 Dec 2009 07:47 Greg Stark wrote: > On Tue, Dec 22, 2009 at 6:30 AM, Heikki Linnakangas > <heikki.linnakangas(a)enterprisedb.com> wrote: >> I think we can just use load_external_function() to load the library and >> call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the >> library name. Walreceiver is quite tightly coupled with the rest of the >> backend anyway, so I don't think we need to come up with a pluggable API >> at the moment. > > Please? I am really interested in replacing walsender and walreceiver > with something which uses a communication bus like spread instead of a > single point to point connection. I think you'd still need to be able to request older WAL segments to resync after a lost connection, restore from base backup etc., which don't really fit into a publish/subscribe style communication bus. I'm sure it could all be solved though. It would be a pretty cool feature, for scaling to a large number of slaves. > ISTM if we start with something tightly coupled it'll be hard to > decouple later. Whereas if we start with a limited interface we'll > learn just how much information is really required by the modules and > will have fewer surprises later when we find suprising > interdependencies. I'm all ears if you have a concrete proposal. I'm not too worried about it being hard to decouple later. The interface is actually quite limited already, as the communication between processes is done via shared memory. It probably wouldn't be hard to turn it into an API, but I don't think there's a hurry to do that until someone actually steps up to write an alternative walreceiver/walsender, -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Fujii Masao on 22 Dec 2009 09:21 On Tue, Dec 22, 2009 at 8:49 PM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > Ah, I see. The changes were not included in the merge commit after all, > but I had simple forgot to "git add" them. Sorry about that, should be > there now. Thanks for doing "git push" again! But the compilation still fails. Attached patch addresses this problem. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
From: Heikki Linnakangas on 4 Jan 2010 10:22 I've merged the replication branch with PostgreSQL CVS HEAD now, including the patch for end-of-backup WAL records I committed earlier today. See 'replication' branch in my git repository. There's also a couple of other small changes: I believe the SSL stuff isn't really necessary, so I removed it. I also moved the START_REPLICATION phase from the walreceiver main loop to WalRcvConnect, as it's simpler that way. I will continue reviewing.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Fujii Masao on 5 Jan 2010 21:31
On Tue, Jan 5, 2010 at 12:22 AM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > I've merged the replication branch with PostgreSQL CVS HEAD now, > including the patch for end-of-backup WAL records I committed earlier > today. See 'replication' branch in my git repository. > > There's also a couple of other small changes: I believe the SSL stuff > isn't really necessary, so I removed it. I also moved the > START_REPLICATION phase from the walreceiver main loop to WalRcvConnect, > as it's simpler that way. I also fixed a couple of small bugs: * The ErrorResponse message from the primary server had been ignored * The segment-boundary had been wrongly handled * Valid replication starting location had been wrongly regarded as invalid git://git.postgresql.org/git/users/fujii/postgres.git branch: replication Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |