Streaming replication and non-blocking I/O
in [PgSql]
|
Prev: Sought after architectures for the PostgreSQL buildfarm?
Next: [HACKERS] 答复: questions about concurrency control in Postgresql
From: Fujii Masao on 21 Dec 2009 22:18 On Tue, Dec 22, 2009 at 2:31 AM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > 2. Move walreceiver altogether into a loadable module, which is linked > as usual to libpq. Like e.g contrib/dblink. > > Thoughts? Both seem reasonable to me. I tested the 2nd option (see > 'replication' branch in my git repository), splitting walreceiver.c into > two: the functions that run in the walreceiver process, and the > functions that are called from other processes to control walreceiver. > That's a quite nice separation, though of course we could do that with > the 1st approach as well. Though I seem not to understand what a loadable module means, I wonder how the walreceiver module is loaded. AFAIK, we need to manually install the dblink functions by executing dblink.sql before using them. Likewise, if we choose the 2nd option, we must manually install the walreceiver module before starting replication? Or we automatically install that by executing system_view.sql, like pg_start_backup? I'd like to reduce the number of installation operations as much as possible. Is my concern besides the point? > PS. I just merged with CVS HEAD. Streaming replication is pretty awesome > with Hot Standby! Thanks! Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 21 Dec 2009 23:46 Fujii Masao <masao.fujii(a)gmail.com> writes: > Though I seem not to understand what a loadable module means, I wonder > how the walreceiver module is loaded. Put it in shared_preload_libraries, perhaps. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 22 Dec 2009 01:30 Fujii Masao wrote: > On Tue, Dec 22, 2009 at 2:31 AM, Heikki Linnakangas > <heikki.linnakangas(a)enterprisedb.com> wrote: >> 2. Move walreceiver altogether into a loadable module, which is linked >> as usual to libpq. Like e.g contrib/dblink. >> >> Thoughts? Both seem reasonable to me. I tested the 2nd option (see >> 'replication' branch in my git repository), splitting walreceiver.c into >> two: the functions that run in the walreceiver process, and the >> functions that are called from other processes to control walreceiver. >> That's a quite nice separation, though of course we could do that with >> the 1st approach as well. > > Though I seem not to understand what a loadable module means, I wonder > how the walreceiver module is loaded. AFAIK, we need to manually install > the dblink functions by executing dblink.sql before using them. Likewise, > if we choose the 2nd option, we must manually install the walreceiver > module before starting replication? I think we can just use load_external_function() to load the library and call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the library name. Walreceiver is quite tightly coupled with the rest of the backend anyway, so I don't think we need to come up with a pluggable API at the moment. That's the way I did it yesterday, see 'replication' branch in my git repository, but it looks like I fumbled the commit so that some of the changes were committed as part of the merge commit with origin/master (=CVS HEAD). Sorry about that. shared_preload_libraries seems like a bad place because the library doesn't need to be loaded in all backends. Just the walreceiver process. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Fujii Masao on 22 Dec 2009 02:21 On Tue, Dec 22, 2009 at 3:30 PM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > I think we can just use load_external_function() to load the library and > call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the > library name. Walreceiver is quite tightly coupled with the rest of the > backend anyway, so I don't think we need to come up with a pluggable API > at the moment. > > That's the way I did it yesterday, see 'replication' branch in my git > repository, but it looks like I fumbled the commit so that some of the > changes were committed as part of the merge commit with origin/master > (=CVS HEAD). Sorry about that. Umm.., I still cannot find the place where the walreceiver module is loaded by using load_external_function() in your 'replication' branch. Also the compilation of that branch fails. Is the 'pushed' branch the latest? Sorry if I'm missing something. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Greg Stark on 22 Dec 2009 06:36
On Tue, Dec 22, 2009 at 6:30 AM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > I think we can just use load_external_function() to load the library and > call WalReceiverMain from AuxiliaryProcessMain(). Ie. hard-code the > library name. Walreceiver is quite tightly coupled with the rest of the > backend anyway, so I don't think we need to come up with a pluggable API > at the moment. Please? I am really interested in replacing walsender and walreceiver with something which uses a communication bus like spread instead of a single point to point connection. ISTM if we start with something tightly coupled it'll be hard to decouple later. Whereas if we start with a limited interface we'll learn just how much information is really required by the modules and will have fewer surprises later when we find suprising interdependencies. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |