Streaming replication and non-blocking I/O
in [PgSql]
|
Prev: Sought after architectures for the PostgreSQL buildfarm?
Next: [HACKERS] 答复: questions about concurrency control in Postgresql
From: Tom Lane on 15 Jan 2010 18:59 Andrew Dunstan <andrew(a)dunslane.net> writes: > I ran an extra cycle. Still a bit of work to do: > <http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=dawn_bat&dt=2010-01-15%2023:04:54> Well, at least now we're down to the variables that haven't got PGDLLIMPORT, rather than wondering what's wrong with the build ... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 16 Jan 2010 02:49 Tom Lane wrote: > Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> writes: >> Before we sprinkle all the global variables it touches with that, let me >> explain what I meant by dividing walreceiver code differently between >> dynamically loaded module and backend code. Right now I have to go to >> sleep, though, but I'll try to get back to during the weekend. > > Yeah, nothing to be done till we get another buildfarm cycle anyway. Ok, looks like you did that anyway, let's see if it fixed it. Thanks. So what I'm playing with is to pull walreceiver back into the backend executable. To avoid the link dependency, walreceiver doesn't access libpq directly, but loads a module dynamically which implements this interface: bool walrcv_connect(char *conninfo, XLogRecPtr startpoint) Establish connection to the primary, and starts streaming from 'startpoint'. Returns true on success. bool walrcv_receive(int timeout, XLogRecPtr *recptr, char **buffer, int *len) Retrieve any WAL record available through the connection, blocking for maximum of 'timeout' ms. void walrcv_disconnect(void); Disconnect. This is the kind of API Greg Stark requested earlier (http://archives.postgresql.org/message-id/407d949e0912220336u595a05e0x20bd91b9fbc08d4d(a)mail.gmail.com), though I'm not planning to make it pluggable for 3rd party implementations yet. The module doesn't need to touch backend internals much at all, no tinkering with shared memory for example, so I would feel much better about moving that out of src/backend. Not sure where, though; it's not an executable, so src/bin is hardly the right place, but I wouldn't want to put it in contrib either, because it should still be built and installed by default. So I'm inclined to still leave it in src/backend/replication/ I've pushed that 'replication-dynmodule' branch in my git repo. The diff is hard to read, because it mostly just moves code around, but I've attached libpqwalreceiver.c here, which is the dynamic module part. You can also browse the tree via the web interface (http://git.postgresql.org/gitweb?p=users/heikki/postgres.git;a=tree;h=refs/heads/replication-dynmodule;hb=replication-dynmodule) I like this division of labor much more than making the whole walreceiver process a dynamically loaded module, so barring objections I will review and test this more, and commit next week. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
From: Heikki Linnakangas on 16 Jan 2010 02:54 Heikki Linnakangas wrote: > I've pushed that 'replication-dynmodule' branch in my git repo. The diff > is hard to read, because it mostly just moves code around, but I've > attached libpqwalreceiver.c here, which is the dynamic module part. You > can also browse the tree via the web interface > (http://git.postgresql.org/gitweb?p=users/heikki/postgres.git;a=tree;h=refs/heads/replication-dynmodule;hb=replication-dynmodule) I just noticed that the comment at the top of libpqwalreceiver.c is a leftover, not much relevant to the contents of the file anymore, all the signal handling and interaction with startup process is in src/backend/replication/walreceiver.c now. That obviously needs to be fixed before committing.. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Dimitri Fontaine on 16 Jan 2010 07:55 Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> writes: > The module doesn't need to touch backend internals much at all, no > tinkering with shared memory for example, so I would feel much better > about moving that out of src/backend. Not sure where, though; it's not > an executable, so src/bin is hardly the right place, but I wouldn't want > to put it in contrib either, because it should still be built and > installed by default. So I'm inclined to still leave it in > src/backend/replication/ It should be possible to be in contrib and installed by default, even with the current tool set, by tweaking initdb to install the contrib into template1. But that would be a packaging / dependency issue I guess then. Of course the extension system would ideally "create extension foo;" for all foo in contrib at initdb time, then a user would have to "install extension foo;" and be done with it. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Euler Taveira de Oliveira on 16 Jan 2010 08:21
Dimitri Fontaine escreveu: > It should be possible to be in contrib and installed by default, even > And it could be uninstall too. Let's not do it for core functionalities. -- Euler Taveira de Oliveira http://www.timbira.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |