Prev: Sought after architectures for the PostgreSQL buildfarm?
Next: [HACKERS] 答复: questions about concurrency control in Postgresql
From: Fujii Masao on 12 Jan 2010 11:58 On Tue, Dec 22, 2009 at 8:49 PM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: >> Umm.., I still cannot find the place where the walreceiver module is >> loaded by using load_external_function() in your 'replication' branch. >> Also the compilation of that branch fails. Is the 'pushed' branch the >> latest? Sorry if I'm missing something. > > Ah, I see. The changes were not included in the merge commit after all, > but I had simple forgot to "git add" them. Sorry about that, should be > there now. This change which moves walreceiver process into a dynamically loaded module caused the following compile error on my MinGW environment. --------------------------- gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv -g -I. -I../../../../src/interfaces/libpq -I../../../../src/include -I./src/include/port/win32 -DEXEC_BACKEND "-I../../../../src/include/port/win32" -DBUILDING_DLL -c -o walreceiverproc.o walreceiverproc.c dlltool --export-all --output-def libwalreceiverprocdll.def walreceiverproc.o dllwrap -o walreceiverproc.dll --dllname walreceiverproc.dll --def libwalreceiverprocdll.def walreceiverproc.o -L../../../../src/backend -lpostgres -L../../../../src/interfaces/libpq -L../../../../src/port -lpq Info: resolving _pg_signal_mask by linking to __imp__pg_signal_mask (auto-import) Info: resolving _pg_signal_queue by linking to __imp__pg_signal_queue (auto-import) Info: resolving _InterruptPending by linking to __imp__InterruptPending (auto-import) Info: resolving _assert_enabled by linking to __imp__assert_enabled (auto-import) Info: resolving _WalRcv by linking to __imp__WalRcv (auto-import) Info: resolving _proc_exit_inprogress by linking to __imp__proc_exit_inprogress (auto-import) Info: resolving _BlockSig by linking to __imp__BlockSig (auto-import) Info: resolving _sync_method by linking to __imp__sync_method (auto-import) Info: resolving _MyProcPid by linking to __imp__MyProcPid (auto-import) Info: resolving _CurrentResourceOwner by linking to __imp__CurrentResourceOwner (auto-import) Info: resolving _TopMemoryContext by linking to __imp__TopMemoryContext (auto-import) Info: resolving _CurrentMemoryContext by linking to __imp__CurrentMemoryContext (auto-import) Info: resolving _PG_exception_stack by linking to __imp__PG_exception_stack (auto-import) Info: resolving _UnBlockSig by linking to __imp__UnBlockSig (auto-import) Info: resolving _ThisTimeLineID by linking to __imp__ThisTimeLineID (auto-import) Info: resolving _error_context_stack by linking to __imp__error_context_stack (auto-import) Info: resolving _InterruptHoldoffCount by linking to __imp__InterruptHoldoffCount (auto-import) c:\MinGW\bin\..\lib\gcc\mingw32\3.4.2\..\..\..\..\mingw32\bin\ld.exe: warning: auto-importing has been activated without --enable-auto-import specified on the command line. This should work unless it involves constant data structures referencing symbols from auto-imported DLLs. fu000001.o:(.idata$2+0xc): undefined reference to `libpostgres_a_iname' fu000003.o:(.idata$2+0xc): undefined reference to `libpostgres_a_iname' fu000005.o:(.idata$2+0xc): undefined reference to `libpostgres_a_iname' fu000006.o:(.idata$2+0xc): undefined reference to `libpostgres_a_iname' fu000008.o:(.idata$2+0xc): undefined reference to `libpostgres_a_iname' fu000009.o:(.idata$2+0xc): more undefined references to `libpostgres_a_iname' follow nmth000000.o:(.idata$4+0x0): undefined reference to `_nm__pg_signal_mask' nmth000002.o:(.idata$4+0x0): undefined reference to `_nm__pg_signal_queue' nmth000004.o:(.idata$4+0x0): undefined reference to `_nm__InterruptPending' nmth000007.o:(.idata$4+0x0): undefined reference to `_nm__assert_enabled' nmth000012.o:(.idata$4+0x0): undefined reference to `_nm__WalRcv' nmth000018.o:(.idata$4+0x0): undefined reference to `_nm__proc_exit_inprogress' nmth000020.o:(.idata$4+0x0): undefined reference to `_nm__BlockSig' nmth000023.o:(.idata$4+0x0): undefined reference to `_nm__sync_method' nmth000026.o:(.idata$4+0x0): undefined reference to `_nm__MyProcPid' nmth000028.o:(.idata$4+0x0): undefined reference to `_nm__CurrentResourceOwner' nmth000030.o:(.idata$4+0x0): undefined reference to `_nm__TopMemoryContext' nmth000032.o:(.idata$4+0x0): undefined reference to `_nm__CurrentMemoryContext' nmth000035.o:(.idata$4+0x0): undefined reference to `_nm__PG_exception_stack' nmth000037.o:(.idata$4+0x0): undefined reference to `_nm__UnBlockSig' nmth000039.o:(.idata$4+0x0): undefined reference to `_nm__ThisTimeLineID' nmth000041.o:(.idata$4+0x0): undefined reference to `_nm__error_context_stack' nmth000043.o:(.idata$4+0x0): undefined reference to `_nm__InterruptHoldoffCount' collect2: ld returned 1 exit status c:\MinGW\bin\dllwrap.exe: c:\MinGW\bin\gcc exited with status 1 make[2]: *** [walreceiverproc.dll] Error 1 make[2]: Leaving directory `/c/postgres/mmm/src/backend/postmaster/walreceiverproc' make[1]: *** [all] Error 2 make[1]: Leaving directory `/c/postgres/mmm/src' make: *** [all] Error 2 --------------------------- Though I marked the variables shown in the above message as PGDLLIMPORT, the "make" still fails in the same way. I struggled with this issue for some time, but could not fix it yet :( Frankly I'm not familiar with that area. So it would be nice if someone could analyze this issue. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Magnus Hagander on 12 Jan 2010 13:37 On Tue, Jan 12, 2010 at 17:58, Fujii Masao <masao.fujii(a)gmail.com> wrote: > On Tue, Dec 22, 2009 at 8:49 PM, Heikki Linnakangas > <heikki.linnakangas(a)enterprisedb.com> wrote: >>> Umm.., I still cannot find the place where the walreceiver module is >>> loaded by using load_external_function() in your 'replication' branch. >>> Also the compilation of that branch fails. Is the 'pushed' branch the >>> latest? Sorry if I'm missing something. >> >> Ah, I see. The changes were not included in the merge commit after all, >> but I had simple forgot to "git add" them. Sorry about that, should be >> there now. > > This change which moves walreceiver process into a dynamically loaded > module caused the following compile error on my MinGW environment. That sounds strange - it should pick those up from the -lpostgres. Any chance you have an old postgres binary around from a non-syncrep build or something? > --------------------------- > > Though I marked the variables shown in the above message as PGDLLIMPORT, > the "make" still fails in the same way. I struggled with this issue > for some time, but > could not fix it yet :( > > Frankly I'm not familiar with that area. So it would be nice if > someone could analyze > this issue. Do you have an environment to try to build it under msvc? in my experience, that gives you easier-to-understand error messages in a lot of cases like this - it removets the mingw black magic. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Fujii Masao on 13 Jan 2010 03:47 Thanks for your advice! On Wed, Jan 13, 2010 at 3:37 AM, Magnus Hagander <magnus(a)hagander.net> wrote: >> This change which moves walreceiver process into a dynamically loaded >> module caused the following compile error on my MinGW environment. > > That sounds strange - it should pick those up from the -lpostgres. Any > chance you have an old postgres binary around from a non-syncrep build > or something? No, there is no old postgres binary. > Do you have an environment to try to build it under msvc? No, unfortunately. > in my > experience, that gives you easier-to-understand error messages in a > lot of cases like this - it removets the mingw black magic. OK. I'll try to build it under msvc. But since there seems to be a long way to go before doing that, I would appreciate if someone could give me some advice. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Heikki Linnakangas on 13 Jan 2010 05:27 Fujii Masao wrote: > Done. Currently there is no new libpq function for replication. The > walreceiver uses only existing functions like PQconnectdb, PQexec, > PQgetCopyData, etc. > > git://git.postgresql.org/git/users/fujii/postgres.git > branch: replication Thanks! I'm afraid we haven't quite nailed the select/poll issue yet. You copied pq_wait() from the libpq pqSocketCheck(), but there's one big difference between the backend and the frontend: the frontend always puts the connection to non-blocking mode, while the backend uses blocking mode. At least with SSL, I think it's possible for pq_wait() to return false positives, if the SSL layer decides to renegotiate the connection causing data to flow in the other direction in the underlying TCP connection. A false positive would lead cause walsender to block indefinitely on the pq_getbyte() call. I don't even want to think about the changes required to put the backend socket to non-blocking mode, I don't know that code well enough. Maybe we could temporarily put it to non-blocking mode, read to see if there's any data available, and put it back to blocking mode. But even then I think we'd need to modify at least secure_read() to work correctly with SSL in non-blocking mode. Another idea is to use poll() to check for POLLHUP, on those platforms that have poll(). AFAICS there is no equivalent for that in select(), so for platforms that don't have poll() we would have to simply ignore the issue or write some other platform-specific work-around (Windows WSAEventSelect() seems to have a FD_CLOSE event for that). That would be a quite localized change. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Fujii Masao on 14 Jan 2010 04:02
On Wed, Jan 13, 2010 at 7:27 PM, Heikki Linnakangas <heikki.linnakangas(a)enterprisedb.com> wrote: > the frontend always puts the > connection to non-blocking mode, while the backend uses blocking mode. Really? By default (i.e., without the expressly setting by using PQsetnonblocking()), the connection is set to blocking mode even in frontend. Am I missing something? > At least with SSL, I think it's possible for pq_wait() to return false > positives, if the SSL layer decides to renegotiate the connection > causing data to flow in the other direction in the underlying TCP > connection. A false positive would lead cause walsender to block > indefinitely on the pq_getbyte() call. Sorry. I could not understand that issue scenario. Could you explain it in more detail? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |