[PATCH 4/4] Add tests to dblink covering use of COPY TO FUNCTION
in [PgSql]
|
Prev: [HACKERS] [PATCH 1/4] Add "COPY ... TO FUNCTION ..." support
Next: Backup history file should be replicated in Streaming Replication?
From: Pavel Stehule on 24 Nov 2009 23:45 2009/11/25 Jeff Davis <pgsql(a)j-davis.com>: > On Tue, 2009-11-24 at 14:39 +0100, Pavel Stehule wrote: >> a) good designed C API  like: >> >>   initialise_functions(fcinfo) -- std fcinfo >>   consument_process_tuple(fcinfo) -- gets standard row -- Datum >> dvalues[] + Row description >>   producent_process_tuple(fcinfo) -- returns standard row  -- Datum >> dvalues[] + Row description (look on SRF API) >>   terminate_funnction(fcinfo) >> > > Don't you still need the functions to accept an argument of type > internal? Otherwise, we lose the ability to copy a buffer to the dblink > connection, which was the original motivation. > It depends on design. I don't thing so internal is necessary. It is just wrong design. Pavel > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Daniel Farina on 25 Nov 2009 00:12 On Tue, Nov 24, 2009 at 8:45 PM, Pavel Stehule <pavel.stehule(a)gmail.com> wrote: > It depends on design. I don't thing so internal is necessary. It is > just wrong design. Depends on how lean you want to be when doing large COPY...right now the cost is restricted to having to call a function pointer and a few branches. If you want to take SQL values, then the semantics of function calling over a large number of rows is probably notably more expensive, although I make no argument against the fact that the non-INTERNAL version would give a lot more people more utility. fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Daniel Farina on 25 Nov 2009 00:31 On Tue, Nov 24, 2009 at 9:13 PM, Jeff Davis <pgsql(a)j-davis.com> wrote: > > I still don't see any reason to force it to be record by record though. > If the point is to push data from a table into a remote table, why > should the copied data be translated out of binary format into a record, > and then back into binary form to send to the remote system? > > Currently, the second argument to copy is a source or destination of > bytes, not records. So forcing it to deal with records is inconsistent. You are correct. It so happens as an artifact of how COPY is written that things are delivered row-by-row, but at some fundamental level it does not matter were that not the case... fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Pavel Stehule on 25 Nov 2009 00:35 2009/11/25 Daniel Farina <drfarina(a)gmail.com>: > On Tue, Nov 24, 2009 at 8:45 PM, Pavel Stehule <pavel.stehule(a)gmail.com> wrote: >> It depends on design. I don't thing so internal is necessary. It is >> just wrong design. > > Depends on how lean you want to be when doing large COPY...right now > the cost is restricted to having to call a function pointer and a few > branches. Â If you want to take SQL values, then the semantics of > function calling over a large number of rows is probably notably more > expensive, although I make no argument against the fact that the > non-INTERNAL version would give a lot more people more utility. I believe so using an "internal" minimalize necessary changes in COPY implementation. Using a funcapi needs more work inside COPY - you have to take some functionality from COPY to stream functions. Probably the most slow operations is parsing - calling a input functions. This is called once every where. Second slow operation is reading from network - it is same. So I don't see too much reasons, why non internal implementation have to be significant slower than your actual implementation. I am sure, so it needs more work. What is significant - when I better join COPY and some streaming function, then I don't need use tuplestore - or SRF functions. COPY reads data directly. > > fdr > -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Daniel Farina on 25 Nov 2009 00:42
On Tue, Nov 24, 2009 at 9:35 PM, Pavel Stehule <pavel.stehule(a)gmail.com> wrote: > 2009/11/25 Daniel Farina <drfarina(a)gmail.com>: >> On Tue, Nov 24, 2009 at 8:45 PM, Pavel Stehule <pavel.stehule(a)gmail.com> wrote: >>> It depends on design. I don't thing so internal is necessary. It is >>> just wrong design. >> >> Depends on how lean you want to be when doing large COPY...right now >> the cost is restricted to having to call a function pointer and a few >> branches. If you want to take SQL values, then the semantics of >> function calling over a large number of rows is probably notably more >> expensive, although I make no argument against the fact that the >> non-INTERNAL version would give a lot more people more utility. > > I believe so using an "internal" minimalize necessary changes in COPY > implementation. Using a funcapi needs more work inside COPY - you > have to take some functionality from COPY to stream functions. > Probably the most slow operations is parsing - calling a input > functions. This is called once every where. Second slow operation is > reading from network - it is same. So I don't see too much reasons, > why non internal implementation have to be significant slower than > your actual implementation. I am sure, so it needs more work. You are probably right. We could try coercing to bytea and back out to bytes, although it seems like a superfluous cost to force *everyone* to pay just to get the same bytes to a network buffer. fdr -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |