Prev: EOL is when?
Next: EOL for 7.4 and 8.0
From: Tom Lane on 24 Jun 2010 10:20
Carsten Kropf <ckropf2(a)fh-hof.de> writes:
> I have a strange issue using a custom built index structure. My index access method support document type composed of words (as tsvector) and points (1-dimensional arrays of them). For internal reasons, I have to save the documents as a whole inside my structure (for proper reorganisations).
> So, I form the tuples using index_form_tuple with the proper description. Everything works fine, as long as the documents are quite small. However, if the tsvector becomes too large, I run into a problem of not being able to store the documents, because (obviously) the tsvector is too large for one page.

Well, of course. I think this is a fundamentally bad index design. You
didn't say exactly what sort of searches you want this index type to
accelerate, but perhaps you need a design closer to GIN, in which you'd
make index entries for individual words not whole documents.

> What I tried to solve this issue here, is to extract the words from the document (in my index) and calling 'Datum toast_compress_datum(Datum value)'in order to compress the tsvector into a proper toast table.

Indexes don't have toast tables.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Carsten Kropf on 24 Jun 2010 10:53
Oh,
thanks. I didn't know about index tables not having access to associated toast values. The index access method is (at least in the textual part) similar to GIN, however, I needed to do some changes to it. Saving the whole document is actually only important for vacuum. I think, I will find some workaround to solve this issue.
However, it is a little bit strange, that I get toasted values (when inserting from another table) and untoasted values, if I insert items directly. Could anybody please explain this to me?

Best regards
Carsten Kropf
Am 24.06.2010 um 16:20 schrieb Tom Lane:

> Carsten Kropf <ckropf2(a)fh-hof.de> writes:
>> I have a strange issue using a custom built index structure. My index access method support document type composed of words (as tsvector) and points (1-dimensional arrays of them). For internal reasons, I have to save the documents as a whole inside my structure (for proper reorganisations).
>> So, I form the tuples using index_form_tuple with the proper description. Everything works fine, as long as the documents are quite small. However, if the tsvector becomes too large, I run into a problem of not being able to store the documents, because (obviously) the tsvector is too large for one page.
>
> Well, of course. I think this is a fundamentally bad index design. You
> didn't say exactly what sort of searches you want this index type to
> accelerate, but perhaps you need a design closer to GIN, in which you'd
> make index entries for individual words not whole documents.
>
>> What I tried to solve this issue here, is to extract the words from the document (in my index) and calling 'Datum toast_compress_datum(Datum value)'in order to compress the tsvector into a proper toast table.
>
> Indexes don't have toast tables.
>
> regards, tom lane
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers


--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Tom Lane on 24 Jun 2010 11:08
Carsten Kropf <ckropf2(a)fh-hof.de> writes:
> However, it is a little bit strange, that I get toasted values (when inserting from another table) and untoasted values, if I insert items directly. Could anybody please explain this to me?

Huh? An index will never ever get passed an externally-toasted value.
See the TOAST_INDEX_HACK code in indextuple.c.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

From: Carsten Kropf on 24 Jun 2010 11:36
Hmm,
k, then I don't know exactly what happens. I have some values in my composite type. The size of the index tuple (if coming from one table and being copied to the target table) is about 15x bytes. Otherwise, if I insert the same tuple directly, it is about 12kb. So, I don't know exactly what happens here, unfortunately.
Best regards
Carsten Kropf
Am 24.06.2010 um 17:08 schrieb Tom Lane:

> Carsten Kropf <ckropf2(a)fh-hof.de> writes:
>> However, it is a little bit strange, that I get toasted values (when inserting from another table) and untoasted values, if I insert items directly. Could anybody please explain this to me?
>
> Huh? An index will never ever get passed an externally-toasted value.
> See the TOAST_INDEX_HACK code in indextuple.c.
>
> regards, tom lane
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers


--
Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

 | 
Pages: 1
Prev: EOL is when?
Next: EOL for 7.4 and 8.0