A thought on Index Organized Tables
in [PgSql]
|
Prev: [COMMITTERS] pgsql: Oops, don't forget to rewind the directory before scanning it to
Next: Time travel on the buildfarm
From: Karl Schnaitter on 25 Feb 2010 19:11 On Thu, Feb 25, 2010 at 3:59 PM, Gokulakannan Somasundaram < gokul007(a)gmail.com> wrote: > I disagree with that, Gokul -- if the ordering operators are volatile or >> just incorrect, during DELETE, you could set xmax in the wrong IndexTuple. >> Then there will be another IndexTuple that says it's visible, but it points >> to a non-visible heap tuple. I think you should follow the pointers to the >> heap before you decide to let an index tuple remain in the index during >> vacuum. This would ensure that all references from an index to a heap tuple >> are removed before vacuuming the heap tuple. I would be worried about what >> might break if this invariant doesn't hold. >> > > Well, Karl, if we have to support function based indexes/IOT, one thing is > for sure. We can't support them for volatile functions / broken data types. > Everyone agrees with that. But the question is how we identify something is > not a volatile function. Only way currently is to let the user make the > decision( Or we should consult some mathematician ). So we need not consult > the heaptuple. > > First of all, volatility is not the only issue. The ordering ops could also be incorrect, e.g., violate the transitivity property. there is no reliable way to determine if a function is volatile and/or incorrectly specified. Of course, PG can't "support" indexing with incorrect functions. However, it's worthwhile to guard against too much damage being done if the user's function has a bug. Maybe I'm wrong? Maybe an index tuple with a dangling pointer is actually harmless? Karl
From: Tom Lane on 25 Feb 2010 19:14 Karl Schnaitter <karlsch(a)gmail.com> writes: > If it's of any interest, I can say something about the hint bits in the > index tuple header. In my implementation, my decision was to use only one > hint bit. It went into the unused 13th bit of the IndexTuple header. When > the hint bit is set, it means that > (xmin is committed OR xmin = InvalidTransactionId) > AND (xmax is committed OR xmax = InvalidTransactionId) > Then there are 12 bytes for xmin/xmax/cid. I did sweat something over this > decision... but maybe it was a wasted effort if the 12 bytes end up > occupying 16 bytes anyway. Actually, if you need to squeeze a few more bits into that word, the thing to do would be to get rid of storing the tuple length there. This would involve adding the same type of indirection header that we use for HeapTuples, so that the length would be available at need without going back to the item pointer. It'd be an invasive code change but reasonably straightforward, and then you'd have room for normal hint bits. Squeezing cmin in there is just fantasy though. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 25 Feb 2010 19:21 Karl Schnaitter <karlsch(a)gmail.com> writes: > Of course, PG can't "support" indexing with incorrect functions. However, > it's worthwhile to guard against too much damage being done if the user's > function has a bug. Maybe I'm wrong? Maybe an index tuple with a dangling > pointer is actually harmless? No, it's far from harmless. As soon as that heap TID gets filled with an unrelated tuple, you run the risk of indexscans alighting on and perhaps modifying the wrong tuple. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Gokulakannan Somasundaram on 25 Feb 2010 23:15 Tom, Actually, if you need to squeeze a few more bits into that word, the > thing to do would be to get rid of storing the tuple length there. > This would involve adding the same type of indirection header that > we use for HeapTuples, so that the length would be available at need > without going back to the item pointer. I I feel the other one is easy. To store the hint bits inside the ItemId, in the place of size. We have 16 bits there.Whenever the size is required, we need to follow the offset and goto the corresponding tuple and then take the size from there. The change seems to be minimal, but please bear with me, if i am very ignorant about something. > Squeezing cmin in there is just fantasy though. > I think we can get away with this, by making the person, who inserts and selects in the same transaction to go and find the visibility through heap. In the Index tuple hint bits, we can note down, if the command is a simple insert/update/delete. By Simple insert, i mean that it doesn't have a select. So if that is the case, it can be made visible to statements within the same transaction. We can even document, that people can just insert a savepoint between their insert and select. This would increase the xid and make that tuple visible within the same transaction. All that seems to be possible. Thanks, Gokul.
From: Gokulakannan Somasundaram on 25 Feb 2010 23:20
> >> > First of all, volatility is not the only issue. The ordering ops could also > be incorrect, e.g., violate the transitivity property. there is no reliable > way to determine if a function is volatile and/or incorrectly specified. > No it is the only issue. If you create a datatype with volatile function for ordering ops, then you have the broken data type(the one you are referring to). So they are one and the same. Thanks, Gokul. |