testing HS/SR - 1 vs 2 performance
in [PgSql]
|
Prev: pg_ctl stop -m immediate on the primary server inflatessequences
Next: [HACKERS] non-reproducible failure of random test on HEAD
From: Simon Riggs on 17 Apr 2010 16:14 On Sat, 2010-04-17 at 15:45 -0400, Tom Lane wrote: > Simon Riggs <simon(a)2ndQuadrant.com> writes: > > On Sat, 2010-04-17 at 11:13 -0400, Tom Lane wrote: > >> It'd be cheaper anyway to sort and search the > >> array using plain <, so why are you so eager to use > >> TransactionIdFollows? > > > The array grows to the right and is laid out one xid per element, > > resulting in a sequence of values that are transactionid-monotonic. > > How do you know that just adding items at the right will produce a > sorted array? It seems quite unlikely to me that transactions can be > guaranteed to arrive at this code in XID order. I think you need to do > an explicitly sorted insertion. Xids don't arrive in sequence, but "known assigned xids" are added in sequence because we infer the existence of the intermediate xids and assuming they are running for the snapshot. > > ... Doing it this way means that we can > > add rows past the head of the array and then move the head atomically, > > so that we can make adding xids lock-free. > > ... and even without that issue, this seems like utter fantasy. How > are you going to do that "atomically"? Have you considered what will > happen on weak-memory-ordering machines like PPC, in particular? We search the array between tail and head. If the head moves by integer overwrite just as already happens for xid assignment, then we would use the new head for the search. The code is careful to fetch only once. I would freely admit I know absolutely nothing about details of weak-memory-ordering machines and have not considered them at all. How would what I have proposed fail to work, yet what we already rely on work correctly? Do the circumstances differ? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 17 Apr 2010 16:48 Simon Riggs <simon(a)2ndQuadrant.com> writes: > On Sat, 2010-04-17 at 15:45 -0400, Tom Lane wrote: >> How do you know that just adding items at the right will produce a >> sorted array? > Xids don't arrive in sequence, but "known assigned xids" are added in > sequence because we infer the existence of the intermediate xids and > assuming they are running for the snapshot. Hm. Okay, maybe that will work. >> ... and even without that issue, this seems like utter fantasy. How >> are you going to do that "atomically"? Have you considered what will >> happen on weak-memory-ordering machines like PPC, in particular? > We search the array between tail and head. If the head moves by integer > overwrite just as already happens for xid assignment, then we would use > the new head for the search. The code is careful to fetch only once. .... but this will not. You need to use a lock, because there is otherwise no guarantee that other processors see the write into the array element before they see the change in the head pointer. > I would freely admit I know absolutely nothing about details of > weak-memory-ordering machines and have not considered them at all. How > would what I have proposed fail to work, yet what we already rely on > work correctly? Do the circumstances differ? Yes. We have memory ordering instructions inserted in the lock acquisition/release code. Trying to access and modify a shared-memory data structure without any locking will not work. There are some places where we suppose that a *single* write into shared memory can safely be done without a lock, if we're not too concerned about how soon other transactions will see the effects. But what you are proposing here requires more than one related write. I've been burnt by this myself: http://archives.postgresql.org/pgsql-committers/2008-06/msg00228.php regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 17 Apr 2010 18:46 On Sat, 2010-04-17 at 16:48 -0400, Tom Lane wrote: > > We search the array between tail and head. If the head moves by integer > > overwrite just as already happens for xid assignment, then we would use > > the new head for the search. The code is careful to fetch only once. > > ... but this will not. You need to use a lock, because there is > otherwise no guarantee that other processors see the write into the > array element before they see the change in the head pointer. > > > I would freely admit I know absolutely nothing about details of > > weak-memory-ordering machines and have not considered them at all. How > > would what I have proposed fail to work, yet what we already rely on > > work correctly? Do the circumstances differ? > > Yes. We have memory ordering instructions inserted in the lock > acquisition/release code. Trying to access and modify a shared-memory > data structure without any locking will not work. > > There are some places where we suppose that a *single* write into shared > memory can safely be done without a lock, if we're not too concerned > about how soon other transactions will see the effects. But what you > are proposing here requires more than one related write. > > I've been burnt by this myself: > http://archives.postgresql.org/pgsql-committers/2008-06/msg00228.php W O W - thank you for sharing. What I'm not clear on is why you've used a spinlock everywhere when only weak-memory thang CPUs are a problem. Why not have a weak-memory-protect macro that does does nada when the hardware already protects us? (i.e. a spinlock only for the hardware that needs it). -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 17 Apr 2010 18:52 Simon Riggs <simon(a)2ndQuadrant.com> writes: > What I'm not clear on is why you've used a spinlock everywhere when only > weak-memory thang CPUs are a problem. Why not have a weak-memory-protect > macro that does does nada when the hardware already protects us? (i.e. a > spinlock only for the hardware that needs it). Well, we could certainly consider that, if we had enough places where there was a demonstrable benefit from it. I couldn't measure any real slowdown from adding a spinlock in that sinval code, so I didn't propose doing so at the time --- and I'm pretty dubious that this code is sufficiently performance-critical to justify the work, either. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Simon Riggs on 18 Apr 2010 03:24
On Sat, 2010-04-17 at 18:52 -0400, Tom Lane wrote: > Simon Riggs <simon(a)2ndQuadrant.com> writes: > > What I'm not clear on is why you've used a spinlock everywhere when only > > weak-memory thang CPUs are a problem. Why not have a weak-memory-protect > > macro that does does nada when the hardware already protects us? (i.e. a > > spinlock only for the hardware that needs it). > > Well, we could certainly consider that, if we had enough places where > there was a demonstrable benefit from it. I couldn't measure any real > slowdown from adding a spinlock in that sinval code, so I didn't propose > doing so at the time --- and I'm pretty dubious that this code is > sufficiently performance-critical to justify the work, either. OK, I'll put a spinlock around access to the head of the array. Thanks for your input. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |