including PID or backend ID in relpath of temp rels
in [PgSql]
|
Prev: [HACKERS] including PID or backend ID in relpath of temp rels
Next: [HACKERS] improve plpgsql's EXECUTE 'select into' message with a hint
From: Alvaro Herrera on 4 May 2010 15:03 Robert Haas escribi�: > On Tue, May 4, 2010 at 2:06 PM, Alvaro Herrera > <alvherre(a)commandprompt.com> wrote: > > Robert Haas escribi�: > > Hey, thanks for writing back! I just spent the last few hours > thinking about this and beating my head against the wall. :-) > >> [smgr.c,inval.c] Do we need to call CacheInvalidSmgr for temporary > >> relations? �I think the only backend that can have an smgr reference > >> to a temprel other than the owning backend is bgwriter, and AFAICS > >> bgwriter will only have such a reference if it's responding to a > >> request by the owning backend to unlink the associated files, in which > >> case (I think) the owning backend will have no reference. > > > > Hmm, wasn't there a proposal to have the owning backend delete the files > > instead of asking the bgwriter to? > > I did propose that upthread; it may have been proposed previously > also. This might be worth doing independently of the rest of the patch > (which I'm starting to fear is doomed, cue ominous soundtrack) since > it would reduce the chance of orphaning data files and possibly > simplify the logic also. +1 for doing it separately, but hopefully that doesn't mean the rest of this patch is doomed ... > >> [dbsize.c] As with relcache.c, there's a problem if we're asked for > >> the size of a temporary relation that is not our own: we can't call > >> relpath() without knowing the ID of the owning backend, and there's no > >> way to acquire that information for pg_class. �I guess we could just > >> refuse to answer the question in that case, but that doesn't seem real > >> cool. �Or we could physically scan the directory for files that match > >> a suitably constructed wildcard, I suppose. > > > > I don't very much like the wildcard idea; but I don't think it's > > unreasonable to refuse to provide a file size. �If the owning backend > > has still got part of the table in local buffers, you'll get a > > misleading answer, so perhaps it's best to not give an answer at all. > > > > Maybe this problem could be solved if we could somehow force that > > backend to write down its local buffers, in which case it'd be nice to > > have a solution to the dbsize problem. > > I'm sure we could add some kind of signaling mechanism that would tell > all backends to flush their local buffers, but I'm not too sure it > would help this case very much, because you likely wouldn't want to > wait for all the backends to complete that process before reporting > results. Hmm, I was thinking in the pg_relation_size function -- given this new mechanism you could get an accurate size of temp tables for other backends. I wasn't thinking in the pg_database_size function, and perhaps it's better to *not* include temp tables in that report at all. > >> [syncscan.c] It seems we pursue this optimization even for temprels; I > >> can't think of why that would be useful in practice. �If it's useless > >> overhead, should we skip it? �This is really independent of this > >> project; just a side thought. > > > > Maybe recently used buffers are more likely to be in the OS page cache, > > so perhaps it's not good to disable it. > > I don't get it. If the whole relation fits in the page cache, it > doesn't much matter where you start a seqscan. If it doesn't, > starting where the last one ended is anti-optimal. Err, I was thinking that a syncscan started a bunch of pages earlier than the point where the previous scan ended, but yeah, that's a bit silly. Maybe we should just ignore syncscan in temp tables altogether, as you propose. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 4 May 2010 15:46 On Tue, May 4, 2010 at 3:03 PM, Alvaro Herrera <alvherre(a)commandprompt.com> wrote: >> > Hmm, wasn't there a proposal to have the owning backend delete the files >> > instead of asking the bgwriter to? >> >> I did propose that upthread; it may have been proposed previously >> also. This might be worth doing independently of the rest of the patch >> (which I'm starting to fear is doomed, cue ominous soundtrack) since >> it would reduce the chance of orphaning data files and possibly >> simplify the logic also. > > +1 for doing it separately, but hopefully that doesn't mean the rest of > this patch is doomed ... I wonder if it would be possible to reject access to temporary relations at a higher level. Right now, if you create a temporary relation in one session, you can issue a SELECT statement against it in another relation, and get back 0 rows. If you then insert data into it and select against it again, you'll get an error saying that you can't access temporary tables of other sessions. If you try to truncate somebody else's temporary relation, it fails; but if you try to drop it, it works. In fact, you can even run ALTER TABLE ... ADD COLUMN on somebody else's temp table, as long as you don't do anything that requires a rewrite. CLUSTER fails; VACUUM and VACUUM FULL both appear to work but apparently actually don't do anything under the hood, so that database-wide vacuums don't barf. The whole thing seems pretty leaky. It would be nice if we could find a small set of control points where we basically reject ALL access to somebody else's temp relations, period. One possible thing we might do (bearing in mind that we might need to wall off access at multiple levels) would be to forbid creating a relcache entry for a non-local temprel. That would, in turn, forbid doing pretty much anything to such a relation, although I'm not sure what else would get broken in the process. But it would eliminate, for example, all the checks for RELATION_IS_OTHER_TEMP, since that Just Couldn't Happen. It would would eliminate the need to install specific handling for this case in dbsize.c - we'd just automatically croak. And it's also probably necessary to do this anyhow if we want to ever eliminate those CacheInvalidSmgr() calls for temp rels, because if I can drop your temprel, that implies I can smgropen() it. ....Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 4 May 2010 17:12 Robert Haas <robertmhaas(a)gmail.com> writes: > One possible thing we might do (bearing in mind that we might need to > wall off access at multiple levels) would be to forbid creating a > relcache entry for a non-local temprel. That would, in turn, forbid > doing pretty much anything to such a relation, although I'm not sure > what else would get broken in the process. Dropping temprels left behind by a crashed backend would get broken by that; which is a deal-breaker, because we have to be able to clean those up. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 4 May 2010 17:17 Alvaro Herrera <alvherre(a)commandprompt.com> writes: > I don't very much like the wildcard idea; but I don't think it's > unreasonable to refuse to provide a file size. If the owning backend > has still got part of the table in local buffers, you'll get a > misleading answer, so perhaps it's best to not give an answer at all. FWIW, that's not the case, anymore than it is for blocks in shared buffer cache for regular rels. smgrextend() results in an observable extension of the file EOF immediately, whether or not you can see up-to-date data for those pages. Now people have often complained about the extra I/O involved in that, and it'd be nice to have a solution, but it's not clear to me that fixing it would be harder for temprels than regular rels. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Robert Haas on 4 May 2010 18:34
On Tue, May 4, 2010 at 5:12 PM, Tom Lane <tgl(a)sss.pgh.pa.us> wrote: > Robert Haas <robertmhaas(a)gmail.com> writes: >> One possible thing we might do (bearing in mind that we might need to >> wall off access at multiple levels) would be to forbid creating a >> relcache entry for a non-local temprel. That would, in turn, forbid >> doing pretty much anything to such a relation, although I'm not sure >> what else would get broken in the process. > > Dropping temprels left behind by a crashed backend would get broken by > that; which is a deal-breaker, because we have to be able to clean those > up. Phooey. It was such a good idea in my head. ....Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |