WIP: preloading of ispell dictionary
in [PgSql]
|
Prev: pgsql: Prevent the injection ofinvalidly encoded strings by PL/Python
Next: Standalone backends run StartupXLOG in anincorrect environment
From: Bruce Momjian on 24 Mar 2010 10:41 Pavel Stehule wrote: > 2010/3/24 Bruce Momjian <bruce(a)momjian.us>: > > Pavel Stehule wrote: > >> 2010/3/24 Craig Ringer <craig(a)postnewspapers.com.au>: > >> > Pavel Stehule wrote: > >> > > >> >> Personally I dislike idea some dictionary precompiler - it is next > >> >> application for maintaining and maybe not necessary. > >> > > >> > That's the sort of thing that can be done when first required by any > >> > backend and the results saved in a file for other backends to mmap(). > >> > It'd probably want to be opened r/w access-exclusive initially, then > >> > re-opened read-only access-shared when ready for use. > >> > > >> > My only concern would be that the cache would want to be forcibly > >> > cleared at postmaster start, so that "restart the postmaster" fixes any > >> > messsed-up-cache issues that might arise (not that they should) without > >> > people having to go rm'ing in the datadir. Even if Pg never has any bugs > >> > that result in bad cache files, the file system / bad memory / cosmic > >> > rays / etc can still mangle a cache file. > >> > > >> > BTW, mmap() isn't an issue on Windows: > >> > ?http://msdn.microsoft.com/en-us/library/aa366556%28VS.85%29.aspx > >> > It's spelled CreateFileMapping, but otherwise is fairly similar, and is > >> > perfect for this sort of use. > >> > > >> > A shared read-only mapping of processed-and-cached tsearch2 dictionaries > >> > would save a HUGE amount of memory if many backends were using tsearch2 > >> > at the same time. I'd make a big difference here. > >> > > >> > >> If you know this area well, please, enhance my first patch. I am not > >> able to oppose to Tom, who has a clean opinion on this patch :( > > > > Should we add a TODO? > > why not ? OK, what would the TODO text be? -- Bruce Momjian <bruce(a)momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com PG East: http://www.enterprisedb.com/community/nav-pg-east-2010.do -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Tom Lane on 24 Mar 2010 10:46
Bruce Momjian <bruce(a)momjian.us> writes: > OK, what would the TODO text be? I think there are really two tasks here: * preprocess the textual dictionary definition files into something that can be slurped directly into memory; * use mmap() instead of read() to read preprocessed files into memory, on machines where such a syscall is available. There would be considerable gain from task #1 even without mmap. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers |