Unicode UTF-8 table formatting for psql text output
in [PgSql]
|
Prev: Hot Standby 0.2.1
Next: Rejecting weak passwords
From: Roger Leigh on 4 Oct 2009 13:44 On Fri, Oct 02, 2009 at 05:34:16PM -0700, Brad T. Sliger wrote: > On Friday 02 October 2009 04:21:35 Roger Leigh wrote: > > I have attached a patch which implements the feature as a pset > > variable. This also slightly simplifies some of the patch since > > the table style is passed to functions directly in printTableContent > > rather than separately. The psql option '-P tablestyle=ascii' is > > passed to psql in pg_regress_main.c which means the testsuite doesn't > > fail any more. The option is documented in the psql docs, and is > > also tab-completed. Users can just put '\pset tablestyle ascii' in > > their .psqlrc if they want the old format in a UTF-8 locale. > > I looked at psql-utf8-table-5.patch. Many thanks for taking the time to do this. I've attached a followup patch which addresses your point below: > Lint(1) says there is an extra trailing ',' in src/bin/psql/print.h. in 'typedef enum printTextRule'. The addition to > src/bin/psql/command.c could use a comment, like adjacent code. Fixed. > 'ASCII' and 'UTF8' may need <acronym></acronym> tags in doc/src/sgml/ref/psql-ref.sgml, like adjacent > code. I'm not sure someone who hasn't seen this patch in action would immediately know what it does from the > documentation. `gmake html` works without the patch, but fails with the patch: Also fixed. I also added some additional explanation of the option which hopefully makes its purpose more obvious. The <acronym> tag isn't used for the itemised option list names, but is used in the descriptive text; I can also add it there if appropriate. It's likely that "tablestyle" could well be named better. "format" is already used, but if there's a more intuitive name that fits better, I'm happy to change it. > openjade:ref/psql-ref.sgml:1692:15:E: document type does not allow element "TERM" here; assuming > missing "VARLISTENTRY" start-tag Also fixed. > After the patch, `\pset format wrapped` produces '\pset: unknown option: format'. I saw this in interactive psql > and from .psqlrc. I think this can be fixed by changing the addition to src/bin/psql/command.c from an 'if' clause to > an 'else if' clause. Oops, yes. Sorry about that hiccup. I've also fixed this. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
From: Peter Eisentraut on 4 Oct 2009 16:22 I have a comment on this bit: > @@ -125,6 +128,17 @@ main(int argc, char *argv[]) > > /* We rely on unmentioned fields of pset.popt to start out > 0/false/NULL */ > pset.popt.topt.format = PRINT_ALIGNED; > + > + /* Default table style to plain ASCII */ > + pset.popt.topt.table_style = &asciiformat; > +#if (defined(HAVE_LANGINFO_H) && defined(CODESET)) > + /* If a UTF-8 locale is available, switch to UTF-8 box drawing > characters */ > + if (pg_strcasecmp(nl_langinfo(CODESET), "UTF-8") == 0 || > + pg_strcasecmp(nl_langinfo(CODESET), "utf8") == 0 || > + pg_strcasecmp(nl_langinfo(CODESET), "CP65001") == 0) > + pset.popt.topt.table_style = &utf8format; > +#endif > + > pset.popt.topt.border = 1; > pset.popt.topt.pager = 1; > pset.popt.topt.start_table = true; Elsewhere in the psql code, notably in mbprint.c, we make the decision on whether to apply certain Unicode-aware processing based on whether the client encoding is UTF8. The same should be done here. There is a patch somewhere in the pipeline that would automatically set the psql client encoding to whatever the locale says, but until that is done, the client encoding should be the sole setting that rules what kind of character set processing is done on the client side. -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Roger Leigh on 5 Oct 2009 15:39 On Sun, Oct 04, 2009 at 11:22:27PM +0300, Peter Eisentraut wrote: > I have a comment on this bit: > > > @@ -125,6 +128,17 @@ main(int argc, char *argv[]) > > > > /* We rely on unmentioned fields of pset.popt to start out > > 0/false/NULL */ > > pset.popt.topt.format = PRINT_ALIGNED; > > + > > + /* Default table style to plain ASCII */ > > + pset.popt.topt.table_style = &asciiformat; > > +#if (defined(HAVE_LANGINFO_H) && defined(CODESET)) > > + /* If a UTF-8 locale is available, switch to UTF-8 box drawing > > characters */ > > + if (pg_strcasecmp(nl_langinfo(CODESET), "UTF-8") == 0 || > > + pg_strcasecmp(nl_langinfo(CODESET), "utf8") == 0 || > > + pg_strcasecmp(nl_langinfo(CODESET), "CP65001") == 0) > > + pset.popt.topt.table_style = &utf8format; > > +#endif > > + > > pset.popt.topt.border = 1; > > pset.popt.topt.pager = 1; > > pset.popt.topt.start_table = true; > > Elsewhere in the psql code, notably in mbprint.c, we make the decision > on whether to apply certain Unicode-aware processing based on whether > the client encoding is UTF8. The same should be done here. > > There is a patch somewhere in the pipeline that would automatically set > the psql client encoding to whatever the locale says, but until that is > done, the client encoding should be the sole setting that rules what > kind of character set processing is done on the client side. OK, that makes sense to a certain extent. However, the characters used to draw the table lines are not really that related to the client encoding for data sent from the database (IMHO). I think that (as you said) making the client encoding the same as the locale character set the same in the future would clear up this discrepancy though. Using the client encoding, there's no guarantee the client locale/terminal can handle UTF-8 when the client encoding is UTF-8. I have attached an updated patch which implements your suggested behaviour. It also renames the option to "linestyle" rather than "tablestyle" which I think represents its purpose a bit more clearly. Thanks, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
From: Tom Lane on 5 Oct 2009 16:32 Roger Leigh <rleigh(a)codelibre.net> writes: > On Sun, Oct 04, 2009 at 11:22:27PM +0300, Peter Eisentraut wrote: >> Elsewhere in the psql code, notably in mbprint.c, we make the decision >> on whether to apply certain Unicode-aware processing based on whether >> the client encoding is UTF8. The same should be done here. >> >> There is a patch somewhere in the pipeline that would automatically set >> the psql client encoding to whatever the locale says, but until that is >> done, the client encoding should be the sole setting that rules what >> kind of character set processing is done on the client side. > OK, that makes sense to a certain extent. However, the characters > used to draw the table lines are not really that related to the > client encoding for data sent from the database (IMHO). Huh? The data *in* the table is going to be in the client_encoding, and psql contains no mechanisms that would translate it to something else. Surrounding it with decoration in a different encoding is just a recipe for breakage. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers(a)postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
From: Roger Leigh on 6 Oct 2009 05:44
On Mon, Oct 05, 2009 at 04:32:08PM -0400, Tom Lane wrote: > Roger Leigh <rleigh(a)codelibre.net> writes: > > On Sun, Oct 04, 2009 at 11:22:27PM +0300, Peter Eisentraut wrote: > >> Elsewhere in the psql code, notably in mbprint.c, we make the decision > >> on whether to apply certain Unicode-aware processing based on whether > >> the client encoding is UTF8. The same should be done here. > >> > >> There is a patch somewhere in the pipeline that would automatically set > >> the psql client encoding to whatever the locale says, but until that is > >> done, the client encoding should be the sole setting that rules what > >> kind of character set processing is done on the client side. > > > OK, that makes sense to a certain extent. However, the characters > > used to draw the table lines are not really that related to the > > client encoding for data sent from the database (IMHO). > > Huh? The data *in* the table is going to be in the client_encoding, and > psql contains no mechanisms that would translate it to something else. > Surrounding it with decoration in a different encoding is just a recipe > for breakage. Ah, I was under the mistaken assumption that this was iconv()ed or otherwise translated for correct display. In that case, I'll leave the patch as is (using the client encoding for table lines). I've attached an updated copy of the patch (it just removes the now unneeded langinfo.h header). Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `- GPG Public Key: 0x25BFB848 Please GPG sign your mail. |