Parser generator (+ Lexer) that actually works?
in [TCL]
|
From: Andreas Kupries on 9 Jun 2010 20:09 Fredrik Karlsson <dargosch(a)gmail.com> writes: > Hi, > > This looks very promising indeed, but still at bit opaque (at least > for me). Given that I have a grammar in PEG format in the file > grammar.peg, how do I get a TclOO that would be a parser for that > grammar? > > What I have is: > ---- > set peg [read [open grammar.peg r] ] > set gram [pt::peg::import $peg] > --- > > , but how to get the parser? And here I thought that I had an example of that in the docs. Apparently I confused myself with the pt_parser_api manpage, which has an example of the use, when you have the class. Ok. ============================================== package require pt::pgen package require fileutil set script [pt::pgen \ peg [fileutil::cat grammar.peg] oo \ -file grammar.peg \ -name name-of-grammar \ -user $tcl_platform(user) \ -class name-of-oo-class] # Options are specific to the output format. fileutil::writeFile your-tcl-file.tcl $script ============================================== There is also the 'pt' application, a light wrapper around pt::pgen. ============================================== pt generate \ peg grammar.peg oo your-tcl-file.tcl \ -name name-of-grammar \ -class name-of-oo-clas ============================================== HTH. > /Fredrik > > On Jun 8, 7:46 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: >> I recently created a replacement, 'pt', also in tcllib. >> >> Documentation starts here >> >> http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt_introduction.html >> and http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt.html >> >> Hopefully that is better. >> -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: JHJL on 10 Jun 2010 09:43 On Jun 10, 1:09 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > Fredrik Karlsson <dargo...(a)gmail.com> writes: > > Hi, > > > This looks very promising indeed, but still at bit opaque (at least > > for me). Given that I have a grammar in PEG format in the file > > grammar.peg, how do I get a TclOO that would be a parser for that > > grammar? > > > What I have is: > > ---- > > set peg [read [open grammar.peg r] ] > > set gram [pt::peg::import $peg] > > --- > > > , but how to get the parser? > > And here I thought that I had an example of that in the > docs. Apparently I confused myself with the pt_parser_api manpage, > which has an example of the use, when you have the class. > > Ok. > ============================================== > package require pt::pgen > package require fileutil > > set script [pt::pgen \ > peg [fileutil::cat grammar.peg] > oo \ > -file grammar.peg \ > -name name-of-grammar \ > -user $tcl_platform(user) \ > -class name-of-oo-class] > # Options are specific to the output format. > > fileutil::writeFile your-tcl-file.tcl $script > ============================================== > > There is also the 'pt' application, a light wrapper around pt::pgen. > > ============================================== > pt generate \ > peg grammar.peg > oo your-tcl-file.tcl \ > -name name-of-grammar \ > -class name-of-oo-clas > ============================================== > > HTH. > > > /Fredrik > > > On Jun 8, 7:46 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > >> I recently created a replacement, 'pt', also in tcllib. > > >> Documentation starts here > > >> http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt_introduction.html > >> and http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt.html > > >> Hopefully that is better. > > -- > So long, > Andreas Kupries <akupr...(a)shaw.ca> > <http://www.purl.org/NET/akupries/> > Developer @ <http://www.activestate.com/> > ------------------------------------------------------------------------------- Hi Andreas Many thanks for you work on providing the TCLLIB Parser Generator code. Alas, I am also confused by the documentation :( What I could really do with is a fully worked up example showing a PEG, the TclOO (or snit) output and a sample operation against some input data! Asking is easy :) The PT TCLIB docs makes a few references such as "please go and read the PEG Language Tutorial" ; a quick Google didn't reveal anything close... Can you provide a URL or am I missing something in my reading of the docs? kind regards Julian kind regards
From: tom.rmadilo on 10 Jun 2010 13:43 On Jun 10, 6:43 am, JHJL <j...(a)hippospace.com> wrote: > On Jun 10, 1:09 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > > > > > Fredrik Karlsson <dargo...(a)gmail.com> writes: > > > Hi, > > > > This looks very promising indeed, but still at bit opaque (at least > > > for me). Given that I have a grammar in PEG format in the file > > > grammar.peg, how do I get a TclOO that would be a parser for that > > > grammar? > > > > What I have is: > > > ---- > > > set peg [read [open grammar.peg r] ] > > > set gram [pt::peg::import $peg] > > > --- > > > > , but how to get the parser? > > > And here I thought that I had an example of that in the > > docs. Apparently I confused myself with the pt_parser_api manpage, > > which has an example of the use, when you have the class. > > > Ok. > > ============================================== > > package require pt::pgen > > package require fileutil > > > set script [pt::pgen \ > > peg [fileutil::cat grammar.peg] > > oo \ > > -file grammar.peg \ > > -name name-of-grammar \ > > -user $tcl_platform(user) \ > > -class name-of-oo-class] > > # Options are specific to the output format. > > > fileutil::writeFile your-tcl-file.tcl $script > > ============================================== > > > There is also the 'pt' application, a light wrapper around pt::pgen. > > > ============================================== > > pt generate \ > > peg grammar.peg > > oo your-tcl-file.tcl \ > > -name name-of-grammar \ > > -class name-of-oo-clas > > ============================================== > > > HTH. > > > > /Fredrik > > > > On Jun 8, 7:46 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > > >> I recently created a replacement, 'pt', also in tcllib. > > > >> Documentation starts here > > > >> http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt_introduction.html > > >> and http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt.html > > > >> Hopefully that is better. > > > -- > > So long, > > Andreas Kupries <akupr...(a)shaw.ca> > > <http://www.purl.org/NET/akupries/> > > Developer @ <http://www.activestate.com/> > > ------------------------------------------------------------------------------- > > Hi Andreas > > Many thanks for you work on providing the TCLLIB Parser Generator > code. Alas, I am also confused by the documentation :( What I could > really do with is a fully worked up example showing a PEG, the TclOO > (or snit) output and a sample operation against some input data! > Asking is easy :) > > The PT TCLIB docs makes a few references such as "please go and read > the PEG Language Tutorial" ; a quick Google didn't reveal anything > close... Can you provide a URL or am I missing something in my reading > of the docs? This link has about as close to an example as I have found: http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt_astree.html Here is the relevant example: Assuming the parsing expression grammar below PEG calculator (Expression) Digit <- '0'/'1'/'2'/'3'/'4'/'5'/'6'/'7'/'8'/'9' ; Sign <- '-' / '+' ; Number <- Sign? Digit+ ; Expression <- '(' Expression ')' / (Factor (MulOp Factor)*) ; MulOp <- '*' / '/' ; Factor <- Term (AddOp Term)* ; AddOp <- '+'/'-' ; Term <- Number ; END; and the input string 120+5 then a parser should deliver the abstract syntax tree below (except for whitespace) set ast {Expression 0 4 {Factor 0 4 {Term 0 2 {Number 0 2 {Digit 0 0} {Digit 1 1} {Digit 2 2} } } {AddOp 3 3} {Term 4 4 {Number 4 4 {Digit 4 4} } } } } Now: there are no examples of what you have to do with the AST produced for the expression 120+5. In other words, these tools (the examples provided) don't seem to allow you to associate terminals to actual "compiled code". Also note that the above example doesn't include what commands need to be executed to produce the result. Is anyone using this API, it seems interesting, but lacks working examples. Somewhere I was expecting an example where I type in 120+5 and get 125 as a result. This is a very extensive API, how was it tested, something more than the partial calculator example must exist.
From: Andreas Kupries on 10 Jun 2010 21:14 JHJL <jhjl(a)hippospace.com> writes: > Hi Andreas > > Many thanks for you work on providing the TCLLIB Parser Generator > code. Alas, I am also confused by the documentation :( What I could > really do with is a fully worked up example showing a PEG, the TclOO > (or snit) output and a sample operation against some input data! > Asking is easy :) > > The PT TCLIB docs makes a few references such as "please go and read > the PEG Language Tutorial" ; a quick Google didn't reveal anything > close... Can you provide a URL or am I missing something in my reading > of the docs? The cross references between the various documents seem to be broken. That is something I have to investigate. It is either trouble in the documents, or in the code generating this for the AS website. To see all the pages for PT start at http://docs.activestate.com/activetcl/8.5/tcllib/toc.html, the scroll down to pt::peg::import::peg The section of PT related pages starts at that point. While many of these are about internals the description at the top of each page should tell you if that is the case. -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: Andreas Kupries on 10 Jun 2010 21:21
"tom.rmadilo" <tom.rmadilo(a)gmail.com> writes: > This link has about as close to an example as I have found: > > http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt_astree.html > > Here is the relevant example: > > Now: there are no examples of what you have to do with the AST > produced for the expression 120+5. In other words, these tools (the > examples provided) don't seem to allow you to associate terminals to > actual "compiled code". If you meant 'actions' to be executed for a (non)terminal, no, this is not present. You are getting an Abstract Syntax Tree (AST) describing the structure of the input as per grammar used to parse it, and then you have to operate on the tree, i.e. transform it, walk, etc. to get the end result, some type of compilation, or sequence of actions, or .... > Also note that the above example doesn't > include what commands need to be executed to produce the result. > Is anyone using this API, it seems interesting, but lacks working > examples. Somewhere I was expecting an example where I type in 120+5 > and get 125 as a result. This is a very extensive API, how was it > tested, something more than the partial calculator example must exist. Tcllib is in CVS at SourceForge, look for http://sf.net/projects/tcllib Then in, in the folder 'Tcllib', module 'pt', the files *.test are the main testsuite drivers, tests/*.test the testsuites, and under tests/data/ok/... are the data files used by the tests. -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> ------------------------------------------------------------------------------- |