Parser generator (+ Lexer) that actually works?
in [TCL]
|
From: Andreas Kupries on 17 Jun 2010 02:23 "tom.rmadilo" <tom.rmadilo(a)gmail.com> writes: > Okay, here is the example script I used (grammar.peg is the calculator > example): > > # calculator-snit.tcl > package require pt::pgen > package require fileutil > set script [pt::pgen peg [fileutil::cat grammar.peg] snit -class > calc] > > puts "$script" > > tom(a)boron:~/activetcl/test$ ../bin/tclsh calculator-snit.tcl > > calc.snit.tcl > tom(a)boron:~/activetcl/test$ ../bin/tclsh > % source calc.snit.tcl > % calc constructor > ::constructor > % ::constructor parset "120+5" > Expression 0 4 {Factor 0 4 {Term 0 2 {Number 0 2 {Digit 0 0} {Digit 1 > 1} {Digit 2 2}}} {AddOp 3 3} {Term 4 4 {Number 4 4 {Digit 4 4}}}} > > To get info about the ast, look at the object commands in pt::rde: > > http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt_rdengine.html > > I edited the ::calc constructor method: > > constructor {} { > # Create the runtime supporting the parsing process. > set myparser [pt::rde ${selfns}::ENGINE] > puts stderr "myparser = '$myparser'" > return > } > > which gave me the parser object name (::calc::Snit_inst1::ENGINE) > Then you can do: > > % ::calc::Snit_inst1::ENGINE ast (or other pt::rde method). A better solution, IMVHO, would be to add delegate method * to myparser to the generated parser. Then you can call the RDE methods on the parser instances, i.e. ::constructor ast ... -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: Andreas Kupries on 17 Jun 2010 02:26 Fredrik Karlsson <dargosch(a)gmail.com> writes: > Hi, > > I've played with it a bit now, and I've gotten a snit parser which > parses according to the grammar I gave it. I gave up on TclOO as the > parser produced seems to depend on two packages I cannot get(?). It seems that I have not completely done the OO implementation yet. My apologies. > However, the legacy language I have to parse has one part of the > syntax which I think causes left recursion. > "Module=a" is a valid expression, as is > "Module=a & Module=b..." > which should be interpreted as "(Module=a) & (Module=b)..." in the > parsing. For some reason, I get "Module=a" parsed in both examples, > but the rest of the expression in the second expression is left out > and not parsed at all. > > Is there a solution, you think? Choice (/) is prioritized. Order the branches so that it tries to parse the larger/longer possibility first. -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: Andreas Kupries on 17 Jun 2010 02:34 "tom.rmadilo" <tom.rmadilo(a)gmail.com> writes: > > How to test this? > > What I am doing is to put the document into a variable: > > % set data "....." > > Then from a tclsh prompt parse the document. You get an error with a > number indicating the index of where the error occurred. Then you can > do: > > % string range $data (some index prior to the error) (some index after > the error) When an error is thrown the error should also contain a list of the symbols (nonterminals and/or terminals) it expected at that point. If that is not working I will have to debug. Can you tell me more ? > This could help you identify the exact character which is throwing off > your parser. Stack traces don't offer me much help since they come > from an auto-generated parser, but -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: tom.rmadilo on 17 Jun 2010 11:48 On Jun 16, 11:34 pm, Andreas Kupries <akupr...(a)shaw.ca> wrote: > "tom.rmadilo" <tom.rmad...(a)gmail.com> writes: > > > How to test this? > > > What I am doing is to put the document into a variable: > > > % set data "....." > > > Then from a tclsh prompt parse the document. You get an error with a > > number indicating the index of where the error occurred. Then you can > > do: > > > % string range $data (some index prior to the error) (some index after > > the error) > > When an error is thrown the error should also contain a list of the > symbols (nonterminals and/or terminals) it expected at that point. > > If that is not working I will have to debug. Can you tell me more ? Yes, you are right, it does show this additional information, like {{n EOL}}, but the output number helped me identify the character which was not allowed in one particular token, I didn't mention the other info because I didn't understand the meaning. I had the reverse problem at the time: trying to identify what was wrong with my PEG grammar definition, since I knew the document was correct.
From: tom.rmadilo on 17 Jun 2010 12:01
On Jun 16, 11:02 pm, Andreas Kupries <akupr...(a)shaw.ca> wrote: > "tom.rmadilo" <tom.rmad...(a)gmail.com> writes: > > > package provide PACKAGE 1 > > return > > % > > > The output for the gen-json.tcl, which for some reason is incorrect in > > the manpages is: > > Huh. Looking ... Oh, My example in the docs is not quiting the /, > i.e. not the \/ you have below, right ? > > > % source gen-json.tcl > > { > > "pt::grammar::peg" : { > > "rules" : { > > "AddOp" : { > > "is" : "\/ {t +} {t -}", > Two issues with the JSON output 1): was missing an ending quote at the end of "Expression :, and 2): also missing commas between some objects. However the output from you code PEG to JSON using the calculator example gave the correct output. "Expression : { "is" : "/ {x {t (} {n Expression} {t )}} {x {n Factor} {* {x {n MulOp} {n Factor}}}}", "mode" : "value" } "Factor" : { "is" : "x {n Term} {* {x {n AddOp} {n Term}}}", "mode" : "value" } Before "Factor" and others at same level, the previous line should be "}," instead of just "}". |