Parser generator (+ Lexer) that actually works?
in [TCL]
|
From: Fredrik Karlsson on 20 Jun 2010 12:03 On Jun 17, 8:26 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > Choice (/) is prioritized. Order the branches so that it tries to > parse the larger/longer possibility first. Hi, ok, then I think I need to some input on what I am doing, since I cannot get it to work: I have the following grammar: --------- PEG EmuQL (expr) expr <- domexpr/ltexpr; domexpr <- '[' expr '^' expr ']' ; ltexpr <- '[' lt compop lseq ']' ; lt <- <alnum>+ ; lseq <- label ('|' label)* ; label <- <alnum>+ ; compop <- '=' ; END ; --------- and the follwing example code: --------- package require snit source emuql_parser.tcl set p [emuql_parser constructor] puts [$p parset {[Structure=a]} ] puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ] ------ which then produces this output: ------ $ tclsh8.6 Make_EmuQL_parser.tcl && tclsh8.6 Test_emuql_parser.tcl expr 0 12 {ltexpr 0 12 {lt 1 9} {compop 10 10} {lseq 11 11 {label 11 11}}} pt::rde 1 {{n lt}} while executing "$myparser complete" (procedure "::emuql_parser::Snit_methodparset" line 9) invoked from within "$p parset {[ [Structure=a] ^ [Same=a] ]} " invoked from within "puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ]" (file "Test_emuql_parser.tcl" line 8) --- How come expr is not found to be a domexpr in the first iterantion, and then ltexpr two times in the subsequent ones... ? /Fredrik
From: tom.rmadilo on 20 Jun 2010 13:23 On Jun 20, 9:03 am, Fredrik Karlsson <dargo...(a)gmail.com> wrote: > On Jun 17, 8:26 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > > > Choice (/) is prioritized. Order the branches so that it tries to > > parse the larger/longer possibility first. > > Hi, > > ok, then I think I need to some input on what I am doing, since I > cannot get it to work: > > I have the following grammar: > > --------- > PEG EmuQL (expr) > expr <- domexpr/ltexpr; > domexpr <- '[' expr '^' expr ']' ; > ltexpr <- '[' lt compop lseq ']' ; > lt <- <alnum>+ ; > lseq <- label ('|' label)* ; > label <- <alnum>+ ; > compop <- '=' ; > END ; > --------- > and the follwing example code: > > --------- > package require snit > > source emuql_parser.tcl > > set p [emuql_parser constructor] > > puts [$p parset {[Structure=a]} ] > puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ] > ------ > > which then produces this output: > ------ > > $ tclsh8.6 Make_EmuQL_parser.tcl && tclsh8.6 Test_emuql_parser.tcl > expr 0 12 {ltexpr 0 12 {lt 1 9} {compop 10 10} {lseq 11 11 {label 11 > 11}}} > pt::rde 1 {{n lt}} > while executing > "$myparser complete" > (procedure "::emuql_parser::Snit_methodparset" line 9) > invoked from within > "$p parset {[ [Structure=a] ^ [Same=a] ]} " > invoked from within > "puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ]" > (file "Test_emuql_parser.tcl" line 8) > --- > > How come expr is not found to be a domexpr in the first iterantion, > and then ltexpr two times in the subsequent ones... ? I don't see any recursion/nesting at the expr level. You can look at my PEG example which contains two types of statements. One is an element which can contain other elements or attributes, and the document itself can contain multiple elements or attributes, but there is no nesting of documents.
From: tom.rmadilo on 20 Jun 2010 13:28 On Jun 20, 10:23 am, "tom.rmadilo" <tom.rmad...(a)gmail.com> wrote: > On Jun 20, 9:03 am, Fredrik Karlsson <dargo...(a)gmail.com> wrote: > > > > > > > On Jun 17, 8:26 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > > > > Choice (/) is prioritized. Order the branches so that it tries to > > > parse the larger/longer possibility first. > > > Hi, > > > ok, then I think I need to some input on what I am doing, since I > > cannot get it to work: > > > I have the following grammar: > > > --------- > > PEG EmuQL (expr) > > expr <- domexpr/ltexpr; > > domexpr <- '[' expr '^' expr ']' ; > > ltexpr <- '[' lt compop lseq ']' ; > > lt <- <alnum>+ ; > > lseq <- label ('|' label)* ; > > label <- <alnum>+ ; > > compop <- '=' ; > > END ; > > --------- > > and the follwing example code: > > > --------- > > package require snit > > > source emuql_parser.tcl > > > set p [emuql_parser constructor] > > > puts [$p parset {[Structure=a]} ] > > puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ] > > ------ > > > which then produces this output: > > ------ > > > $ tclsh8.6 Make_EmuQL_parser.tcl && tclsh8.6 Test_emuql_parser.tcl > > expr 0 12 {ltexpr 0 12 {lt 1 9} {compop 10 10} {lseq 11 11 {label 11 > > 11}}} > > pt::rde 1 {{n lt}} > > while executing > > "$myparser complete" > > (procedure "::emuql_parser::Snit_methodparset" line 9) > > invoked from within > > "$p parset {[ [Structure=a] ^ [Same=a] ]} " > > invoked from within > > "puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ]" > > (file "Test_emuql_parser.tcl" line 8) > > --- > > > How come expr is not found to be a domexpr in the first iterantion, > > and then ltexpr two times in the subsequent ones... ? > > I don't see any recursion/nesting at the expr level. You can look at > my PEG example which contains two types of statements. One is an > element which can contain other elements or attributes, and the > document itself can contain multiple elements or attributes, but there > is no nesting of documents. I'm also not seeing anything accounting for whitespace: {[ [Structure=a] ^ [Same=a] ]} ^--- whitespace at char 1.
From: tom.rmadilo on 20 Jun 2010 13:36 On Jun 20, 10:23 am, "tom.rmadilo" <tom.rmad...(a)gmail.com> wrote: > On Jun 20, 9:03 am, Fredrik Karlsson <dargo...(a)gmail.com> wrote: > > > > > > > On Jun 17, 8:26 am, Andreas Kupries <akupr...(a)shaw.ca> wrote: > > > > Choice (/) is prioritized. Order the branches so that it tries to > > > parse the larger/longer possibility first. > > > Hi, > > > ok, then I think I need to some input on what I am doing, since I > > cannot get it to work: > > > I have the following grammar: > > > --------- > > PEG EmuQL (expr) > > expr <- domexpr/ltexpr; > > domexpr <- '[' expr '^' expr ']' ; > > ltexpr <- '[' lt compop lseq ']' ; > > lt <- <alnum>+ ; > > lseq <- label ('|' label)* ; > > label <- <alnum>+ ; > > compop <- '=' ; > > END ; > > --------- > > and the follwing example code: > > > --------- > > package require snit > > > source emuql_parser.tcl > > > set p [emuql_parser constructor] > > > puts [$p parset {[Structure=a]} ] > > puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ] > > ------ > > > which then produces this output: > > ------ > > > $ tclsh8.6 Make_EmuQL_parser.tcl && tclsh8.6 Test_emuql_parser.tcl > > expr 0 12 {ltexpr 0 12 {lt 1 9} {compop 10 10} {lseq 11 11 {label 11 > > 11}}} > > pt::rde 1 {{n lt}} > > while executing > > "$myparser complete" > > (procedure "::emuql_parser::Snit_methodparset" line 9) > > invoked from within > > "$p parset {[ [Structure=a] ^ [Same=a] ]} " > > invoked from within > > "puts [$p parset {[ [Structure=a] ^ [Same=a] ]} ]" > > (file "Test_emuql_parser.tcl" line 8) > > --- > > > How come expr is not found to be a domexpr in the first iterantion, > > and then ltexpr two times in the subsequent ones... ? > > I don't see any recursion/nesting at the expr level. You can look at > my PEG example which contains two types of statements. One is an > element which can contain other elements or attributes, and the > document itself can contain multiple elements or attributes, but there > is no nesting of documents. Yikes! there is recursion here, so it is probably the whitespace giving you problems.
From: tom.rmadilo on 21 Jun 2010 01:04
On Jun 20, 10:36 am, "tom.rmadilo" <tom.rmad...(a)gmail.com> wrote: > Yikes! there is recursion here, so it is probably the whitespace > giving you problems. I adapted my simple development script to your peg and examples, removing the whitespace seems to solve the problem, but here are the files and script which I used for testing: http://rmadilo.com/files/peg/devel/ (see the emuql.tcl file) PEG files are in ./grammars/ Test texts are in ./emuql/ Run the script (after changing the first line to the correct tclsh): $ ./emuql.tcl emuql The results of running the script with the current test texts are in: http://rmadilo.com/files/peg/devel/emuql/emuql-test-results.txt For instance: --- 3.emuql -- --- [[Structure=a]^[Same=a]] --- expr 0 23 {domexpr 0 23 {expr 1 13 {ltexpr 1 13 {lt 2 10} {compop 11 11} {lseq 12 12 {label 12 12}}}} {expr 15 22 {ltexpr 15 22 {lt 16 19} {compop 20 20} {lseq 21 21 {label 21 21}}}}} |