Parser generator (+ Lexer) that actually works?
in [TCL]
|
From: Andreas Kupries on 17 Jun 2010 01:52 Andreas Kupries <akupries(a)shaw.ca> writes: > JHJL <jhjl(a)hippospace.com> writes: > >> Hi Andreas >> >> Many thanks for you work on providing the TCLLIB Parser Generator >> code. Alas, I am also confused by the documentation :( What I could >> really do with is a fully worked up example showing a PEG, the TclOO >> (or snit) output and a sample operation against some input data! >> Asking is easy :) >> >> The PT TCLIB docs makes a few references such as "please go and read >> the PEG Language Tutorial" ; a quick Google didn't reveal anything >> close... Can you provide a URL or am I missing something in my reading >> of the docs? > > The cross references between the various documents seem to be broken. > That is something I have to investigate. It is either trouble in the > documents, or in the code generating this for the AS website. The problem was in the code of the dtp processor (in tclapps) I used, it was never updated to use the title of manpages for cross-references, like it was done for the dtplite processor (in tcllib). This has been fixed, and the updated docs are up on our website, see http://docs.activestate.com/activetcl/8.5/tcllib/pt/pt_introduction.html Regarding larger examples, I worked on that off and on in the last week, and committed them this evening. I hope to get them up on the doc site in the next few days. -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: Andreas Kupries on 17 Jun 2010 01:59 JHJL <jhjl(a)hippospace.com> writes: > ============================================== > loading history file ... 48 events added > buffer line limit: 512 max line length: unlimited > Main console display active (Tcl8.6b1.2 / Tk8.6b1.2) > (PEG) 49 % ls > /Volumes/jhjl/Development/PEG: > example.peg* testpeg.tcl* > (PEG) 50 % pt > invalid command name "pt" pt is an application, not a builtin command of Tcl. > (PEG) 51 % package require pt::pgen > 1 > (PEG) 52 % package require fileutil > 1.14.2 > (PEG) 53 % set script [pt::pgen \ > peg [fileutil::cat example.peg] Here is the bug, I forgot the continuation line. > oo \ > -file example.peg \ > -name calculator \ > -user $tcl_platform(user) \ > -class calculator] > unknown or ambiguous subcommand "pt::grammar::peg {rules {AddOp {is > {/ > {t +} {t -}} mode value} Digit {is {/ {t 0} {t 1} {t 2} {t 3} {t 4} {t > 5} {t 6} {t 7} {t 8} {t 9}} mode value} Expression {is {/ {x {t (} {n > Expression} {t )}} {x {n Factor} {* {x {n MulOp} {n Factor}}}}} mode > value} Factor {is {x {n Term} {* {x {n AddOp} {n Term}}}} mode value} > MulOp {is {/ {t *} {t /}} mode value} Number {is {x {? {n Sign}} {+ {n > Digit}}} mode value} Sign {is {/ {t -} {t +}} mode value} Term {is {n > Number} mode value}} start {n Expression}}": My apologies. I wrote the example on the quick and must have mistyped something. -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: Andreas Kupries on 17 Jun 2010 02:02 "tom.rmadilo" <tom.rmadilo(a)gmail.com> writes: > > package provide PACKAGE 1 > return > % > > The output for the gen-json.tcl, which for some reason is incorrect in > the manpages is: Huh. Looking ... Oh, My example in the docs is not quiting the /, i.e. not the \/ you have below, right ? > % source gen-json.tcl > { > "pt::grammar::peg" : { > "rules" : { > "AddOp" : { > "is" : "\/ {t +} {t -}", -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: Andreas Kupries on 17 Jun 2010 02:11 JHJL <jhjl(a)hippospace.com> writes: > Many thanks Tom for the above kick in the right direction and to > Andreas, for what is an incredible body of work, greatly appreciated. > > Now that I can generate an AST from the example I just have towrk out > how to cook it :) Well, the/a helper/support package providing commands to traverse AST structures is pt::ast. The internal package pt::peg::from::peg for example uses 'pt::ast bottomup' to tranform the AST for a PEG into a PEG serialization, via a bottom up walk and a semi-ensemble command which interprets each symbol in the AST as a method which handles the transform of that symbol. -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> -------------------------------------------------------------------------------
From: Andreas Kupries on 17 Jun 2010 02:19
"tom.rmadilo" <tom.rmadilo(a)gmail.com> writes: > On Jun 14, 12:20 am, JHJL <j...(a)hippospace.com> wrote: >> On Jun 13, 5:58 pm, "tom.rmadilo" <tom.rmad...(a)gmail.com> wrote: >> >> > On Jun 13, 5:37 am, JHJL <j...(a)hippospace.com> wrote: > > >> Many thanks Tom for the above kick in the right direction and to >> Andreas, for what is an incredible body of work, greatly appreciated. >> >> Now that I can generate an AST from the example I just have towrk out >> how to cook it :) > > Note that I was unable to get the "oo" output to work. One of the > required packages was/is missing. Maybe the work hasn't been done > yet. Can you give me more information on that ? > However, the "snit" output works. There are comments in the generated > "oo" parser indicating something needs to be inherited from snit or > written. Hm ... Looking ... I might not have done the basic rde for TclOO yet, or incomplete. This will be more work than 'just' adding more examples to the docs. Lets see where I find (slices of) time for this. > I'll post a test script which helped me print out information on the > result of parsing a text (and how to do the parse once you have the > snit script). > > I'd like to put out one example for snit that uses something other > than the calculator example. In the CVS, the tests/data/ok/*/3_peg_itself is the PEG grammar in many forms, tests/data/ok/peg_tclparam-snit/3_peg_itself the snit. This one is bit large for an example however. > If anyone has a working example of this code which they can release, > please do so. I think a reasonable goal would be to provide an example > of parsing and transforming JSON since it is actually used in the pt > package, but the parser is hand written. Well, I use Tcllib's json package, which predated my work on pt, so I saw no need. I agree that this looks like a reasonably small example to do. -- So long, Andreas Kupries <akupries(a)shaw.ca> <http://www.purl.org/NET/akupries/> Developer @ <http://www.activestate.com/> ------------------------------------------------------------------------------- |