Compiling C++ Templates as opposed to Preprocessing them.
in [C++]
|
Prev: basic concatenation question
Next: type casting issue
From: Walter Bright on 5 Aug 2010 12:29 CornedBee wrote: > Token-caching implementations like VC++ and DMC+ > + use mechanics that are similar to the preprocessor, but I doubt > there's any code sharing going on. Digital Mars C++ does not use token caching for the preprocessor, it stores the macros as text. I was told that it was impossible to do a correct preprocessor implementation this way, but DMC++ is 100% correct (passes Mensonidas' suite). (BTW, it was far harder to get a correct pp implementation than to do the two phase name lookup. My main issue with the pp was getting it to be super fast, which is devilishly hard while maintaining all the weird rules of the pp. I figured the only way to get the speed was by making it a text based one.) -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Walter Bright on 5 Aug 2010 12:28 CornedBee wrote: > On Aug 1, 10:51 pm, Walter Bright <newshou...(a)digitalmars.com> wrote: >> CornedBee wrote: >>> I am pretty sure, from observing its behaviour, that MSVC caches the >>> token stream instead of building an AST, but that's not a good >>> approach: there are various bugs that can be traced to this >>> implementation choice. >> Please give an example. > Here's an interesting bug, though. It demonstrates that VC++ most > likely uses token-caching, and also some other details about how it > works. I'm recreating it from memory here, so I can't guarantee this > really demonstrates the bug I encountered. #include <iostream> > namespace detail { > struct S {}; > } > namespace foo { > template <typename X> > void f(X, detail::S *) { > std::cerr << "Template\n"; > } > void f(int, bool) { > std::cerr << "Plain\n"; > } > > namespace detail { > } > } > int main() { > detail::S s; > foo::f(1, &s); > } > > A conforming compiler will compile this program to print "Template". VC > ++ compiled it to print "Plain". Digital Mars C++ compiles it and prints "Template", yet uses token caching. > This is what surprises me about your statement that token-caching is > easier than tree substitution. I would have thought that really > getting name lookup (in particular two-phase name lookup) right for > token caching is devilishly hard. No, it's straightforward, though it took me a while to figure it out. > You either have to attach the lookup > result to the cached identifier tokens (but you can't do that, because > you need a proper parse to determine if you're even supposed to look > up a name), or you need to be able to recreate the exact lookup > context at the template definition point. I don't know about DMC++, > but in Clang that would be a major challenge. > If you have a better idea, I'd be interested in hearing about it. Just keep track of the source locations of each definition in the symbol table. You probably do that already for the symbolic debug info. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Walter Bright on 5 Aug 2010 12:30 Nikolay Ivchenkov wrote: > On 5 Aug, 06:15, Walter Bright <newshou...(a)digitalmars.com> wrote: >> Nikolay Ivchenkov wrote: >>> On 4 Aug, 08:30, Walter Bright <newshou...(a)digitalmars.com> wrote: >>>> My reading of C++98 was that the compiler was not required to issue a diagnostic >>>> for a malformed definition if it was never instantiated. >>> What's the malformed definition? >> One that doesn't follow the grammar. >> >>> (I want to see normative criteria if >>> they exist). For example, can we consider the following sequence of >>> tokens >>> %+*!^~ >>> as a malformed template definition (according to your interpretation)? >> Yes. [...] > There shall be standard criteria for determining which option is > right. Can you show these criteria? Consider this quote from C++98 14.6.7 which I believe overrides the quote from the introduction: "Knowing which names are type names allows the syntax of every template definition to be checked. No diagnostic shall be issued for a template definition for which a valid specialization can be generated. If no valid specialization can be generated for a template definition, and that template is not instantiated, the template definition is illformed, no diagnostic required. [Note: if a template is instantiated, errors will be diagnosed according to the other rules in this Standard. Exactly when these errors are diagnosed is a quality of implementation issue. ]" This suggests that if the template is never instantiated, errors in it do not have to be diagnosed. BTW, the token collection scheme for templates counts the braces. Your example won't compile with DMC++. The template definition "parser" does just enough to correctly find the end of the definition. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Nikolay Ivchenkov on 5 Aug 2010 12:36 On 5 Aug, 23:48, Walter Bright <newshou...(a)digitalmars.com> wrote: > CornedBee wrote: > > On Aug 4, 7:15 pm, Walter Bright <newshou...(a)digitalmars.com> wrote: > >> Felipe Magno de Almeida wrote: > > >>> On Aug 4, 1:30 am, Walter Bright <newshou...(a)digitalmars.com> wrote: > >>>> My reading of C++98 was that the compiler was not required to issue a > diagnostic > >>>> for a malformed definition if it was never instantiated. > >>> Unless I'm missing something, I think it is required to diagnose. > >> At this point, I'll require a specific quote from C++98 <g>. > > > [temp.res]p8 gives an explicit example of completely nonsensical code > > (not even syntactically correct), with the note that it *may* be > > diagnosed even if the template is never instantiated. In other words, > > as long as a template is not instantiated, you can have whatever you > > want in there, as long as the parser is still able to find the end of > > the template. > > Right, so it's not required. Examples and notes are not normative part of the standard. Moreover, some examples are provided with inappropriate comments. 1) 3.4.1/3: [quote] typedef int f; struct A { friend void f(A &); operator int(); void g(A a) { f(a); } }; The expression f(a) is a cast-expression equivalent to int(a). [/quote] f(a) is not an expression, since f(a); is treated as a declaration of a variable with name "a". The issue is fixed in C++0x. 2) 5/4: [quote] i = v[i++]; // the behavior is unspecified i = 7, i++, i++; // i becomes 9 i = ++i + 1; // the behavior is unspecified i = i + 1; // the value of i is incremented [/quote] Here both occurrences of "the behavior is unspecified" must be replaced with "the behavior is undefined". The issue is fixed in C+ +0x. 3) 5.2.7/9: [quote] class A { virtual void f(); }; class B { virtual void g(); }; class D : public virtual A, private B {}; void g() { D d; B* bp = (B*)&d; // cast needed to break protection A* ap = &d; // public derivation, no cast needed D& dr = dynamic_cast<D&>(*bp); // fails ap = dynamic_cast<A*>(bp); // fails bp = dynamic_cast<B*>(ap); // fails ap = dynamic_cast<A*>(&d); // succeeds bp = dynamic_cast<B*>(&d); // fails } class E : public D, public B {}; class F : public E, public D {}; void h() { F f; A* ap = &f; // succeeds: finds unique A D* dp = dynamic_cast<D*>(ap); // fails: yields 0 // f has two D sub-objects E* ep = (E*)ap; // ill-formed: // cast from virtual base E* ep1 = dynamic_cast<E*>(ap); // succeeds } [/quote] The comment in the line bp = dynamic_cast<B*>(&d); // fails shall be changed to "ill-formed". The issue is fixed in C++0x. Similarly, in 14.6/7 int j; template<class T> class X { // ... void f(T t, int i, char* p) { t = i; // diagnosed if X::f is instantiated // and the assignment to t is an error p = i; // may be diagnosed even if X::f is // not instantiated p = j; // may be diagnosed even if X::f is // not instantiated } void g(T t) { +; // may be diagnosed even if X::g is // not instantiated } }; the sequence of tokens void g(T t) { +; } shall be completely removed from the example, because otherwise the entire code does not contain a template definition. This issue is not fixed yet. In any case examples do not define standard conformance. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Nikolay Ivchenkov on 6 Aug 2010 10:01
On 6 Aug, 07:30, Walter Bright <newshou...(a)digitalmars.com> wrote: > Nikolay Ivchenkov wrote: > > On 5 Aug, 06:15, Walter Bright <newshou...(a)digitalmars.com> wrote: > >> Nikolay Ivchenkov wrote: > >>> On 4 Aug, 08:30, Walter Bright <newshou...(a)digitalmars.com> wrote: > >>>> My reading of C++98 was that the compiler was not required to issue a diagnostic > >>>> for a malformed definition if it was never instantiated. > >>> What's the malformed definition? > >> One that doesn't follow the grammar. > > >>> (I want to see normative criteria if > >>> they exist). For example, can we consider the following sequence of > >>> tokens > >>> %+*!^~ > >>> as a malformed template definition (according to your interpretation)? > >> Yes. > > [...] > > > There shall be standard criteria for determining which option is > > right. Can you show these criteria? > > Consider this quote from C++98 14.6.7 which I believe overrides the quote from > the introduction: > > "Knowing which names are type names allows the syntax of every template > definition to be checked. It allows to distinguish template definitions from any other sequences of tokens. > If no > valid specialization can be generated for a template definition, and that > template is not instantiated, the template definition is illformed, > no diagnostic required. We may apply this rule only to a sequence of tokens that can be recognized as a template definition according to syntactic rules. A template definition can be semantically incorrect, but every template definition is syntactically correct _by definition_ (there are no criteria to distinguish template definitions from any other sequences of tokens other than syntactic rules). > This suggests that if the template is never instantiated, errors in it do not > have to be diagnosed. That's right only for semantic errors. > BTW, the token collection scheme for templates counts the braces. Look at the grammar of C++. Do you see something like template-declaration: export opt template < template-parameter-list > potential- declaration potential-declaration: potential-function-declaration potential-class-declaration potential-static-data-member-definition potential-function-declaration: simple-declaration potential-function-definition template-declaration potential-function-definition: decl-specifier-seq opt declarator p-ctor-initializer opt potential- function-body decl-specifier-seq opt declarator potential-function-try-block potential-function-body: { potential-statement-seq opt } potential-statement-seq: potential-statement-seq opt { potential-statement-seq opt } not-a-brace-seq not-a-brace-seq: not-a-brace-seq opt not-a-brace not-a-brace: token (where not-a-brace is any token except { and }) in the standard? I don't see. So if your interpretation would be correct, how exactly the bounds of a template should be determined? > Your example > won't compile with DMC++. The template definition "parser" does just enough to > correctly find the end of the definition. If your interpretation would be correct, the end of this template should be strictly defined, because it affects whether the diagnostic is required. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] |