Incremental Java Compile
in [Java Programming]
|
From: Tom Anderson on 14 May 2010 17:25 On Wed, 12 May 2010, Joshua Maurice wrote: > //AA.java > public class AA { public final int x = 1; } > > //BB.java > public class BB { public int x = new AA().x; } > > //javap -verbose -classpath . BB > public BB(); > Code: > Stack=3, Locals=1, Args_size=1 > 0: aload_0 > 1: invokespecial #1; //Method java/lang/Object."<init>":()V > 4: aload_0 > 5: new #2; //class AA > 8: dup > 9: invokespecial #3; //Method AA."<init>":()V > 12: invokevirtual #4; //Method java/lang/Object.getClass:()Ljava/lang/Class; > 15: pop > 16: iconst_1 > 17: putfield #5; //Field x:I > 20: return > > Specifically note that the instructions to initialize BB.x involve > "iconst_1", which, as I understand it, puts the constant 1 on the > stack. javac, even with -g, inlined the value of a final not-static > int field. Yeah, this is a bit weird. Also, what the hell is that getClass call all about? I see that in code i compile too (javac 1.6.0_16). A bit of googling reveals it's the code generated to force a null check of a variable, and this is used in compiling certain contortions involving inner classes. But there's no inner class here, and there is no way in a month of sundays that the top of stack can be null at instruction 12 - it's produced by applying dup to the result of new, and new can never produce a null (right?). So what's it doing? Anyway, turning back to the initialisation of x. if you look at the bytecode of AA, that's also weird. It has a constructor which does iconst_1 + putfield to initialise x - but x *also* has a ConstantValue attribute, giving it the value 1. Why both? If you write a verion of AA where x is static, then there's only a ConstantValue, and no synthetic clinit or anything touching it. Or instead make it non-final, and of course it keeps the constructor but loses the ConstantValue. The good news is that it looks like you can detect 'silently inlinable' variables by the presence of a ConstantValue attribute. The bad news is that javac does seem to be violating the VM spec (AIUI) here. And on the gripping hand, you still have no way to discover the relevance of AA from CC (the class you mention in a later post). When i looked into this a while ago, my planned approach was: 1. Keep a table of explicit dependencies between classes (ie CC -> BB, but not CC -> AA) 2. Keep a tree of direct inheritance relationships, probably including interface implementation (ie BB -> AA) 3. Define the 'signature' of a class to be the aggregation of its kind (class or interface), name, list of direct supertypes, the names and types of its non-private fields, the values of its constant fields, and the names, parameter types, return types, and exception lists of its methods. Anything else? 4. When a source file changes, recompile, and compare the signature of the new class to that of the old class 5. If the signature has changed, walk the inheritance tree, and build the set of all classes which descend from the class - call this, including the original class, the family. 6. Use the dependency table to find every class which depends on a member of the family. Call these the friends. 7. Recompile the family and friends. 8. Repeat the analysis on the newly recompiled files; this is necessary because changes to constant values can propagate. If you extend javap to report constant field values, then you can use the hash of the output of javap has a practical stand-in for a complete signature. It's a bit oversensitive, because it will change if you add or remove a static block, or cause the set of secret inner-class backdoor methods to change, neither of which really change the signature. I didn't know about ghost dependencies, so i didn't deal with those at all. But on that subject - am i right in thinking that to build the set of ghost dependencies, you need to know every name used by the class? If so, doesn't that already cover this situation? CC uses the name BB.x, and presumably you have to have an inheritance rule like the above that means that a change to AA.x means a change to BB.x if there is no actual BB.x. It is really bloody annoying that compile-time constants can be inlined like this. Would it be legal for a compiler to *not* inline them? If so, an option to javac to tell it to do that would be incredibly useful in a situation like this. tom -- inspired by forty-rod whiskey
From: Joshua Maurice on 14 May 2010 18:44 On May 14, 2:25 pm, Tom Anderson <t...(a)urchin.earth.li> wrote: > On Wed, 12 May 2010, Joshua Maurice wrote: > > //AA.java > > public class AA { public final int x = 1; } > > > //BB.java > > public class BB { public int x = new AA().x; } > > > //javap -verbose -classpath . BB > > public BB(); > > Code: > > Stack=3, Locals=1, Args_size=1 > > 0: aload_0 > > 1: invokespecial #1; //Method java/lang/Object."<init>":()V > > 4: aload_0 > > 5: new #2; //class AA > > 8: dup > > 9: invokespecial #3; //Method AA."<init>":()V > > 12: invokevirtual #4; //Method java/lang/Object.getClass:()Ljava/lang/Class; > > 15: pop > > 16: iconst_1 > > 17: putfield #5; //Field x:I > > 20: return > > > Specifically note that the instructions to initialize BB.x involve > > "iconst_1", which, as I understand it, puts the constant 1 on the > > stack. javac, even with -g, inlined the value of a final not-static > > int field. > > Yeah, this is a bit weird. > > Also, what the hell is that getClass call all about? I see that in code i > compile too (javac 1.6.0_16). A bit of googling reveals it's the code > generated to force a null check of a variable, and this is used in > compiling certain contortions involving inner classes. But there's no > inner class here, and there is no way in a month of sundays that the top > of stack can be null at instruction 12 - it's produced by applying dup to > the result of new, and new can never produce a null (right?). So what's it > doing? > > Anyway, turning back to the initialisation of x. if you look at the > bytecode of AA, that's also weird. It has a constructor which does > iconst_1 + putfield to initialise x - but x *also* has a ConstantValue > attribute, giving it the value 1. Why both? If you write a verion of AA > where x is static, then there's only a ConstantValue, and no synthetic > clinit or anything touching it. Or instead make it non-final, and of > course it keeps the constructor but loses the ConstantValue. > > The good news is that it looks like you can detect 'silently inlinable' > variables by the presence of a ConstantValue attribute. The bad news is > that javac does seem to be violating the VM spec (AIUI) here. > > And on the gripping hand, you still have no way to discover the relevance > of AA from CC (the class you mention in a later post). > > When i looked into this a while ago, my planned approach was: > > 1. Keep a table of explicit dependencies between classes (ie CC -> BB, but > not CC -> AA) > > 2. Keep a tree of direct inheritance relationships, probably including > interface implementation (ie BB -> AA) > > 3. Define the 'signature' of a class to be the aggregation of its > kind (class or interface), name, list of direct supertypes, the names > and types of its non-private fields, the values of its constant fields, > and the names, parameter types, return types, and exception lists of > its methods. Anything else? > > 4. When a source file changes, recompile, and compare the signature of the > new class to that of the old class > > 5. If the signature has changed, walk the inheritance tree, and build > the set of all classes which descend from the class - call this, > including the original class, the family. > > 6. Use the dependency table to find every class which depends on a member > of the family. Call these the friends. > > 7. Recompile the family and friends. > > 8. Repeat the analysis on the newly recompiled files; this is necessary > because changes to constant values can propagate. > > If you extend javap to report constant field values, then you can use the > hash of the output of javap has a practical stand-in for a complete > signature. It's a bit oversensitive, because it will change if you add or > remove a static block, or cause the set of secret inner-class backdoor > methods to change, neither of which really change the signature. > > I didn't know about ghost dependencies, so i didn't deal with those at > all. But on that subject - am i right in thinking that to build the set of > ghost dependencies, you need to know every name used by the class? If so, > doesn't that already cover this situation? CC uses the name BB.x, and > presumably you have to have an inheritance rule like the above that means > that a change to AA.x means a change to BB.x if there is no actual BB.x. See http://www.jot.fm/issues/issue_2004_12/article4.pdf for "ghost dependencies". I don't think as presented in the paper that ghost dependencies will catch this. Again, take the example //AA.java public class AA { public final int x = 1; } //BB.java public class BB extends AA {} //CC.java public class CC { public final int x = new BB().x; } CC.java has ghost dependencies "CC", "BB", "x", aka all names in the class file (using the Java technical definition of "name" as a single identifier, or a list of identifiers separated by dots '.'), then get all possible interpretations under all imports (including the implicit import <this-package>.*;), then close over all such prefixes. (Or something like that. The details are somewhat involved. See the paper.) AA.class exports the name "AA", aka the full name of the class. BB.class exports the name "BB", aka the full name of the class. I'm not sure offhand if there is a good way to extend ghost dependencies to catch this case without introduces a lot of false positives. -- I've also given some thought as you had to maintain this list keeping track of super classes. I'm not sure how it would interact with this example: //AAA.java public class AAA { public static int aaa = 1; } //BBB.java public class BBB { public static AAA bbb = null; } //CCC.java public class CCC { public static BBB ccc = null; } //DDD.java public class DDD { public final int ddd = CCC.ccc.bbb.aaa; } If we chance AAA.aaa to "public static double aaa = 2", then BBB.class would be a noop recompile, CCC.class would be a noop recompile, but DDD.class would need a recompile. Again, I think I would need the same information to make this work without endless cascading; I would need to know that DDD (directly) uses AAA. I thus think that your / my scheme of keeping tracking of super classes would not be terribly effective / productive.
From: Arne Vajhøj on 14 May 2010 18:46 On 13-05-2010 21:14, Joshua Maurice wrote: > On May 13, 5:07 pm, Arne Vajh�j<a...(a)vajhoej.dk> wrote: >> On 13-05-2010 03:43, Joshua Maurice wrote: >>> On May 12, 2:37 pm, Arne Vajh�j<a...(a)vajhoej.dk> wrote: >>>> On 12-05-2010 17:35, Joshua Maurice wrote: >>>>> Bad news. It appears that class files do not contain the necessary >>>>> dependency information for my goal of not rebuilding all java files >>>>> downstream. Ex: >> >>>>> //AA.java >>>>> public class AA { public final int x = 1; } >> >>>>> //BB.java >>>>> public class BB { public int x = new AA().x; } >> >>>> Well - we told you that last week, so that should not >>>> come as a surprise. >> >>> I believe no one suggested specifically that "The dependency >>> information contained in class files compiled with debug information >>> ala javac -g and ghost dependencies obtained via JavacTask in >>> tools.jar (which alone is highly underspecified) is insufficient to do >>> a Java file level incremental build which does not cascade endlessly >>> downstream." >> >> No but the case of constants not being in the class file >> where explicitly discussed. > > I'm sorry. Perhaps I am mistaken, but I recall only discussion of > "static finals", not "constants". It does appear that "constants" is > the correct conclusion. static final is constant in Java. Arne
From: Joshua Maurice on 14 May 2010 19:25 On May 14, 3:46 pm, Arne Vajhøj <a...(a)vajhoej.dk> wrote: > On 13-05-2010 21:14, Joshua Maurice wrote: > > > > > On May 13, 5:07 pm, Arne Vajhøj<a...(a)vajhoej.dk> wrote: > >> On 13-05-2010 03:43, Joshua Maurice wrote: > >>> On May 12, 2:37 pm, Arne Vajhøj<a...(a)vajhoej.dk> wrote: > >>>> On 12-05-2010 17:35, Joshua Maurice wrote: > >>>>> Bad news. It appears that class files do not contain the necessary > >>>>> dependency information for my goal of not rebuilding all java files > >>>>> downstream. Ex: > > >>>>> //AA.java > >>>>> public class AA { public final int x = 1; } > > >>>>> //BB.java > >>>>> public class BB { public int x = new AA().x; } > > >>>> Well - we told you that last week, so that should not > >>>> come as a surprise. > > >>> I believe no one suggested specifically that "The dependency > >>> information contained in class files compiled with debug information > >>> ala javac -g and ghost dependencies obtained via JavacTask in > >>> tools.jar (which alone is highly underspecified) is insufficient to do > >>> a Java file level incremental build which does not cascade endlessly > >>> downstream." > > >> No but the case of constants not being in the class file > >> where explicitly discussed. > > > I'm sorry. Perhaps I am mistaken, but I recall only discussion of > > "static finals", not "constants". It does appear that "constants" is > > the correct conclusion. > > static final is constant in Java. Ok. Let me try again. I recall only discussions of static finals. The problem of expanding inline "things" also happens with non-static final fields. I do not recall any discussion previous to that post of mine about non-static final fields being a problem. I'm sorry; I did not know that "constant" means "static final" in Java.
From: lewis on 14 May 2010 20:24
Arne Vajhøj wrote: >> static final is constant in Java. > Joshua Maurice <joshuamaur...(a)gmail.com> wrote: > Ok. Let me try again. I recall only discussions of static finals. The > problem of expanding inline "things" also happens with non-static > final fields. I do not recall any discussion previous to that post of > mine about non-static final fields being a problem. I'm sorry; I did > not know that "constant" means "static final" in Java. > Strictly speaking, it doesn't. It's a really, really, really good idea to study the language spec if you actually want to learn Java. The relevant section is: <http://java.sun.com/docs/books/jls/third_edition/html/ typesValues.html#10931> -- Lew |