Incremental Java Compile
in [Java Programming]
|
From: Lew on 10 May 2010 00:42 Mike Schilling wrote: > If the change is from > > public static final int I = 2; > > to > > public static final int I = 1 + 1; > > I'd accept a system that recompiles all users of I as still "minimal". (Not > that it's a difficult optimization to make, since the rules for what's a > compile-time constant are straightforward.) If this change isn't to a > compile-time constant, it would have no effect on anything defined in a > different source file. That source change would produce no change in the bytecode of the class wherein 'I' is defined. Ergo there would be no need for users of I to be recompiled. So a system that recompiles no users of I in that case would be "minimal". I'm having a hard time wrapping my mind around your last sentence in that paragraph. -- Lew
From: Mike Schilling on 10 May 2010 02:18 Lew wrote: > Mike Schilling wrote: >> If the change is from >> >> public static final int I = 2; >> >> to >> >> public static final int I = 1 + 1; >> >> I'd accept a system that recompiles all users of I as still >> "minimal". (Not that it's a difficult optimization to make, since >> the rules for what's a compile-time constant are straightforward.) >> If this change isn't to a compile-time constant, it would have no >> effect on anything defined in a different source file. > > That source change would produce no change in the bytecode of the > class wherein 'I' is defined. > > Ergo there would be no need for users of I to be recompiled. > > So a system that recompiles no users of I in that case would be > "minimal". I'm saying that one that says "definition of I changed, we need to recompile I's users" would be minimal enough for me. > > I'm having a hard time wrapping my mind around your last sentence in > that paragraph. If the sole change were void doit() { int i = 1 + 1; //formerly int i = 2 } it's still true that the class's bytecode doesn't change. It's also true that nothing about A that could possibly require anything else to recompile changed, since there are never dependencies on method implementations. Thus there's no need to recognize that A's bytecode didn't change, because it's really irrelevant; there's also be no need to recompile any other files if the sole change were void doit() { int i = 1 + 2; //formerly int i = 2 }
From: Joshua Maurice on 12 May 2010 17:35 Bad news. It appears that class files do not contain the necessary dependency information for my goal of not rebuilding all java files downstream. Ex: //AA.java public class AA { public final int x = 1; } //BB.java public class BB { public int x = new AA().x; } //javap -verbose -classpath . BB Compiled from "BB.java" public class BB extends java.lang.Object SourceFile: "BB.java" minor version: 0 major version: 50 Constant pool: const #1 = Method #7.#19; // java/lang/Object."<init>":()V const #2 = class #20; // AA const #3 = Method #2.#19; // AA."<init>":()V const #4 = Method #7.#21; // java/lang/Object.getClass:()Ljava/ lang/Class ; const #5 = Field #6.#22; // BB.x:I const #6 = class #23; // BB const #7 = class #24; // java/lang/Object const #8 = Asciz x; const #9 = Asciz I; const #10 = Asciz <init>; const #11 = Asciz ()V; const #12 = Asciz Code; const #13 = Asciz LineNumberTable; const #14 = Asciz LocalVariableTable; const #15 = Asciz this; const #16 = Asciz LBB;; const #17 = Asciz SourceFile; const #18 = Asciz BB.java; const #19 = NameAndType #10:#11;// "<init>":()V const #20 = Asciz AA; const #21 = NameAndType #25:#26;// getClass:()Ljava/lang/Class; const #22 = NameAndType #8:#9;// x:I const #23 = Asciz BB; const #24 = Asciz java/lang/Object; const #25 = Asciz getClass; const #26 = Asciz ()Ljava/lang/Class;; { public int x; public BB(); Code: Stack=3, Locals=1, Args_size=1 0: aload_0 1: invokespecial #1; //Method java/lang/Object."<init>":()V 4: aload_0 5: new #2; //class AA 8: dup 9: invokespecial #3; //Method AA."<init>":()V 12: invokevirtual #4; //Method java/lang/Object.getClass:()Ljava/ lang/Clas s; 15: pop 16: iconst_1 17: putfield #5; //Field x:I 20: return LineNumberTable: line 1: 0 LocalVariableTable: Start Length Slot Name Signature 0 21 0 this LBB; } ///// ///// Specifically note that the instructions to initialize BB.x involve "iconst_1", which, as I understand it, puts the constant 1 on the stack. javac, even with -g, inlined the value of a final not-static int field. If it can do this, I don't know what else it can do, and I don't want to try to write tests to catch all possible variations. So, I'm back to calling javac -verbose once per java file (but still only one JVM), and parsing the verbose output to get the actual class compile dependencies. From initial testing, this slows down a clean build by a factor of 4x to 5x for my code base. So, I'm back to square one. Anyone know a way to get the dependency information javac -verbose would supply per java file without calling javac -verbose (using the tools.jar API) once per java file? Anyone have an inkling of how hard this would be to just modify javac itself to output the information I need? I assume the change would be quite minor, relatively speaking. It already prints out a message when it loads a class file, and it has to know which java file it's loading it for; it's just that it only prints the message the first time loading the class file (presumably caching its contents), and does not print out the java file for which it's being loaded.
From: Arne Vajhøj on 12 May 2010 17:37 On 12-05-2010 17:35, Joshua Maurice wrote: > Bad news. It appears that class files do not contain the necessary > dependency information for my goal of not rebuilding all java files > downstream. Ex: > > //AA.java > public class AA { public final int x = 1; } > > //BB.java > public class BB { public int x = new AA().x; } Well - we told you that last week, so that should not come as a surprise. Arne
From: Joshua Maurice on 13 May 2010 03:43
On May 12, 2:37 pm, Arne Vajhøj <a...(a)vajhoej.dk> wrote: > On 12-05-2010 17:35, Joshua Maurice wrote: > > > Bad news. It appears that class files do not contain the necessary > > dependency information for my goal of not rebuilding all java files > > downstream. Ex: > > > //AA.java > > public class AA { public final int x = 1; } > > > //BB.java > > public class BB { public int x = new AA().x; } > > Well - we told you that last week, so that should not > come as a surprise. I believe no one suggested specifically that "The dependency information contained in class files compiled with debug information ala javac -g and ghost dependencies obtained via JavacTask in tools.jar (which alone is highly underspecified) is insufficient to do a Java file level incremental build which does not cascade endlessly downstream." Nevertheless, I have finished the implementation, and it appears to be working quite well, passing a small suite of tests I wrote beforehand. I plan to implement this on ~1/8 of a piece of my product's code base, ~100 Maven poms, and see how well it performs. I've never written something quite like this in Java before (yes, I'm a C++ guy in case you couldn't tell), and the design is new as well. I'm cautiously optimistic. |