Incremental Java Compile
in [Java Programming]
|
From: Lew on 3 May 2010 07:38 On 05/03/2010 12:50 AM, Joshua Maurice wrote: Arne Vajhøj wrote: >> Structure things better. >> >> With a good OO model a series of related changes should not >> require changes in "a large portion of 20000 Java files". > So, how do you suggest doing that when there's a code generator under > active development which generates Java code, and a large portion of > the Java code directly or indirectly works with the output of this > code generator? We model the object domains in a simple modeling Generate code into packages. Generate different parts of the project into separate modules. > language which is then compiled to C++ and Java code to allow > serializing a description of a unit of work from the Java tools to the > C++ tools and back. Most of the infrastructure and apps work with the > output of this code generator in some form or another. The generator can be forced to follow good practices, rather than have bad practices use "the generator" as an excuse. > Unfortunately, one cannot fiat interfaces into being stable. But one can *design* interfaces to be modular. Try it. -- Lew
From: Mike Schilling on 3 May 2010 09:26 Joshua Maurice wrote: > On May 2, 11:54 pm, "Mike Schilling" <mscottschill...(a)hotmail.com> > wrote: >> Arne Vajh�j wrote: >>> On 02-05-2010 17:11, Joshua Maurice wrote: >>>> On May 1, 10:43 am, Lew<no...(a)lewscanon.com> wrote: >>>>> Mike Schilling wrote: >>>>>> "The most obvious example of these limitations is that the task >>>>>> can't tell which classes to recompile when a constant primitive >>>>>> data type exported by other classes is changed. For example, a >>>>>> change in the definition of something like >>>>>> public final class Constants { >>>>>> public final static boolean DEBUG=false; >>>>>> } >> >>>>>> will not be picked up by other classes. " >> >>>>>> That is, it's an incremental (no pun intended) improvement on the >>>>>> usual Ant algorithm of "recompile what onviouslt needs >>>>>> recompilation; if that doesn't seem to work, do a clean build" >> >>>>> You can't blame Ant for that one. The class that depends on the >>>>> compile-time constant, such as 'DEBUG' in your example, compiles >>>>> the constant into its class, not the symbol. Without some external >>>>> indication of the dependency, there's not any way for a compiler >>>>> or build tool to detect that it exists. With respect to >>>>> dependencies where the symbol is stored in the >>>>> class rather than its value, even 'javac' handles the situation >>>>> pretty well. >> >>>> I'm not blaming anyone in particular. I just want to know how to >>>> get a fully correct, aka 100% incremental build under the actions: >>>> adding, removing, modifying java files, and adding, removing, or >>>> modifying build steps of "take these jars, compile them to class >>>> files, then jar them", aka the standard developer actions. >> >>> To me the entire idea is rather pointless. >> >>> The tool can not be made to work with binaries. >> >> I think it could. At least, reducing the problem to: >> >> * You have a complete correct build of the system >> * You have a set of changes to the source since that build was done >> * Do a minimal (more or less) amount of recompliation to arrive at a >> new complete and correct build >> >> is feasible. >> >> But I don't see the point, really. It's simpler, cheaper, and more >> reliable to throw hardware at the problem. Buy a fast machine to do >> a continuous build and archive the last N days worth of builds. You >> can now fetch a completely built system at any release level with no >> compilation required at all. > > My company tried to do that, but I think they missed the important > part of the memo: that it only works when the code is decoupled, > modular, and relatively stable and well defined interfaces instead of > the ~25,000 source file mess we have now. Why doesn't it work, even with the mess you have now?
From: Roedy Green on 3 May 2010 12:28 On Sat, 1 May 2010 04:34:31 -0700 (PDT), Joshua Maurice <joshuamaurice(a)gmail.com> wrote, quoted or indirectly quoted someone who said : >system: A developer should be able to do any combination of the >following actions and trigger a small / minimal / incremental build, IF you use ANT, you don't need to bother with this. The time in a traditional compile is mostly loading Javac.exe. With ANT it gets compiled only once. Further JAVAC looks at dates of *.java and *.class files and avoids most unnecessary recompilation. Compiling is almost inconsequential. Building Jars and Zips takes much more of the time. See http://mindprod.com/jgloss/ant.html -- Roedy Green Canadian Mind Products http://mindprod.com It�s amazing how much structure natural languages have when you consider who speaks them and how they evolved.
From: Joshua Maurice on 3 May 2010 14:29 On May 3, 4:38 am, Lew <no...(a)lewscanon.com> wrote: > On 05/03/2010 12:50 AM, Joshua Maurice wrote: > > Arne Vajhøj wrote: > >> Structure things better. > > >> With a good OO model a series of related changes should not > >> require changes in "a large portion of 20000 Java files". > > So, how do you suggest doing that when there's a code generator under > > active development which generates Java code, and a large portion of > > the Java code directly or indirectly works with the output of this > > code generator? We model the object domains in a simple modeling > > Generate code into packages. Generate different parts of the project into > separate modules. > > > language which is then compiled to C++ and Java code to allow > > serializing a description of a unit of work from the Java tools to the > > C++ tools and back. Most of the infrastructure and apps work with the > > output of this code generator in some form or another. > > The generator can be forced to follow good practices, rather than have bad > practices use "the generator" as an excuse. So, I ask again: what if the generator changes, which it does "somewhat" frequently? I'd like to do a build in that case. The generator is an example of what ties all of the code together, though there's a couple more things. What's good practices for the generator, never change? Well, ideally yes, but that's beyond my control. > > Unfortunately, one cannot fiat interfaces into being stable. > > But one can *design* interfaces to be modular. Try it. Again, I do not hold sufficient sway, and we're dealing with a product with a code level published API which wasn't well designed, so we've coded ourselves into a corner, so to speak.
From: Joshua Maurice on 3 May 2010 14:36
On May 3, 9:28 am, Roedy Green <see_webs...(a)mindprod.com.invalid> wrote: > On Sat, 1 May 2010 04:34:31 -0700 (PDT), Joshua Maurice > <joshuamaur...(a)gmail.com> wrote, quoted or indirectly quoted someone > who said : > > >system: A developer should be able to do any combination of the > >following actions and trigger a small / minimal / incremental build, > > IF you use ANT, you don't need to bother with this. The time in a > traditional compile is mostly loading Javac.exe. With ANT it gets > compiled only once. Further JAVAC looks at dates of *.java and > *.class files and avoids most unnecessary recompilation. Did you even read any of my other posts in this thread? Ant's incremental compile is woefully incorrect, so incorrect as to be near useless on an automated build machine. As a developer, I would rather take the extra 10 min - 1.5 hours to do a full clean build to not have to debug bizarre obscure issues which result from a clean build. There's nothing quite like debugging a system in which you have inconsistent dlls / jars for a day straight; it's quite aggravating. > Compiling is almost inconsequential. Building Jars and Zips takes > much more of the time. Do you actually have timing numbers for any of this? I rewrote an ant / make system which loads Sun's tools.jar and invokes javac through the tools.jar Java interface, thus I loaded javac into memory just once like Ant. The full clean compilation of a small portion of my product, ~3000 files, took ~3 minutes, whereas a separate build invocation to produce the jars from no jars took ~15 seconds (5-8 sec of which is just reading in the build script files aka makefiles, stat- ing files, checking dependencies, etc.). It seems that the conventional wisdom is quite wrong here. It seems that making jars is actually quite quick. Well, it's at least quick if you turn off compression with the "0" flag to jar, as you should during development. |