Incremental Java Compile
in [Java Programming]
|
From: Lew on 3 May 2010 19:59 Joshua Maurice wrote: > Did you even read any of my other posts in this thread? Ant's > incremental compile is woefully incorrect, so incorrect as to be near > useless on an automated build machine. As a developer, I would rather That's a damn snarky tone to take with Roedy, who was just giving you good advice, especially considering how you keep blaming Ant, Java and everything else when it's clear from your own admission that it's your own process that's at fault, as you keep throwing back at us every time someone makes a useful suggestion. It's not the tools' fault, it's your'n. > take the extra 10 min - 1.5 hours to do a full clean build to not have > to debug bizarre obscure issues which result from a clean build. > There's nothing quite like debugging a system in which you have > inconsistent dlls / jars for a day straight; it's quite aggravating. So fix your system and quit whining about it. -- Lew
From: Joshua Maurice on 3 May 2010 20:18 On May 3, 4:59 pm, Lew <no...(a)lewscanon.com> wrote: > Joshua Maurice wrote: > > Did you even read any of my other posts in this thread? Ant's > > incremental compile is woefully incorrect, so incorrect as to be near > > useless on an automated build machine. As a developer, I would rather > > That's a damn snarky tone to take with Roedy, who was just giving you good > advice, especially considering how you keep blaming Ant, Java and everything > else when it's clear from your own admission that it's your own process that's > at fault, as you keep throwing back at us every time someone makes a useful > suggestion. It's snarky because of what I consider to be this near absurd level of deference given to the tools. If this was any other piece of software, and there was a product out there which ran 10x to 100x faster, it was be a no brainer conclusion which to use. Instead, I see far too many people say "Meh. Just do a clean build. It's not that bad." > It's not the tools' fault, it's your'n. No. If you read my posts, you would know that I blame both process and tool. The best the process could do is divide the build into more manageable chunks, but as a developer I would still have to spend an hour or so waiting on a build when most of the work is extraneous > > take the extra 10 min - 1.5 hours to do a full clean build to not have > > to debug bizarre obscure issues which result from a clean build. > > There's nothing quite like debugging a system in which you have > > inconsistent dlls / jars for a day straight; it's quite aggravating. > > So fix your system and quit whining about it. I am fixing it. I am not whining. I was asking for help on how to do it. I have asked for real solutions to the real problems I am facing writing it, such as how to get a list of class files per compiled java file as if I called javac once per java file in the dir.
From: Joshua Maurice on 3 May 2010 20:29 On May 3, 3:46 pm, "Mike Schilling" <mscottschill...(a)hotmail.com> wrote: > Joshua Maurice wrote: > > Currently ~145 min compile and package, no tests, on the automated > > build machine. 188 min more for the standard regression / acceptance / > > integration test suite. Some of the tests are currently distributed > > across several automated build machines, with the longest suite at 87 > > min. Double those times, or thereabouts, for a lower end developer > > computer. Any change requires a full clean build as we no > > incrementally correct build, and it has not been componentized into > > smaller chunks. For example, the serialization framework > > implementation changes slightly frequently, which affects a lot of the > > code, such as the file persistence, database persistence, engine, and > > GUI "components". > > Does the serialization framework change often? That would be horrific, and > there's probably nothing to be done to improve the build cost when it does. > But I presume that it changes as the result of some feature being added, so > that can be mitigated by not checking the change into source control until > the feature (or better yet, set of fesatures) is complete. I wish I knew. I just got an email today from the serialization team asking "What's with this error?" I "hacked" the C++ Maven plugin to report "<> has detected visual studios warning <>, deletion of a pointer to an incomplete type. This is formally undefined behavior according to the C++ spec. Fix it." Apparently changes are still ongoing. > Also, developers are usually good at optimizing their own work. If a > developer is adding new classes or changing implementation rather than > interface, there's no need to recompile the world. Even when changing > interfaces, the developer usually has a good idea of which bits of the > system use those interfaces, and can recompile just those parts. As such a developer, perhaps, but when I mess up, I break the mainline build, and because the build on the automated build machine, or private perforce branch build machine, can take the better part of a day, it's sometimes hard to isolate down who broke it, and especially when ML is broken this leaves people in a bind. Currently we lock ML on such events. Rollback is possible. Devops is floating that idea around at the moment. > Anyway, I'd suggest: > > 1. Invest in a good SCM system, one that handles multiple branches and > shared branches well. Done. Perforce is so awesome for the record. > 2. Encourage developers to stay isolated, rather than intergating often and > updating other developers' changes often. Sounds like integration hell. We do have separate teams working on their own little view for weeks or a month or two on end, and each team has their own private branch in perforce which is integrated roughly weekly with mainline. > 3. Do a continuous build that allows developers to grab the most recent > complete, tested code, so they can recompile only the code they have checked > out and the code that depends on it. Throw lots of hardware at this, so > that failures are found early. Also done. The problem is that it's not helping. It's way too much code, way too many tests, taking way too long to build.
From: Arne Vajhøj on 3 May 2010 20:33 On 03-05-2010 14:36, Joshua Maurice wrote: > On May 3, 9:28 am, Roedy Green<see_webs...(a)mindprod.com.invalid> > wrote: >> On Sat, 1 May 2010 04:34:31 -0700 (PDT), Joshua Maurice >> <joshuamaur...(a)gmail.com> wrote, quoted or indirectly quoted someone >> who said : >>> system: A developer should be able to do any combination of the >>> following actions and trigger a small / minimal / incremental build, >> >> IF you use ANT, you don't need to bother with this. The time in a >> traditional compile is mostly loading Javac.exe. With ANT it gets >> compiled only once. Further JAVAC looks at dates of *.java and >> *.class files and avoids most unnecessary recompilation. > > Did you even read any of my other posts in this thread? Most likely not. > Ant's > incremental compile is woefully incorrect, so incorrect as to be near > useless on an automated build machine. Ant is very useful for automated builds. But it is common practice to clean and rebuild. Your project structure is just not suited for this. Arne
From: Arne Vajhøj on 3 May 2010 20:38
On 03-05-2010 00:50, Joshua Maurice wrote: > On May 2, 6:27 pm, Arne Vajh�j<a...(a)vajhoej.dk> wrote: >> Structure things better. >> >> With a good OO model a series of related changes should not >> require changes in "a large portion of 20000 Java files". > > So, how do you suggest doing that when there's a code generator under > active development which generates Java code, and a large portion of > the Java code directly or indirectly works with the output of this > code generator? We model the object domains in a simple modeling > language which is then compiled to C++ and Java code to allow > serializing a description of a unit of work from the Java tools to the > C++ tools and back. Most of the infrastructure and apps work with the > output of this code generator in some form or another. > > Unfortunately, one cannot fiat interfaces into being stable. You are working on fixing the symptoms not the problem. Something is horrible wrong with the object model if so many classes change all the time. If you fix that problem (better requirements or more time spend designing before coding or whatever necessary), then you will be much better off. Arne |