A case where conventional wisdom may not apply
in [C++]
|
From: Joshua Maurice on 18 Jun 2010 00:22 { this discussion seems rapidly departing from C++ *language* toward software engineering, which is more suited for 'comp.software-eng'. please either bring it back, or wrap it up. thanks. -mod } On Jun 17, 11:54 am, Peter Olcott <NoS...(a)OCR4Screen.com> wrote: > Conventional wisdom says to get the code working and then profile it and > then make those parts that are too slow faster. > > What if the original goal is to make the code in the ball park of as > fast as possible? > > For example how fast is fast enough for a compiler? In the case of the > time it takes for a compiler to compile, any reduction in time is a > reduction of wasted time. > > *** NOTE *** > Since we are only shooting for the ballpark of the fastest code, we only > focus on those things that can have a big impact on speed. In other > words the fastest basic design where the code is extremely readable. > Readability is the first priority and speed is the secondary priority. > > I can see where this would take more time to get the code working. It > also seems (from my many years of doing this) to result in high quality > fast code in much less time than the conventional wisdom method. > > It results in high quality fast code in less steps than the conventional > wisdom method because it is much faster to improve a design than it is > to improve code. Design is at a much higher level of abstraction than > code so there is much less to change. Because there is much less to > change it takes much less time to improve a design than it does to > improve code. > > *** NOTE *** All of the above only applies to the case where the > original design goal is to make the code in the ballpark of as fast as > possible. It may be as much as the majority of the time where the > benefit of extra speed is not worth the additional development costs. I wrote up my own thoughts on this on the WikiWiki a while back. There's so many competing "axioms" and maxims like "you aren't going to need it", "keep it simple, stupid", "rules of optimization: 1- don't do it, don't do it yet, experts only - profile first". I think all of these things are just pushback against the practices seen in certain contexts and cultures. IMHO, they fail to capture what is really needed, and they fail to capture the actual reasoning which should be used. Here is my own thoughts on the subject. Axiom: The developer is out to make the company money. (Sure other "moral" axioms are involved like "don't commit murder", but we all understand what I mean when our primary goal is to make the company money.) This is the end goal of writing code professionally. We can then derive some intermediate goals: - Write maintainable code for future developer productivity gains. - Write code which compiles quickly for future developer productivity gains. - Write code which executes its task quickly to be attractive to customers. - Write robust and correct code to be attractive to customers. And more. When you are about to write code, you can do "big design up front", "agile aka hack it as you go and refactor quickly", etc. I think most of these are just reactionary to certain bad programmer practices. This is especially true for most developer's understanding of these things. They become mantras and maxims. They lose their original intent. In short, when writing code, there's generally a tradeoff between maintainable and extensible code vs fast executing code. Some people focus too much on writing fast executing code. I have a fellow colleague who took a couple hours to time the system clock() call to see if it could be a performance bottleneck in a function which allocates megabytes of data per call (a dynamic memory allocator). Most people recognize this is overkill and one shouldn't worry about such things until a profiler tells use we need to. The opposite end is people who would try to implement a hard number cruncher in Ruby. We recognize it just as folly. As an example, let's consider some programmers working on implementing a GNU Make clone. They have a couple of distinct options when they first start: 1- Just whip up some working example that covers some portion of use cases. 2- Big design up front for the entire system. - Or more likely 3- Somewhere in the middle. Presuming that they went with 2 or 3, they then have another option: A- Write the task execution engine to be single threaded from the start. B- Write the task execution engine to be multi-threaded from the start. C- Write the task execution engine to be distributed from the start. In this case, there are definite costs associated with implementing each solution, where A is easier than B, and B is much easier than C. However, C will outperform B at runtime, and B will outperform A at runtime. Without knowing more about the specific problem, the company, the company's goals, the company's time horizon, the economy, etc., something, a developer cannot make an informed decision between A, B, and C. Every such question as the OP's basically boils down to the question "A, B, or C?". It's the question of "Should I write a fast framework from the beginning?", or "Should I write my code in such a way that the suspected hotspots can be optimized later if need be?". Specifically, when writing a piece of code, one must always ask: 1- Is there a better way to do it? 2- How much more expensive is it (in developer time) to do the better way now? 3- How much "better" execution time (or other metric of the end product) will the better way give? And what's the likelyhood of a customer needing this extra power? How important is this better execution metric to the company and to its customers? 4- How much "better" maintainable and extensible will the code be if written the better way? Aka how much time developer time will be saved down the road? Which depends on how likely is this extensibility to be used in the future? And how important are the features which will use this extensibility? 5- *Important* Technical debt - how much harder would it be to implement later if we did the quicker way now? 6- *Important* What is the company's time horizon? Even if the product would be 10 times better with this feature, if it causes the product to slip the company could lose marketshare and go out of business. 7- *Important* Opportunity cost - can your time be better spent elsewhere on the product (or on another product of the company)? For my example of measuring the runtime of clock() in a function that allocates 100 MB, the answer is obvious: don't worry about it. For my example of GNU Make, the answer is not immediately clear. When the answer is not clear, you, or your product manager, or your team, etc., have to make a educated best guess, but for the love of -- something, don't just repeat a maxim like "keep it simple stupid" without understanding and making clear the assumptions behind such a maxim. PS: My own maxims, purely an educated guess on fact and unsupported by hard scientific studies, are: As a principle but not a hard rule, designs should be kept modular and decoupled. Micro optimizations which can be contained behind a single interface should not be pursued until a profiler tells you to. (Ex: Peter Olcott's hackery with his UTF8 finite state machine.) Doing the "quicker to implement" solution won't accrue any "technical debt", so it's much more likely in your situation for opportunity cost to override doing the "faster execution" code now, though again it depends on the particulars of the situation, aka questions 1-7. Macro optimizations that cross modules, that affect interfaces and the architecture, should be considered carefully up front. (Ex: single threaded vs multi threaded vs distributed.) Here, doing the "quicker to implement" solution will much more likely accrue a large amount of "technical debt", so it's more likely that the "right" decision is to do the "longer to implement" but "faster executing" code now, though again it depends on the particulars of the situation, aka questions 1-7. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Peter Olcott on 18 Jun 2010 00:24 On 6/17/2010 8:13 PM, Nick Hounsome wrote: > On 17 June, 19:54, Peter Olcott<NoS...(a)OCR4Screen.com> wrote: >> Conventional wisdom says to get the code working and then profile it and >> then make those parts that are too slow faster. >> >> What if the original goal is to make the code in the ball park of as >> fast as possible? > > At what cost? It may cost as much as 50% more than just getting the code working. Very often this also results in code that is essentially simpler, thus easier to maintain. > >> For example how fast is fast enough for a compiler? In the case of the >> time it takes for a compiler to compile, any reduction in time is a >> reduction of wasted time. >> >> *** NOTE *** >> Since we are only shooting for the ballpark of the fastest code, we only >> focus on those things that can have a big impact on speed. > > How do we know where the ballpark is? > > How do we know "those things" without profiling? If we're going to try > to find them by analysis then how much time and money do we spend and > how do we know that we've analyzed the the right things? > I just keep analyzing until I know that the process can not be accomplished in fewer steps. Minimizing the instructions in the execution trace. I don't strive to do this perfectly. >> In other >> words the fastest basic design where the code is extremely readable. >> Readability is the first priority and speed is the secondary priority. > > If the code is extremely readable then most likely it is either not > the fastest design or it is the design we would have come up with > anyway without being hung up on performance. It has been my experience that when performance is ignored, performance suffers. A major system that I worked on recently suffered a 30-fold degradation in performance versus this same system's prior performance specifically because the performance impact of major redesign choices was ignored. > >> I can see where this would take more time to get the code working. > > Why? If it is extremely readable then there should be few bugs. Paying constant attention to performance tends to take more initial development time than ignoring performance considerations. > It might well take more time to DESIGN but that is the problem - How > long do you spend looking for the fastest design (without profiling)? Far less time that it would take to implement enough of the design so that it could be profiled. > >> It >> also seems (from my many years of doing this) to result in high quality >> fast code in much less time than the conventional wisdom method. >> It results in high quality fast code in less steps than the conventional >> wisdom method because it is much faster to improve a design than it is >> to improve code. > > But it is much easier to analyse and test code than design. How could this be true? This would be like saying it is faster to read a 500 page book, than it is to read the synopsis of this same book. It has been my experience that it is orders of magnitudes faster to analyze a design than it is to analyze code that implements this same design. > >> Design is at a much higher level of abstraction than >> code so there is much less to change. Because there is much less to >> change it takes much less time to improve a design than it does to >> improve code. >> >> *** NOTE *** All of the above only applies to the case where the >> original design goal is to make the code in the ballpark of as fast as >> possible. It may be as much as the majority of the time where the >> benefit of extra speed is not worth the additional development costs. > > When has anyone EVER said to you "Make this as fast as possible and > damn the cost and delivery date"? > It's never happened to me in over 25 years. > Not as fast as possible. In the ballpark (within at least an order of magnitude) of as fast as possible. Also even this goal only applies to things such as compilers and operating systems. > In the real world the first priority is the delivery date every time - > even to the point that most places I've worked would rather send a > customer a product that doesn't meet REAL performance requirements > (not just "as fast as possible") than slip the delivery date. > -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Ulrich Eckhardt on 18 Jun 2010 00:29 Paul Bibbings wrote: > From your beginning I would have supposed you to go on to discuss > issues that relate to *execution* time. He does. > However... [...] > ... you then bring in *build* time This is the execution time of an exemple program, a compiler. > , and then ... [...] > ... we're suddenly talking about *development* time. Yes, development time is the cost, execution time is the result. If I understand him correctly, the question he wants to discuss is what approach leads to the best result/cost ratio, under the premise that you need high performance for the resulting program. Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: Pete Becker on 18 Jun 2010 00:32 On 2010-06-17 15:15:16 -0400, Peter Olcott said: > On 6/17/2010 5:40 PM, Bo Persson wrote: >> Peter Olcott wrote: >>> Conventional wisdom says to get the code working and then profile >>> it and then make those parts that are too slow faster. >>> >>> What if the original goal is to make the code in the ball park of as >>> fast as possible? >>> >>> For example how fast is fast enough for a compiler? In the case of >>> the time it takes for a compiler to compile, any reduction in time >>> is a reduction of wasted time. >>> >> >> Yes, but you still have to find out where it spends most of its time. >> Is it disk I/O? Parsing the source? Symbol table management? >> Instantiating templates? Something els�? >> > > I just make sure that it does not spend much more time than necessary > anywhere. The danger is that you can end up spending far more of your own time than necessary because you're optimizing things that have no significant effect on performance. This isn't the olden days when it made sense to spend a great deal of programmer time at $5 per hour to reduce CPU time that cost $200 per minute (IBM 7094 in 1970, if I remember correctly). CPU time is cheap and programmer time is not. -- Pete Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of "The Standard C++ Library Extensions: a Tutorial and Reference (www.petebecker.com/tr1book) [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: bf on 18 Jun 2010 00:30
On Jun 18, 12:40 am, "Bo Persson" <b...(a)gmb.dk> wrote: > It doesn't help a bit if your parsing is lightning fast, if that just > means the compiler has to wait longer for the next I/O operation to > complete. This is perfectly true when the only important thing is the time spend by that program for its task. However, most programs run in an environment together with lots of other programs. Every resource consumed by one program is a resource that is unavailable for others. So, if the program consumes every available CPU-cycle to just barely avoid waiting for any I/O operation, the time needed for that program to complete its task is the same as if it was blazingly fast in its computation but had to wait a lot. However, the latter program will make the system as a whole remain snappy, while the former will make the system sluggish. I know which one I'd prefer to run on my system. As usual, there's a trade off. It's almost always a waste to spend months squeezing that last cycle out of the program, especially if it's waiting for I/O. On the other hand, if you know that your program uses a large data set, selecting a good algorithm and cache-friendly data representation from the beginning may be a very good choice indeed. _ /Bjorn -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] |