Prev: Speaking of thread safety?
Next: solutions manual to A First Course In Probability 7th Edition by Sheldon M. Ross
From: Patricia Shanahan on 18 Mar 2010 21:22 Arne Vajh�j wrote: .... > What is important is the number of bugs possible to introduce. It's not so much the number of bugs as their total cost. The cost of a bug has two components - the cost of fixing it, and the cost of whatever harm it does before being fixed. > Multithreaded code often result in concurrency problems, because > there are so many ways to create problems that some problems will > get created. > > And those problems: > - has a higher chance of slipping through code review and test > - can be very difficult to troubleshoot > - can have distasterous impact > > I hate those bugs! > I regard them as very expensive bugs, precisely because intermittent, timing dependent bugs have an unusually high chance of getting through testing. Patricia
From: Arne Vajhøj on 18 Mar 2010 22:01 On 18-03-2010 21:49, Stefan Ram wrote: > Patricia Shanahan<pats(a)acm.org> writes: >>> Multithreaded code often result in concurrency problems, because >> I regard them as very expensive bugs, precisely because intermittent, >> timing dependent bugs have an unusually high chance of getting through >> testing. > > �It is our basic belief that extreme caution is > warranted when designing and building multi-threaded > applications, particularly those which have a GUI > component. Use of threads can be very deceptive. In many > cases they appear to greatly simplify programming by > allowing design in terms of simple autonomous entities > focused on a single task. In fact in some cases they do > simplify design and coding. However, in almost all cases > they also make debugging, testing, and maintenance > vastly more difficult and sometimes impossible. Neither > the training, experience, or actual practices of most > programmers, nor the tools we have to help us, are > designed to cope with the non-determinism. For example, > thorough testing (which is always difficult) becomes > nearly impossible when bugs are timing dependent. This > is particularly true in Java where one program can run > on many different types of machines and OS platforms, > and where each program must work under both preemptive > or non-preemptive scheduling. > > As a result of these inherent difficulties, we urge you > to think twice about using threads in cases where they > are not absolutely necessary.� > > http://java.sun.com/products/jfc/tsc/articles/threads/threads1.html Threads can somewhat be avoided in desktop apps, but threads are usually unavoidable in server apps. Arne
From: Arved Sandstrom on 19 Mar 2010 16:56 Arne Vajh�j wrote: > On 18-03-2010 21:49, Stefan Ram wrote: >> Patricia Shanahan<pats(a)acm.org> writes: >>>> Multithreaded code often result in concurrency problems, because >>> I regard them as very expensive bugs, precisely because intermittent, >>> timing dependent bugs have an unusually high chance of getting through >>> testing. >> >> �It is our basic belief that extreme caution is >> warranted when designing and building multi-threaded >> applications, particularly those which have a GUI >> component. Use of threads can be very deceptive. In many >> cases they appear to greatly simplify programming by >> allowing design in terms of simple autonomous entities >> focused on a single task. In fact in some cases they do >> simplify design and coding. However, in almost all cases >> they also make debugging, testing, and maintenance >> vastly more difficult and sometimes impossible. Neither >> the training, experience, or actual practices of most >> programmers, nor the tools we have to help us, are >> designed to cope with the non-determinism. For example, >> thorough testing (which is always difficult) becomes >> nearly impossible when bugs are timing dependent. This >> is particularly true in Java where one program can run >> on many different types of machines and OS platforms, >> and where each program must work under both preemptive >> or non-preemptive scheduling. >> >> As a result of these inherent difficulties, we urge you >> to think twice about using threads in cases where they >> are not absolutely necessary.� >> >> http://java.sun.com/products/jfc/tsc/articles/threads/threads1.html > > Threads can somewhat be avoided in desktop apps, but threads > are usually unavoidable in server apps. > > Arne > And it's simply amazing how many of them work and how few bugs have anything to do with concurrency even though *everything* is running in a multi-threaded environment. At one client site where I have been for about a year and a half the developers, internal and external, have probably fixed over a thousand defects in that time. In two fairly good-sized J2EE apps. Not one - I repeat, not one - defect has had anything to do with concurrency issues. Good thing I didn't read that Sun blurb - I might have avoided working with J2EE at all. For that same client I wrote a moderately complex inherently multi-threaded socket server that went into production over a year ago. It has had no unscheduled downtime, and has had less than five defects logged against it, none of them serious, and none of them having anything to do with concurrency problems. Good thing I didn't read that Sun blurb first - I would have avoided java.util.concurrent classes like the plague, because none of them could possibly have worked well. While working for an employer back about 7 or 8 years ago I was in charge of the in-house servlet engine, which was partially written in C but mostly in Java. Plenty of multi-threading happening there. Given how incredibly difficult writing this kind of code is, according to Sun, I'm amazed at how few concurrency problems we had back then. We must have just lucked out, I guess. The stated intent of JCIP, and other books like it, is to arm developers with the information they need to write reliable and maintainable concurrent apps. Clearly a fair few people are able to do precisely that. So are the authors of JCIP, and all those programmers out there who have actually demonstrably written reliable and maintainable concurrent applications, all full of it? I don't think so. AHS
From: Mike Schilling on 19 Mar 2010 17:50 Arved Sandstrom wrote: > > For that same client I wrote a moderately complex inherently > multi-threaded socket server that went into production over a year > ago. It has had no unscheduled downtime, and has had less than five > defects logged against it, none of them serious, and none of them > having anything to do with concurrency problems. Good thing I didn't > read that Sun blurb first - I would have avoided java.util.concurrent > classes like the plague, because none of them could possibly have > worked well. That is, you're using classes designed, created, implemented and debugged by Sun that have been exercised so heavily that the commonly encountered bugs have ben found If you had to implement something like a ConcurrentLinkedQueue from scratch, do you think it would just work, or might there be subtle, difficult-to-detect bugs in it?
From: Arved Sandstrom on 20 Mar 2010 10:15
Mike Schilling wrote: > Arved Sandstrom wrote: >> For that same client I wrote a moderately complex inherently >> multi-threaded socket server that went into production over a year >> ago. It has had no unscheduled downtime, and has had less than five >> defects logged against it, none of them serious, and none of them >> having anything to do with concurrency problems. Good thing I didn't >> read that Sun blurb first - I would have avoided java.util.concurrent >> classes like the plague, because none of them could possibly have >> worked well. > > That is, you're using classes designed, created, implemented and debugged by > Sun that have been exercised so heavily that the commonly encountered bugs > have ben found If you had to implement something like a > ConcurrentLinkedQueue from scratch, do you think it would just work, or > might there be subtle, difficult-to-detect bugs in it? In this _particular_ case I chose classes designed, created, implemented and debugged by Sun, yes. Normally, now, I'd be inclined to make that same choice rather than create my own low-level stuff from scratch. As to the question of whether my own implementations of classes that are in java.util.concurrent would have subtle, difficult-to-detect bugs in them, I can't answer that in a way that would probably satisfy you. I could make the point that prior to using java.util.concurrent classes that, in one of the other projects I mentioned, that we were using strictly the lower-level Java concurrency facilities (the Object and Thread methods), and although there were bugs related to our concurrency implementation, the number and difficulty of them wasn't of such a nature as to make them stand out compared to defects encountered in other areas of the code. In fact it proved not to be all that tough to write a reliable server using the lower-level Java facilities. The more complicated logic there, through the life of the HTTP/S server/servlet engine, had to do with properly implementing HTTP and the servlet API. Not the concurrency. But we did have good programmers. That surely makes a difference. I've already stated that I wouldn't expect junior programmers to be writing good concurrency code. I wouldn't expect them to be writing particularly good code in any problem area, quite frankly. I might also add that in this same socket server that you called out, the one where I chose to use java.util.concurrent classes for the concurrency problem, the majority of the code has nothing whatsoever to do with concurrency. A majority of the most complicated code has to do with classloader management so as to allow hot-unloading/reloading of classes without stopping/starting/restarting the server. I wasn't familiar with OSGi at the time so I went that route. Point being, _that_ code is functioning very reliably as well, over 15 months now in high-volume 24/7 use, so maybe just maybe if I'd written that concurrency code using lower-level Java facilities I might have known what I was doing - since I've spent a lot of time doing professional development on the subject - and there wouldn't have been a whole bunch of subtle and difficult-to-detect concurrency defects. Perhaps I can rephrase my main argument. I'm of the belief that concurrency problems, when we consider their difficulty, fall into a spectrum just like problems in every other area of programming do - easy, moderately difficult, and difficult. I also believe that when concurrency problems are tackled by intermediate/senior developers who have a decent knowledge of concurrency programming that most of those problems are easy or moderately difficult...just like most of the non-concurrency problems. Let me ask you this, Mike. I would have to assume that you've done substantially the same reading I have (nature of, if not the same titles), and likely also have considerable experience in concurrent programming. Let's take Java concurrent programming as an example, because of the NG we're in, and examine the JCIP book specifically. After reading and re-reading that book, is there anything in it that leads you to believe that only a tiny elite of superstar developers can ever "get" concurrent programming in Java? AHS |