First class developer: who ?
in [Java Help]
|
Prev: Speaking of thread safety?
Next: solutions manual to A First Course In Probability 7th Edition by Sheldon M. Ross
From: BGB / cr88192 on 18 Mar 2010 11:25 "Mike Schilling" <mscottschilling(a)hotmail.com> wrote in message news:hnrt2o$tqb$1(a)news.eternal-september.org... > Arne Vajh�j wrote: <snip> >> >> My experience is that concurrency problems have a bad tendency to >> happen even with good programmers and concurrency problems have >> a very nasty characteristics of being very difficult to find. > > Very like memory leaks and memory corruption (in non-GC'd environments) in > both of those tendencies. > agreed. being a primarily C programmer, one can note that threads are not that much worse than dealing with memory, in this regard. like memory, one needs to find a good strategy to work with things (as naively doing whatever will turn out well, and some strategies work much better than others). actually, it is not that much worse in general than using other sorts of non-deterministic algos (where what exactly happens comes down to a matter of probability). determinism can make things a little easier in some cases, but limits the types of algos one can employ (there are some interesting and useful algos that don't really work in a deterministic setting). to some extent, I had adopted a variant of the Erlang-style approach to threads, which helps make threading a lot easier since then one has a fairly solid idea where threads will be (in terms of the code), and allows the "vast majority" of the code to be essentially done in a single-threaded style (although I generally shy away from shared global state, for a number of reasons). as it so happens, in my codebase the vast majority of this "single-threaded" code doesn't actually take place in the main thread. of note is that in general I haven't really had any notable concurrency related issues. however, if one just naively creates threads which can wander all over the place (they are not confined to a particular region of code and a specific task), then it is a lot easier to create a mess (since pretty much any reachable code could be potentially accessed by threads, and so effort would be needed to make it thread-safe). if the threads are confined, however, then most of the other code need not know or care that they exist.
From: Mike Schilling on 18 Mar 2010 11:38 Arved Sandstrom wrote: > Mike Schilling wrote: >> Arved Sandstrom wrote: >>> Mike Schilling wrote: >>>> Arved Sandstrom wrote: >>>>> I don't doubt that 80-90 percent of the people who currently work >>>>> as programmers couldn't competently write reliable concurrent >>>>> code, but then OTOH they can't write reliable code period, so >>>>> it's really got nothing to do with concurrency. A software >>>>> developer who can write high-quality code can write high-quality >>>>> concurrent code, and not have to agonize over it either. >>>> Concurrency leads to difficult and subtle problems, particualry as >>>> code evolves over the years. That makes it more difficult to write >>>> correctly and more fragile than code which does not need to be >>>> concurrent. >>> I don't believe for a second that concurrency _inherently_ leads to >>> difficult and subtle problems. I do believe that not understanding >>> concurrent programming leads to difficult and subtle problems. >> >> Then your intuition disagrees with what I've observed over decades. I'm >> afraid I won't defer to it. > > Nobody is asking you to defer to anyone else's observations. I'm > simply stating that I don't believe that concurrent programming is > very difficult. If you think that concurrent programming is > inherently very difficult then that's your prerogative. > >>> I'll agree that you probably don't want junior programmers writing >>> your multi-threaded code. But too many people make it sound like >>> writing concurrent code is insanely difficult. Well, no, it's not. >>> The execution of concurrent code is not a roll of the dice; it's as >>> well-behaved as any other code if you knew what you were doing when >>> you wrote it. >> >> Multithreaded code beahves different every time you run it. The >> same inputs can lead toi different output, which is bad enough, but >> in the real world you oten find quite different inputs. > > Wait, hold on - let's be clear on what you just said. "Same inputs can > lead to different output". And "multithreaded code behaves different > every time you run it". > > That, translated, means that the code isn't thread-safe. Not at all. Thread-safe code is code that's always correct even though it behaves differently every time. > >> It has problems like thread >> starvation that non-concurrent code does not have. It's inherently >> less portable than other Java code, because different platforms have >> different threading models; in particular, this can severely affect >> efficiency. >>> Is "Java Concurrency In Practice" a really hard read? No. Is there >>> such a huge amount of material in there that it's difficult to learn >>> what you really need to know? No. Similarly for .NET - is >>> "Concurrent Programming on Windows" (by Joe Duffy) such a hard >>> read? No. >> >> And? The JLS isn't particularly a hard read. That doesn't mean >> that Java programming is simple, regardless of the problem you're >> trying to solve. > > Are you finding it difficult, though? Leaving aside the specific > real-world problems, are you finding the implementations in Java to be > really hard work? I doubt that you are. > > In the same vein, do you really find concurrent programming to be > _very_ difficult, or simply deserving of some extra care and > attention just like you'd apply to the efficient and correct > implementation of a complex algorithm? I find it much more like the > latter, rather than that multi-threading suddenly makes everything > really, really hard. What I'm finding, as I said, is that multi-threaded code is more prone to subtle bugs and more fragile than single-threaded code.
From: Mike Schilling on 18 Mar 2010 11:41 Thomas Pornin wrote: > According to Mike Schilling <mscottschilling(a)hotmail.com>: >> Very like memory leaks and memory corruption (in non-GC'd >> environments) in both of those tendencies. > > That's the point, actually. Tremendous design efforts went into Java > to make many aspects of programming easier. In particular, the GC is > an automatic system to solve the problem of memory allocation, problem > which proved to be very difficult to tackle by most programmers, even > good programmers, regardless of how "simple" it may look from an > overly theoretical point of view. > > The same amount of efforts did not went (yet) into concurrency. Right > now, Java has some primitives ('synchronized' and wait()/notify(), > mostly), which are quite low-level. They are much better than nothing > at all, but I would still classify them at the same "level" than > malloc()/free() with regards to memory allocation. In that view, > Java's support for concurrency is still very "C". You may observe, for > instance, that if you write a multi-threaded application in C, it will > use, for the concurrency part, the same structure than a similar Java > application (you replace synchronized blocks with pthread_mutex_lock() > or EnterCriticalSection(), and wait()/notify() with conditions or > semaphores). Java also has a well-defined memory model for communicating across threads with shared memory. Once again, that's low-level and more a building b,ock than a solution, but once again it's more than C has.
From: Tom Anderson on 18 Mar 2010 17:54 On Wed, 17 Mar 2010, Mike Schilling wrote: > Tom Anderson wrote: >> On Tue, 16 Mar 2010, Arne Vajh?j wrote: >> >>> So a 50 MB log4j log file from each cluster member. >>> >>> :-( >> >> That reminds me: i had a 3.1 GB error log file then other day. It >> filled up a partition and stopped everything working. >> >> I have no idea why the app did that. I just deleted the file and >> restarted. Seemed to be fine. > > You didn't examine the file to see which messages were being logged > repeatedly? Nope. Had other fish to fry. If it does it again, i'll take a look at it. I'd guess that during development, something was badly broken (as is often the case), and left like that overnight, so the app just kept pouring stuff into the log file. tom -- I need a proper outlet for my tendency towards analytical thought. -- Geneva Melzack
From: Arne Vajhøj on 18 Mar 2010 21:02
On 18-03-2010 06:02, Arved Sandstrom wrote: > Arne Vajh�j wrote: >> On 17-03-2010 18:35, Arved Sandstrom wrote: >>> Mike Schilling wrote: >>>> Arved Sandstrom wrote: >>>>> I don't doubt that 80-90 percent of the people who currently work as >>>>> programmers couldn't competently write reliable concurrent code, but >>>>> then OTOH they can't write reliable code period, so it's really got >>>>> nothing to do with concurrency. A software developer who can write >>>>> high-quality code can write high-quality concurrent code, and not have >>>>> to agonize over it either. >>>> >>>> Concurrency leads to difficult and subtle problems, particualry as >>>> code evolves over the years. That makes it more difficult to write >>>> correctly and more fragile than code which does not need to be >>>> concurrent. >>> >>> I don't believe for a second that concurrency _inherently_ leads to >>> difficult and subtle problems. I do believe that not understanding >>> concurrent programming leads to difficult and subtle problems. >>> >>> I'll agree that you probably don't want junior programmers writing your >>> multi-threaded code. But too many people make it sound like writing >>> concurrent code is insanely difficult. Well, no, it's not. The execution >>> of concurrent code is not a roll of the dice; it's as well-behaved as >>> any other code if you knew what you were doing when you wrote it. >>> >>> Is "Java Concurrency In Practice" a really hard read? No. Is there such >>> a huge amount of material in there that it's difficult to learn what you >>> really need to know? No. Similarly for .NET - is "Concurrent Programming >>> on Windows" (by Joe Duffy) such a hard read? No. >>> >>> I still believe that programmers who are finding that writing concurrent >>> code is really, really hard probably find most other kinds of coding >>> pretty difficult too. In my experience most programmers who are having >>> problems reasoning about what their code may or may not do in a >>> concurrent situation don't fully understand what their code is doing in >>> a single-threaded environment, nor do they adequately understand the >>> concurrency capabilities of their language. >> >> My experience is that concurrency problems have a bad tendency to >> happen even with good programmers and concurrency problems have >> a very nasty characteristics of being very difficult to find. > > That's suggestive language, Arne: "concurrency problems have a bad > tendency to happen". Well, perhaps they do for many programmers, > although it's then arguable as to whether they are good concurrency > programmers or simply good programmers except when it comes to concurrency. > > The reason I say it's suggestive is because it makes it sound like the > programmer didn't cause the concurrency problems. They "just" happened. > Well, no, they didn't just happen - a misunderstanding of how to write > concurrent code is what led to the programmer writing incorrect code. > > I will concede that if a programmer has written a dog's breakfast as far > as concurrency is concerned that it's then frequently a losing cause to > try to diagnose defects. Of course it is the programming writing code with a bug in. But whether it is possible to write guaranteed correct code is not very important for for real world software. What is important is the number of bugs possible to introduce. If there are lots of possibilities to introduce bugs, then some of them will be introduced. A normal development team has a few very good programmers, a bunch of OK programmers and a few very bad programmers. Any of those programmers has a few relative brilliant days, a bunch of OK days and a few horrible days. Programming language designs and development methodologies that assumes very good programmers not having bad days are useless. Because those assumptions are rarely met. Multithreaded code often result in concurrency problems, because there are so many ways to create problems that some problems will get created. And those problems: - has a higher chance of slipping through code review and test - can be very difficult to troubleshoot - can have distasterous impact I hate those bugs! Arne |