Why so little parallelism?
|
From: Anne & Lynn Wheeler on 12 Dec 2006 19:19 eugene(a)cse.ucsc.edu (Eugene Miya) writes: > Well I would not give Nick credit. This idea went back to the IBM TF-1 > and people thought for 3 orders of magnitude they would take the "power" > for no languages. They'd be willing to hand machine code. They didn't > buy into it and the TF-1 was never completed. Sure the GF-11 was, > barely and it ran 8 GFs. Nor was a big RP3 ever made. the major funding organizaton for RP3 asked my wife to do a technical audit of the project ... her basic conclusion was that they shouldn't continue to fund it (which they stopped doing) ... not too long after that, it died. that may have contributed to http://www.garlic.com/~lynn/2006w.html#21 SNA/VTAM for NSFNET and http://www.garlic.com/~lynn/2006w.html#20 cluster-in-a-rack actually some of the goings on mentioned in "SNA/VTAM for NSFNET" happened before my wife audited RP3 and recommended terminating RP3 funding.
From: Jan Vorbrüggen on 13 Dec 2006 06:43 > In the case of simple array operations, a run-time system is likely to > code-generate better than a programmer. Where it falls down is in > deciding how to distribute the array - and that is an unsolved problem, > whether it be done manually or automatically, despite many attempts > at systematising the issue. I think that for each individual "task" (array expression etc) you could fairly easily define the best data distribution. But then you end up with a sequence of incompatible data distributions once you put all those tasks together into an application. Now it turns difficult, because you have to reconcile all those different distrubutions either into a single one - which is likely suboptimal for each individual task, but yields a reasonable solution to the whole problem - or you even add tasks which do nothing but reorganize the data: for instance, see the recurrent discussion on comp.lang.fortran whether it isn't sometimes advantageous, from a perfomance- point of view, to let the compiler perform a gather-scatter on an array slice, which you usually try to avoid. Jan
From: Jan Vorbrüggen on 13 Dec 2006 06:48 > My point is that this is the exactly wrong way to go about it. Rather than > hope the compiler will do the right thing, you should be able to write the > code in such a way that the compiler understands that it is expected to > parallelize the loop in a particular way (or better, if that can be defined > "objectively") and that it's a bug in the source code if it can't. Then I don't need a compiler any more. And I will end up with brittle code, i.e., code that works well on one incarnation of a certain system and be abysmal on a (perhaps only slightly) different one. I'm all for giving my tools a helping hand in providing as much information to them as possible, in the form of annotations or whatever. What I don't want to do is to make premature decisions that are not required: and serialising an array expression is an example for such a premature decision, because there might be a different execution of that expression that is better for the system in hand. Jan
From: Nick Maclaren on 13 Dec 2006 07:07 In article <4ua79oF17k2hbU1(a)mid.individual.net>, =?ISO-8859-1?Q?Jan_Vorbr=FCggen?= <jvorbrueggen(a)not-mediasec.de> writes: |> > In the case of simple array operations, a run-time system is likely to |> > code-generate better than a programmer. Where it falls down is in |> > deciding how to distribute the array - and that is an unsolved problem, |> > whether it be done manually or automatically, despite many attempts |> > at systematising the issue. |> |> I think that for each individual "task" (array expression etc) you could |> fairly easily define the best data distribution. But then you end up with a |> sequence of incompatible data distributions once you put all those tasks |> together into an application. ... There are a fair number of tasks that are hard in themselves, but you are right that even applications built up out of simple tasks aren't necessarily simple. Think Dirichlet tesselation for a simple example of a foul task for splitting; unless you know a fair amount about the data distribution, you will always have problems. Regards, Nick Maclaren.
From: Eugene Miya on 15 Dec 2006 11:49
>> barely and it ran 8 GFs. Nor was a big RP3 ever made. In article <m3irggwsuj.fsf(a)garlic.com>, Anne & Lynn Wheeler <lynn(a)garlic.com> wrote: >the major funding organizaton for RP3 asked my wife to do a technical >audit of the project ... her basic conclusion was that they shouldn't >continue to fund it (which they stopped doing) ... not too long after >that, it died. Sure. Your major funding organization makes business machines, not research machines. It merely happens that it has 3 things it chose to name Research Centers and a number of lesser "Science" centers and similarly acronymed facilities. >that may have contributed to >http://www.garlic.com/~lynn/2006w.html#21 SNA/VTAM for NSFNET I would not have used the word "contributed". >http://www.garlic.com/~lynn/2006w.html#20 cluster-in-a-rack I know it's sitting at the Museum. >actually some of the goings on mentioned in "SNA/VTAM for NSFNET" >happened before my wife audited RP3 and recommended terminating RP3 >funding. I sat in on Greg Pfister's presentation at SJ RC (with numerous people outside IBM like DEC). Sounded like an interesting case study in O(N lg n) interconnection architectures like the NYU Ultracomputer. Then one of your employees asked the question about whether the RP3 would be instruction set compatible with the 370-line (he clearly had not been listening to the earlier CPU portion of the talk but also aligned with the pre-PC mainframe mind set). But that's history. Later when we were approached to buy one, gee that's over 20 years ago, and unlike most NDAs that I've signed, IBM is still in existence. Anyways, it was your choice. You guys don't have research in your names. Other than Greg's book, I only hope we can preserve history, as opposed to economic revisionism. What's left of the hardware, what ever is left except for the cited RIOS, is on the East Coast. -- |