Parfor and non-deterministic iid variables
in [Matlab]
|
Prev: Color Image Enhancement Using Brightness Preserving Dynamic Histogram Equalization N
Next: Opening a mixed text data file and deleting the last x characters of every line
From: Floris Zoutman on 15 Apr 2010 05:04 I have a line of code that looks something like this: INCOME=some vector Nx1; weight=some vector Nx1; M=1000; alpha=zeros(M,1); xmin=zeros(M,1); for i=1:M % A random sample from INCOME with replacement and using the weights defined in vector weight. income=randsample(INCOME,N,true,'weights',weight); % A function calculating some statiscal properties of the data alpha and xmin [alpha(i) xmin(i)]=plfit(income); end I tried to speed up the process by using parfor instead of for, but received the following error message: "temporary variable income is non-deterministic" I totally agree with Matlab that income is non-deterministic, but I do not see why this is an issue. As far as I can see the values in income are independent over i. Therefore I would think that parallel computing of this loop should not be an issue. Does anybody have some suggestions on how to fix this issue? Many thanks in advance.
From: Rune Allnor on 15 Apr 2010 05:23 On 15 apr, 11:04, "Floris Zoutman" <fzout...(a)hotmail.com> wrote: > I have a line of code that looks something like this: > INCOME=some vector Nx1; > weight=some vector Nx1; > M=1000; > alpha=zeros(M,1); > xmin=zeros(M,1); > for i=1:M > % A random sample from INCOME with replacement and using the weights defined in vector weight. > income=randsample(INCOME,N,true,'weights',weight); > % A function calculating some statiscal properties of the data alpha and xmin > [alpha(i) xmin(i)]=plfit(income); > end > I tried to speed up the process by using parfor instead of for, but received the following error message: > "temporary variable income is non-deterministic" > I totally agree with Matlab that income is non-deterministic, but I do not see why this is an issue. Because 1) Writing parallel programs is hard at the best of times. 2) It's even harder when substantial parts of the program is not known at compile time (that is, when the matlab executable is compiled, not when the matlab script or function is interpreted). 3) Interpreter writers can not assume that users know what they are doing. > As far as I can see the values in income are independent over i. Therefore I would think that parallel computing of this loop should not be an issue.. *You* might know that, but whoever wrote the matlab interpreter can not trust that you, the user, knows what you are doing. In fact, he will have to assume that you do not. > Does anybody have some suggestions on how to fix this issue? Many thanks in advance. Pull the line out of the loop, and initialize income as an array before the loop. As far as I can tell, the allocation of income is the first thing that happens in that loop, so you will still gain from parallel computations on the elements of income. Rune
From: Floris Zoutman on 15 Apr 2010 05:33
Thanks, I should have thought of this solution myself. I will try it right away. Rune Allnor <allnor(a)tele.ntnu.no> wrote in message <4e9c933f-4ba4-4db7-995c-613cb72606d0(a)g30g2000yqc.googlegroups.com>... > On 15 apr, 11:04, "Floris Zoutman" <fzout...(a)hotmail.com> wrote: > > I have a line of code that looks something like this: > > INCOME=some vector Nx1; > > weight=some vector Nx1; > > M=1000; > > alpha=zeros(M,1); > > xmin=zeros(M,1); > > for i=1:M > > % A random sample from INCOME with replacement and using the weights defined in vector weight. > > income=randsample(INCOME,N,true,'weights',weight); > > % A function calculating some statiscal properties of the data alpha and xmin > > [alpha(i) xmin(i)]=plfit(income); > > end > > I tried to speed up the process by using parfor instead of for, but received the following error message: > > "temporary variable income is non-deterministic" > > I totally agree with Matlab that income is non-deterministic, but I do not see why this is an issue. > > Because > > 1) Writing parallel programs is hard at the best of times. > 2) It's even harder when substantial parts of the program is > not known at compile time (that is, when the matlab > executable is compiled, not when the matlab script or > function is interpreted). > 3) Interpreter writers can not assume that users know what they > are doing. > > > As far as I can see the values in income are independent over i. Therefore I would think that parallel computing of this loop should not be an issue. > > *You* might know that, but whoever wrote the matlab interpreter > can not trust that you, the user, knows what you are doing. In > fact, he will have to assume that you do not. > > > Does anybody have some suggestions on how to fix this issue? Many thanks in advance. > > Pull the line out of the loop, and initialize income as an array > before the loop. As far as I can tell, the allocation of income > is the first thing that happens in that loop, so you will still > gain from parallel computations on the elements of income. > > Rune |