Index matrix to sparse matrix... saving memory
in [Matlab]
|
From: Bruno Luong on 10 Jun 2010 15:18 "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <hurcjs$h31$1(a)fred.mathworks.com>... > Bruno or anyone that has looked through the thread, > > I'm beating myself up here... I just can't see through to the end. Histcn returns the same thing Histc does is return one output "loc" rather than multiple outputs of "bin." If I cat the bins > > [bin1 bin2 bin3] == [ loc ] > > Where I hit a wall is making use of loc. Loc has all of my bin locations, which I need and it does it very quickly, but I need to relate each column of "loc" to one another, compare every combination of bin value. This iterative comparison is what slows me down, and what I have been stuck on conceptually, which was what originally slowed down my understanding of your function histcn. > > Does anyone know of a function that will compress these for loop after I have used histcn to find the bin values? The for loops step through the bin values of each column of "loc" returning the correct index values of "loc". I have show you on other thread (where it should belong the few last posts) the use of ACCUMARRAY after HISTC. In this thread, the ACCUMARRAY does carry out the sum (because that was originally asked), but you can adapt it to your specific need, including find all the indexes that belong to the n-dimensional bins. Again, that was answered in Apr 27th, 2010. Bruno
From: Bruno Luong on 10 Jun 2010 15:38 I wrote: > > I have show you on other thread (where it should belong the few last posts) the use of ACCUMARRAY after HISTCN ... Sorry, should read "HISTCN" Bruno
From: Anthony Hopf on 10 Jun 2010 15:57 I understood, and appreciate your help. accumarray it is!! I'll post my solution when I figure this function out. Anthony "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <hureus$mfm$1(a)fred.mathworks.com>... > I wrote: > > > > I have show you on other thread (where it should belong the few last posts) the use of ACCUMARRAY after HISTCN ... > > Sorry, should read "HISTCN" > > Bruno
From: Anthony Hopf on 13 Jun 2010 10:45 Bruno... I can't believe I missed the funcitonality of histcn and accumarray!! This definitely gets me closer to my ultimate result. The below countermatrix returns the number of points within each bin given by histcn... This works extremely fast even for very large 3d matrices. I ran into a bit of a problem with "out of range" points, but dealt with these by tossing them in outer bins and taking them off of the countermatrix in the final steps. (code is at the bottom of this post). This matrix isn't exactly what I was looking for though... 1. I'm curious if there is a way to collect the index values, corresponding to the original val3d matrix, into bin vectors. What I would like is a vector of index values for each point within countermatrix, at this point the structure could be arbitrary but optimally it could be indexed by each bin value... cell arrays of cell array, rbincell{tbincell{pbincell{[idx_vals for countermatrix(r,t,p)]}}} (This is my first time working with the ideas of cells). Keeping the index values accumulated seems doable with accumarray because it "touches" each one of these values... the array being operated on by accumarray would just be the index values corresponding to the points within val3d, rather than val3d itself 2. Given the above I would also like to fill the original matrix val3d with the accumulated results, for example if val3d(i,j,k) fell into the sum corresponding to countermatrix(r,t,p), then we would put that sum in countermatrix(r,t,p) back into val3d(i,j,k), and all other points that where used for that particular sum. I do this iteratively now after finding the matrix mentioned in 1 above... but again this is a for loop that I want to get rid of. here is the code that creates countermatrix, in this case val3d is an array of ones and we should find that count and countermatrix are equal: %variables x = 1e3.*(1:.1:2); y = 1e3.*(0:.1:1); z = 1e3.*(0:.1:1); [x3d,y3d,z3d] = meshgrid(x,y,z); [t3d,p3d,r3d] = cart2sph(x3d,y3d,z3d); %vector to accumulate from val3d = ones(size(x3d)); %maximum values of the bins maxt = max(t3d(:)); maxp = max(p3d(:))-(pi/180); maxr = max(r3d(:)); %bins tbin = linspace(0,maxt,6); pbin = linspace(0,maxp,6); rbin = linspace(1e3,maxr,6); %histogram data = [r3d(:) t3d(:) p3d(:)]; [count,edges,mid,loc] = histcn(data,rbin,tbin,pbin); % temp = loc(:,1);temp(temp ==0)=max(temp)+1;loc(:,1) = temp; %place "out of range" values at the max bin for each rbin,tbin,pbin nir = zeros(1,3); for oor = 1:3 temp = loc(:,oor); if any(temp==0) %if there are out of range values temp(temp ==0)=max(temp)+1; loc(:,oor) = temp; clear temp nir(oor)=1; end end countermatrix_accum = accumarray(loc,val3d(:)); %remove "out of range" accumulated bins if nir(1) countermatrix_accum = countermatrix_accum(1:end-1,:,:); end if nir(2) countermatrix_accum = countermatrix_accum(:,1:end-1,:); end if nir(3) countermatrix_accum = countermatrix_accum(:,:,1:end-1); end isequal(count,countermatrix_accum) "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <hurg2g$5kt$1(a)fred.mathworks.com>... > I understood, and appreciate your help. accumarray it is!! > > I'll post my solution when I figure this function out. > > Anthony > > "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <hureus$mfm$1(a)fred.mathworks.com>... > > I wrote: > > > > > > I have show you on other thread (where it should belong the few last posts) the use of ACCUMARRAY after HISTCN ... > > > > Sorry, should read "HISTCN" > > > > Bruno
From: Anthony Hopf on 13 Jun 2010 21:17
Thank you very much Bruno, and others, for giving your advise... I had all of the resources to accomplish number 2... haven't figured out number 1 yet. Here is the code I am working with now, in case you are curious, i'm sure it isn't optimized: %variables x = 1e3.*(1:.1:2); y = 1e3.*(0:.1:1); z = 1e3.*(0:.1:1); [x3d,y3d,z3d] = meshgrid(x,y,z); [t3d,p3d,r3d] = cart2sph(x3d,y3d,z3d); %vector to accumulate from val3d = ones(size(x3d)); %maximum values of the bins maxt = max(t3d(:)); maxp = max(p3d(:))-(pi/180); maxr = max(r3d(:)); %bins tbin = linspace(0,maxt,6); pbin = linspace(0,maxp,6); rbin = linspace(1e3,maxr,6); %histogram data = [r3d(:) t3d(:) p3d(:)]; [count,edges,mid,loc] = histcn(data,rbin,tbin,pbin); %place "out of range" values at the max bin for each rbin,tbin,pbin nir = zeros(1,3); for oor = 1:3 temp = loc(:,oor); if any(temp==0) %if there are out of range values temp(temp ==0)=max(temp)+1; loc(:,oor) = temp; clear temp nir(oor)=1; end end countermatrix_accum = accumarray(loc,val3d(:)); %set countermatrix "out of range" values to zero if nir(1) countermatrix_accum(end,:,:) = 0; end if nir(2) countermatrix_accum(:,end,:) = 0; end if nir(3) countermatrix_accum(:,:,end) = 0; end %g will be the number of points at each common grid point gidx = sub2ind(size(countermatrix_accum),loc(:,1),loc(:,2),loc(:,3)); g = countermatrix_accum(gidx); %reshape g to get our matrix newval3d = reshape(g,size(val3d)); "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <hv2qti$j1l$1(a)fred.mathworks.com>... > Bruno... I can't believe I missed the funcitonality of histcn and accumarray!! This definitely gets me closer to my ultimate result. > > The below countermatrix returns the number of points within each bin given by histcn... This works extremely fast even for very large 3d matrices. I ran into a bit of a problem with "out of range" points, but dealt with these by tossing them in outer bins and taking them off of the countermatrix in the final steps. (code is at the bottom of this post). > > This matrix isn't exactly what I was looking for though... > > 1. I'm curious if there is a way to collect the index values, corresponding to the original val3d matrix, into bin vectors. What I would like is a vector of index values for each point within countermatrix, at this point the structure could be arbitrary but optimally it could be indexed by each bin value... cell arrays of cell array, rbincell{tbincell{pbincell{[idx_vals for countermatrix(r,t,p)]}}} (This is my first time working with the ideas of cells). Keeping the index values accumulated seems doable with accumarray because it "touches" each one of these values... the array being operated on by accumarray would just be the index values corresponding to the points within val3d, rather than val3d itself > > 2. Given the above I would also like to fill the original matrix val3d with the accumulated results, for example if val3d(i,j,k) fell into the sum corresponding to countermatrix(r,t,p), then we would put that sum in countermatrix(r,t,p) back into val3d(i,j,k), and all other points that where used for that particular sum. I do this iteratively now after finding the matrix mentioned in 1 above... but again this is a for loop that I want to get rid of. > > here is the code that creates countermatrix, in this case val3d is an array of ones and we should find that count and countermatrix are equal: > > %variables > x = 1e3.*(1:.1:2); > y = 1e3.*(0:.1:1); > z = 1e3.*(0:.1:1); > > [x3d,y3d,z3d] = meshgrid(x,y,z); > [t3d,p3d,r3d] = cart2sph(x3d,y3d,z3d); > %vector to accumulate from > val3d = ones(size(x3d)); > > %maximum values of the bins > maxt = max(t3d(:)); > maxp = max(p3d(:))-(pi/180); > maxr = max(r3d(:)); > %bins > tbin = linspace(0,maxt,6); > pbin = linspace(0,maxp,6); > rbin = linspace(1e3,maxr,6); > %histogram > data = [r3d(:) t3d(:) p3d(:)]; > > [count,edges,mid,loc] = histcn(data,rbin,tbin,pbin); > > % temp = loc(:,1);temp(temp ==0)=max(temp)+1;loc(:,1) = temp; > > %place "out of range" values at the max bin for each rbin,tbin,pbin > nir = zeros(1,3); > for oor = 1:3 > temp = loc(:,oor); > if any(temp==0) %if there are out of range values > temp(temp ==0)=max(temp)+1; > loc(:,oor) = temp; > clear temp > nir(oor)=1; > end > end > > countermatrix_accum = accumarray(loc,val3d(:)); > > %remove "out of range" accumulated bins > if nir(1) > countermatrix_accum = countermatrix_accum(1:end-1,:,:); > end > if nir(2) > countermatrix_accum = countermatrix_accum(:,1:end-1,:); > end > if nir(3) > countermatrix_accum = countermatrix_accum(:,:,1:end-1); > end > > isequal(count,countermatrix_accum) > > > "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <hurg2g$5kt$1(a)fred.mathworks.com>... > > I understood, and appreciate your help. accumarray it is!! > > > > I'll post my solution when I figure this function out. > > > > Anthony > > > > "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <hureus$mfm$1(a)fred.mathworks.com>... > > > I wrote: > > > > > > > > I have show you on other thread (where it should belong the few last posts) the use of ACCUMARRAY after HISTCN ... > > > > > > Sorry, should read "HISTCN" > > > > > > Bruno |