Index matrix to sparse matrix... saving memory
in [Matlab]
|
From: Bruno Luong on 30 May 2010 08:02 "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <htthum$sg$1(a)fred.mathworks.com>... > > Bruno, > > I had downloaded your histcn function at one point to try and understand how it could be applied to my problem, but I must not fully understand it's abilities. To me, I would think you would need to work from the edge of a matrix to use the HISTC and HISTCN functions. Starting from an arbitrary point (r=0 in the above case could be in the center of the matrix), I don't understand how the function could be used to bin evenly spaced cartesian 3d space into bins defined by their spherical transforms? If problem address in this thread needs can be solved with the built-in function HISTC (one dimensional binning). You have to understand how it works. The way you are using FIND for the same purpose is very inefficient. The HISTCN I showed in other thread is an extension of HISTC. You don't need it here. And in the other thread I showed you the code that gives the rigorously identical result that with the double/triple for-loops in the first post. It seems you are trying to visualize what HISTN does in term of geometry, and it gets you lost in the way. Think in a more abstract way of mapping and you'll be fine. Bruno
From: Anthony Hopf on 30 May 2010 08:20 "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <httk3s$hb0$1(a)fred.mathworks.com>... > "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <htthum$sg$1(a)fred.mathworks.com>... > > > > > Bruno, > > > > I had downloaded your histcn function at one point to try and understand how it could be applied to my problem, but I must not fully understand it's abilities. To me, I would think you would need to work from the edge of a matrix to use the HISTC and HISTCN functions. Starting from an arbitrary point (r=0 in the above case could be in the center of the matrix), I don't understand how the function could be used to bin evenly spaced cartesian 3d space into bins defined by their spherical transforms? > > If problem address in this thread needs can be solved with the built-in function HISTC (one dimensional binning). You have to understand how it works. The way you are using FIND for the same purpose is very inefficient. > > The HISTCN I showed in other thread is an extension of HISTC. You don't need it here. And in the other thread I showed you the code that gives the rigorously identical result that with the double/triple for-loops in the first post. It seems you are trying to visualize what HISTN does in term of geometry, and it gets you lost in the way. Think in a more abstract way of mapping and you'll be fine. > > Bruno Bruno, I think you are right on with your statement "It seems you are trying to visualize what HISTN does in term of geometry, and it gets you lost in the way." I will look back at the code you had posted before and think about this problem in a more abstract way. Thanks again, Anthony
From: Anthony Hopf on 8 Jun 2010 15:47 Sorry to "double" post this (seems to fit here much better than the other thread)... but I think I am starting to see the light on this today... I haven't looked at it in awhile. for the case of binning in range, code shown above, I am using this: r_bin = (bin_r/2:bin_r:bin_r_max*bin_r+bin_r/2);%[m] [n,bin] = histc(r_3d(:),r_bin); This gives me n and bin with all the information I need, I am playing with it to figure out how to extract the index values so I have a n x max(n) matrix. But can't I use your histcn function to do all the binning I need to do... r,theta, and phi... at once like this? r_bin = (bin_r/2:bin_r:bin_r_max*bin_r+bin_r/2);%[m] theta_bin = (bin_t/2:bin_t:bin_t_max*bin_size+bin_t/2);%[deg] phi_bin = (bin_p/2:bin_p:bin_p_max*bin_p+bin_p/2);%[deg] hugearray = [r_3d(:) theta_3d(:) phi_3d(:) ]; [count edges mid loc] = histcn(hugearray,r_bin,theta_bin,phi_bin); where count and loc give me essentially the same thing as n and bin that I can then try and place into a sparse matrix to store for later use? Thank you, Anthony "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <httl5l$ns3$1(a)fred.mathworks.com>... > "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <httk3s$hb0$1(a)fred.mathworks.com>... > > "Anthony Hopf" <anthony.hopf(a)gmail.com> wrote in message <htthum$sg$1(a)fred.mathworks.com>... > > > > > > > > Bruno, > > > > > > I had downloaded your histcn function at one point to try and understand how it could be applied to my problem, but I must not fully understand it's abilities. To me, I would think you would need to work from the edge of a matrix to use the HISTC and HISTCN functions. Starting from an arbitrary point (r=0 in the above case could be in the center of the matrix), I don't understand how the function could be used to bin evenly spaced cartesian 3d space into bins defined by their spherical transforms? > > > > If problem address in this thread needs can be solved with the built-in function HISTC (one dimensional binning). You have to understand how it works. The way you are using FIND for the same purpose is very inefficient. > > > > The HISTCN I showed in other thread is an extension of HISTC. You don't need it here. And in the other thread I showed you the code that gives the rigorously identical result that with the double/triple for-loops in the first post. It seems you are trying to visualize what HISTN does in term of geometry, and it gets you lost in the way. Think in a more abstract way of mapping and you'll be fine. > > > > Bruno > > Bruno, > > I think you are right on with your statement "It seems you are trying to visualize what HISTN does in term of geometry, and it gets you lost in the way." I will look back at the code you had posted before and think about this problem in a more abstract way. > > Thanks again, > > Anthony
From: Bruno Luong on 8 Jun 2010 16:25 First, this a an month-old thread. I'm sorry but if you haven't not pay enough attention to what has been suggested for month (including Matt's remark about the miss-use of sparse), the a lot time is waste. Now it will be the last time to write something on this topic, then I'll move on. Here we go: First I change your original function by removing the garbage inside, and also change one comparison test (from > to >=) to match HISTC. %% function [ index_range ] = range_bin_sparse( r_4d, bin_size ) %RANGE_BIN Identifies the index values in r_4d that are within a bin of a given size % each radar is done individually and the ouput file is a sparse % matrix % [ range_index ] = range_bin_sparse( r_4d, bin_size ) %single range bin binning of a single radar r_bin_max = ceil((((max(r_4d(:)))))/bin_size); r_bin = (bin_size/2:bin_size:r_bin_max*bin_size+bin_size/2);%[m] index_range = zeros(length(r_bin),250); %filter variables rH = r_bin + bin_size/2; rL = r_bin - bin_size/2; %find exceptable values for each range bin for v=1:length(r_bin) index_temp = find((r_4d) >= rL(v) & (r_4d) <rH(v)); % change here index_range(v,1:length(index_temp)) = index_temp; end index_range = sparse(index_range'); end % Now here is the data I use to test r_4d=peaks(50)+7; % + 7 to make data positive bin_size = 0.1; [ index_range ] = range_bin_sparse( r_4d, bin_size ); % Here is a way to use HISTC nbins = ceil(max(r_4d(:))/bin_size); edges = (0:nbins)*bin_size; [~,bin] = histc(r_4d,edges); bool = sparse(1:numel(bin), bin,true); % Note the the logical sparse matrix BOOL is not exactly like range, but it contains the % same information, if you want to find which index on the 50th bins, you can do this: >> index_range(:,50) ans = (1,1) 525 (2,1) 526 (3,1) 527 (4,1) 528 (5,1) 580 (6,1) 826 (7,1) 830 (8,1) 1013 (9,1) 1064 (10,1) 1206 (11,1) 1406 (12,1) 1517 % Or this: >> find(bool(:,50)) ans = 525 526 527 528 580 826 830 1013 1064 1206 1406 1517 % You see that they contain the same thing. In practice, next calculation does not need the FIND command, because BOOL can be used as matrix for LOGICAL indexing, that's what Matt suggest you. For example figure out values of thet array r_4d() falling the 50th bins >> r_4d(bool(:,50)) ans = 4.9948 4.9059 4.9003 4.9737 4.9775 4.9397 4.9658 4.9965 4.9447 4.9068 4.9602 4.9365 % Check if they fall inside the bins >> r_4d(bool(:,50))>=edges(50) & r_4d(bool(:,50))<edges(51) ans = 1 1 1 1 1 1 1 1 1 1 1 1 >> % Bruno
From: Anthony Hopf on 8 Jun 2010 16:40
Thank you for your help, I apologize for my slow learning... you guys really are very good! Anthony "Bruno Luong" <b.luong(a)fogale.findmycountry> wrote in message <hum8vg$65s$1(a)fred.mathworks.com>... > First, this a an month-old thread. I'm sorry but if you haven't not pay enough attention to what has been suggested for month (including Matt's remark about the miss-use of sparse), the a lot time is waste. > > Now it will be the last time to write something on this topic, then I'll move on. Here we go: > > First I change your original function by removing the garbage inside, and also change one comparison test (from > to >=) to match HISTC. > > %% > function [ index_range ] = range_bin_sparse( r_4d, bin_size ) > %RANGE_BIN Identifies the index values in r_4d that are within a bin of a given size > % each radar is done individually and the ouput file is a sparse > % matrix > % [ range_index ] = range_bin_sparse( r_4d, bin_size ) > > %single range bin binning of a single radar > r_bin_max = ceil((((max(r_4d(:)))))/bin_size); > r_bin = (bin_size/2:bin_size:r_bin_max*bin_size+bin_size/2);%[m] > index_range = zeros(length(r_bin),250); > > %filter variables > rH = r_bin + bin_size/2; > rL = r_bin - bin_size/2; > %find exceptable values for each range bin > for v=1:length(r_bin) > index_temp = find((r_4d) >= rL(v) & (r_4d) <rH(v)); % change here > index_range(v,1:length(index_temp)) = index_temp; > end > index_range = sparse(index_range'); > > end > > % Now here is the data I use to test > r_4d=peaks(50)+7; % + 7 to make data positive > bin_size = 0.1; > [ index_range ] = range_bin_sparse( r_4d, bin_size ); > > % Here is a way to use HISTC > nbins = ceil(max(r_4d(:))/bin_size); > edges = (0:nbins)*bin_size; > [~,bin] = histc(r_4d,edges); > bool = sparse(1:numel(bin), bin,true); > > % Note the the logical sparse matrix BOOL is not exactly like range, but it contains the > % same information, if you want to find which index on the 50th bins, you can do this: > > >> index_range(:,50) > > ans = > > (1,1) 525 > (2,1) 526 > (3,1) 527 > (4,1) 528 > (5,1) 580 > (6,1) 826 > (7,1) 830 > (8,1) 1013 > (9,1) 1064 > (10,1) 1206 > (11,1) 1406 > (12,1) 1517 > > % Or this: > > >> find(bool(:,50)) > > ans = > > 525 > 526 > 527 > 528 > 580 > 826 > 830 > 1013 > 1064 > 1206 > 1406 > 1517 > > % You see that they contain the same thing. > > In practice, next calculation does not need the FIND command, because BOOL can be used as matrix for LOGICAL indexing, that's what Matt suggest you. For example figure out values of thet array r_4d() falling the 50th bins > > >> r_4d(bool(:,50)) > > ans = > > 4.9948 > 4.9059 > 4.9003 > 4.9737 > 4.9775 > 4.9397 > 4.9658 > 4.9965 > 4.9447 > 4.9068 > 4.9602 > 4.9365 > > % Check if they fall inside the bins > > >> r_4d(bool(:,50))>=edges(50) & r_4d(bool(:,50))<edges(51) > > ans = > > 1 > 1 > 1 > 1 > 1 > 1 > 1 > 1 > 1 > 1 > 1 > 1 > > >> > > % Bruno |