Getting indexes of rows of matrix with more than n repetitions
in [Matlab]
|
From: us on 6 Jan 2010 02:48 "Sudheer Tumu" > Now It is taking 30 sec . It is great when It is compared to 1300 sec. Is there any fast way to do than taking 30 sec. one of the solutions - replace ISMEMBER by ISMEMBC... n=3; v=[ 1, 3, 5, 1, 1, 3, 1, 5, 7, 1, 3 2, 4, 6, 2, 2, 4, 2, 6, 8, 2, 4 ].'; [vu,vx,vx]=unique(v,'rows'); vn=histc(vx,1:max(vx)); % ix=find(ismember(vx,find(vn>=n))).' % <- ISMEMBER ix=find(ismembc(vx,find(vn>=n))).' % <- ISMEMBC % ix = 1 2 4 5 6 7 10 11 % timing on a wintel sys ic2/2*2.6gzh/2gb/winxp.sp3.32/r2009b % using the previous IX tic; for i=1:10000; ix=find(ismember(vx,find(vn>=n))); end; t1=toc tic; for i=1:10000; ix=find(ismembc(vx,find(vn>=n))); end; t2=toc % t1 = 0.3857 % t2 = 0.082618 the big bottleneck, though, is the FIND, which you need given your task... us
From: us on 6 Jan 2010 02:49 "Jan Simon" > > > n=3; > > > v=[ > > > 1, 3, 5, 1, 1, 3, 1, 5, 7, 1, 3 > > > 2, 4, 6, 2, 2, 4, 2, 6, 8, 2, 4 > > > ].'; > > > [vu,vx,vx]=unique(v,'rows'); > > > vn=histc(vx,1:max(vx)); > > > ix=find(ismember(vx,find(vn>=n))).' > > > % ix = 1 2 4 5 6 7 10 11 > Then UNIQUE does not need the 'rows' flag and sorting might become faster. > There is some potential in us' method: UNIQUE and ISMEMBER sort their input (and perhaps HISTC does also?). So it would be faster to sort the large array once and operate with the sorted index. ?? us
From: Jos (10584) on 6 Jan 2010 03:04 "us " <us(a)neurol.unizh.ch> wrote in message <hi1f7i$plv$1(a)fred.mathworks.com>... > "Sudheer Tumu" > > Now It is taking 30 sec . It is great when It is compared to 1300 sec. Is there any fast way to do than taking 30 sec. > > one of the solutions > - replace ISMEMBER by ISMEMBC... > > n=3; > v=[ > 1, 3, 5, 1, 1, 3, 1, 5, 7, 1, 3 > 2, 4, 6, 2, 2, 4, 2, 6, 8, 2, 4 > ].'; > [vu,vx,vx]=unique(v,'rows'); > vn=histc(vx,1:max(vx)); > % ix=find(ismember(vx,find(vn>=n))).' % <- ISMEMBER > ix=find(ismembc(vx,find(vn>=n))).' % <- ISMEMBC > % ix = 1 2 4 5 6 7 10 11 > > % timing on a wintel sys ic2/2*2.6gzh/2gb/winxp.sp3.32/r2009b > % using the previous IX > tic; for i=1:10000; ix=find(ismember(vx,find(vn>=n))); end; t1=toc > tic; for i=1:10000; ix=find(ismembc(vx,find(vn>=n))); end; t2=toc > % t1 = 0.3857 > % t2 = 0.082618 > > the big bottleneck, though, is the FIND, which you need given your task... > > us ... avoid FIND ... n=3; v=[ 1, 3, 5, 1, 1, 3, 1, 5, 7, 1, 3 2, 4, 6, 2, 2, 4, 2, 6, 8, 2, 4 ].'; [vu,vx,vx]=unique(v,'rows'); idx1 = 1:size(vu,1) ; vn=histc(vx,idx1) idx2 = 1:size(v,1) t3 = idx2(ismembc(vx, idx1(vn>=n))) I leave the accurate testing for speed to others ... Jos
From: us on 6 Jan 2010 04:02 "Jos (10584) " > .. avoid FIND ... > idx1 = 1:size(vu,1) ; > vn=histc(vx,idx1) > idx2 = 1:size(v,1) > t3 = idx2(ismembc(vx, idx1(vn>=n))) > I leave the accurate testing for speed to others ... > Jos well, it does not look too different... - and even a bit better for FIND... - on a wintel sys ic2/2*2.6ghz/2gb/winxp.sp3.32/r2009b the test function below yields these result 7.1716 % <- JOS: logical indexing 6.9788 % <- US : find 102.76 % <- %gain function t=foo % the data nt=100000; % <- #runs nr=100; % <- #repetitions of original V n=3; v=[ 1, 3, 5, 1, 1, 3, 1, 5, 7, 1, 3 2, 4, 6, 2, 2, 4, 2, 6, 8, 2, 4 ].'; t=nan(3,1); v=repmat(v,nr,1); [vu,vx,vx]=unique(v,'rows'); % JOS tic; for i=1:nt r1=f1(n,v,vu,vx); end t(1)=toc; % US tic; for i=1:nt r2=f2(n,v,vu,vx); end t(2)=toc; t(3)=100*t(1)/t(2); isequal(r1(:),r2(:)); % <- TRUE disp(t); end % JOS function ix=f1(n,v,vu,vx) idx1=1:size(vu,1) ; idx2=1:size(v,1); vn=histc(vx,idx1); ix=idx2(ismembc(vx, idx1(vn>=n))); end % US function ix=f2(n,v,vu,vx) vn=histc(vx,1:max(vx)); ix=find(ismembc(vx,find(vn>=n))); end just a thought... us
From: Jos (10584) on 6 Jan 2010 04:36
"us " <us(a)neurol.unizh.ch> wrote in message <hi1jie$4bp$1(a)fred.mathworks.com>... > "Jos (10584) " > > .. avoid FIND ... > > idx1 = 1:size(vu,1) ; > > vn=histc(vx,idx1) > > idx2 = 1:size(v,1) > > t3 = idx2(ismembc(vx, idx1(vn>=n))) > > I leave the accurate testing for speed to others ... > > Jos > > well, it does not look too different... > - and even a bit better for FIND... I already thought so ... Jos |