Prev: Remove small objects from binary image without erode.
Next: remove duplicate rows in sparse matrix
From: Jan Simon on 10 Jul 2010 17:57
Dear Adam,

I'm still confused. Is "textmatrix{2}(i)" a CELL or a CHAR?! After TEXTREAD('%s') is shoould be a cell string and
sprintf('%d %s', tm{2,1}, textmatrix{2}(i))
should actually fail ??
And TEXTREAD('%d') should reply a DOUBLE, not an UINT32 for the year, month and day.

> I have the function output the current index that is being accessed while it is running so i can monitor the progress and it is quite fast at first (approx 1000/sec) but as it accesses the higher indicies it becomes very very slow (almost 5 sec for 1000 by the time its at index 100,000) and continues to get slower as this increases.

This is the expected behaviour, if an array is growing in each step of a loop. The repeated allocation eats up processing time.
Create the struct x with the needed number of elements at first. E.g. (I'm not sure what "count" is):
x(1, count) = struct('Symbol', [], 'Date', [], 'Price', []);

It would be much faster to create a specific variable instead of accessing "textmatrix{1}" again and again, e.g.:
SymbolC = textmatrix{1};
YearV = textmatrix{2};
TimeC = textmatrix{3};

Some further simplifications are possible, e.g.:

> symb = textmatrix{1}(i);
> while(strcmp(symb,symbol)==0)
> i = i+1;
> if(i>= length(textmatrix{1}))
> break
> end
> symb = textmatrix{1}(i);
> end
> if(i>= length(textmatrix{1}))
> break
> end

symb = '';
for i = 1:length(SymbolC)
if any(strcmp(SymbolC{i}, symbol))
symb = SymbolC{i};
end
end
if isempty(symb)
break;
end

and then, e.g. with the method of Us:
% The date does not depend on the for loop counter q.
% Therefore it is much faster to compute it once only!
aDate = datevec(sprintf('%d %s', YearV(i), TimeC{i}),'yyyymmdd HH:MM:SS');
for q=1:count
x(j).Symbol = symb;
x(j).Date = aDate;
j = j+1;
end
end

I'd expect a line, which adds "symb" to the list "symbols".
It would be helpful, if you'd post a cleaned and working piece of code again, perhaps with 3 lines of test data.
If the pre-allocation problem is fixed and the date is not converted more times than needed, my DATEVEC replacement posted above might be a further helpful acceleration.

Kind regards, Jan
First  |  Prev  | 
Pages: 1 2
Prev: Remove small objects from binary image without erode.
Next: remove duplicate rows in sparse matrix