Prev: Regexp IF statement
Next: Using static pointers to MATLAB object in mex-file.
From: Will Thompson on 23 Jul 2010 10:46
hey,

I am running a very data intensive project that requires parsing through very long text files and picking out information. To accomplish this, I have been using fgets to go line by line through the file and to look for this information with strread. However, this process is being very slow and it seems to be due to the sttread function.

Does anyone have a recommendation for a faster version of sttread?

Thanks,

will
From: John Zoidberg on 23 Jul 2010 11:01
You could try using fileread to read in the whole file at once into one string.
That's what I did here:
http://www.mathworks.com/matlabcentral/newsreader/view_thread/287600

I'm not sure how well it works with huge files, but having everything already in memory should be faster in principle.

You could also try compiling the script into an exe using the matlab compiler:
http://www.mathworks.com/matlabcentral/newsreader/view_thread/164864
From: Andres on 24 Jul 2010 18:02
"Will Thompson" <williamthompson2010(a)u.northwestern.edu> wrote in message <i2c9vd$cmp$1(a)fred.mathworks.com>...
> hey,
>
> I am running a very data intensive project that requires parsing through very long text files and picking out information. To accomplish this, I have been using fgets to go line by line through the file and to look for this information with strread. However, this process is being very slow and it seems to be due to the sttread function.
>
> Does anyone have a recommendation for a faster version of sttread?
>
> Thanks,
>
> will

I suppose textscan is faster than the (outdated) strread.
It would be helpful if you give an example or some more details. E.g., do you need to get some information out of every line, or just from some of the lines? If, in the latter case, the lines of interest can be identified by some few keywords, it might be worth having a look at my file exchange submission txt2mat with its line filtering options (and probably read mode 'char' or 'cell', see the doc) to get around using fgets through the whole file (which isn't very fast either).
Good luck,
Andres
From: Jan Simon on 24 Jul 2010 18:53
Dear Will,

> > I am running a very data intensive project that requires parsing through very long text files and picking out information. To accomplish this, I have been using fgets to go line by line through the file and to look for this information with strread. However, this process is being very slow and it seems to be due to the sttread function.

Please show us some relevant lines of the file and your STRREAD command. Without more details it is hard to guess, what might accelerate your program.

Kind regards, Jan
 | 
Pages: 1
Prev: Regexp IF statement
Next: Using static pointers to MATLAB object in mex-file.