SOS! Very slow Matlab xlsread...
in [Matlab]
|
Prev: Question on ML04 Calculator Exercises from Mathworks Training Services
Next: Apply Mask To Create Average of Nearby Pixel Values + Parallelize
From: Luna Moon on 2 Feb 2010 18:02 Hi all, I have a big Excel file that I need to read into Mablab. It has 40 sheets, and iteratively for each sheet, I need to read in a bunch of stuff as the following: [tmp cellstrDates]=xlsread1(strDataFileName, strCurrentSheet, 'B23:B65535'); Because the content on the sheets will keep growing, so there is no way for me to know how many rows there are on the sheets (each sheet may contain different number of rows). That's why I have to put 65535 above. As you can see, I've also tried "xlsread1" from Mathwork File Exchange, it didn't speed up too much for me. The whole process takes about tens of hours for me... Anybody can help? Thanks a lot!
From: Luna Moon on 2 Feb 2010 18:05 On Feb 2, 6:02 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote: > Hi all, > > I have a big Excel file that I need to read into Mablab. > > It has 40 sheets, and iteratively for each sheet, I need to read in a > bunch of stuff as the following: > > [tmp cellstrDates]=xlsread1(strDataFileName, strCurrentSheet, > 'B23:B65535'); > > Because the content on the sheets will keep growing, so there is no > way for me to know how many rows there are on the sheets (each sheet > may contain different number of rows). That's why I have to put 65535 > above. > > As you can see, I've also tried "xlsread1" from Mathwork File > Exchange, it didn't speed up too much for me. > > The whole process takes about tens of hours for me... > > Anybody can help? > > Thanks a lot! I am also thinking of doing the whole thing incrementally, ie. every time, I just have to read in the parts that have grown... however, how do I know for each sheet, for each column, where are the contents that have been grown/expanded? Thanks a lot!
From: Luna Moon on 2 Feb 2010 18:34 On Feb 2, 6:05 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote: > On Feb 2, 6:02 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote: > > > > > Hi all, > > > I have a big Excel file that I need to read into Mablab. > > > It has 40 sheets, and iteratively for each sheet, I need to read in a > > bunch of stuff as the following: > > > [tmp cellstrDates]=xlsread1(strDataFileName, strCurrentSheet, > > 'B23:B65535'); > > > Because the content on the sheets will keep growing, so there is no > > way for me to know how many rows there are on the sheets (each sheet > > may contain different number of rows). That's why I have to put 65535 > > above. > > > As you can see, I've also tried "xlsread1" from Mathwork File > > Exchange, it didn't speed up too much for me. > > > The whole process takes about tens of hours for me... > > > Anybody can help? > > > Thanks a lot! > > I am also thinking of doing the whole thing incrementally, ie. every > time, I just have to read in the parts that have grown... > > however, how do I know for each sheet, for each column, where are the > contents that have been grown/expanded? > > Thanks a lot! Even if I record down the previous content area, it still doesnot work. Let's take column B as an example: Let's say previously I have read in everything until B4654, and so this time, being incremental and "smarter", I put: [tmp cellstrDates]=xlsread1(strDataFileName, strCurrentSheet, 'B4655:B65535'); This still doesn't help too much, right? Any thoughts? Thanks a lot!
From: ade77 on 2 Feb 2010 19:31 Luna Moon <lunamoonmoon(a)gmail.com> wrote in message <c98e586c-1269-4012-ac1e-246333422f6e(a)m4g2000vbn.googlegroups.com>... > On Feb 2, 6:05 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote: > > On Feb 2, 6:02 pm, Luna Moon <lunamoonm...(a)gmail.com> wrote: > > > > > > > > > Hi all, > > > > > I have a big Excel file that I need to read into Mablab. > > > > > It has 40 sheets, and iteratively for each sheet, I need to read in a > > > bunch of stuff as the following: > > > > > [tmp cellstrDates]=xlsread1(strDataFileName, strCurrentSheet, > > > 'B23:B65535'); > > > > > Because the content on the sheets will keep growing, so there is no > > > way for me to know how many rows there are on the sheets (each sheet > > > may contain different number of rows). That's why I have to put 65535 > > > above. > > > > > As you can see, I've also tried "xlsread1" from Mathwork File > > > Exchange, it didn't speed up too much for me. > > > > > The whole process takes about tens of hours for me... > > > > > Anybody can help? > > > > > Thanks a lot! > > > > I am also thinking of doing the whole thing incrementally, ie. every > > time, I just have to read in the parts that have grown... > > > > however, how do I know for each sheet, for each column, where are the > > contents that have been grown/expanded? > > > > Thanks a lot! > > Even if I record down the previous content area, it still doesnot > work. Let's take column B as an example: > > Let's say previously I have read in everything until B4654, > > and so this time, being incremental and "smarter", > > I put: > > [tmp cellstrDates]=xlsread1(strDataFileName, strCurrentSheet, > 'B4655:B65535'); > > This still doesn't help too much, right? > > Any thoughts? Thanks a lot! If the contents of your sheets has the same column of data, you can try something like this to avoid a loop: [F,PathName,FilterIndex] = uigetfile({'*.*','All Files(*.*)'}, 'Select the File you want'); b = strcat(PathName,F); Input = importdata(b); if (isstruct(Input)) Input = struct2cell(Input); Input = cell2mat(Input); end The if structure will take care of the many sheets you have, and all the matrix will be in the variable 'input'. All the contents of the each sheet will be appended to the end of the variable 'input'. This might not be what you are looking for, but just something you should be aware of.
From: ImageAnalyst on 2 Feb 2010 22:51
You just create the Excel object once, right? You don't create it 40 times. You create it once and call xlsread1() 40 times and then close the Excel object just once. I'd say you're probably not handling this right. It shouldn't take 10 hours unless you're reading terabytes of data or else you're launching and closing Excel way too many times. Where's your code for creating (instantiating) the Excel object class? |