SOS! Very slow Matlab xlsread...
in [Matlab]
|
Prev: Question on ML04 Calculator Exercises from Mathworks Training Services
Next: Apply Mask To Create Average of Nearby Pixel Values + Parallelize
From: Luna Moon on 3 Feb 2010 08:12 On Feb 2, 10:51 pm, ImageAnalyst <imageanal...(a)mailinator.com> wrote: > You just create the Excel object once, right? You don't create it 40 > times. You create it once and call xlsread1() 40 times and then close > the Excel object just once. I'd say you're probably not handling this > right. It shouldn't take 10 hours unless you're reading terabytes of > data or else you're launching and closing Excel way too many times. > Where's your code for creating (instantiating) the Excel object class? Of course I create the object once and used xlsread1(), but the xlsread1 is called thousands of times... (considering 40+ iterations for 40+ sheets).... Any thoughts?
From: Luna Moon on 3 Feb 2010 08:29 On Feb 2, 10:51 pm, ImageAnalyst <imageanal...(a)mailinator.com> wrote: > You just create the Excel object once, right? You don't create it 40 > times. You create it once and call xlsread1() 40 times and then close > the Excel object just once. I'd say you're probably not handling this > right. It shouldn't take 10 hours unless you're reading terabytes of > data or else you're launching and closing Excel way too many times. > Where's your code for creating (instantiating) the Excel object class? Is there a way to do it incrementally?
From: ImageAnalyst on 3 Feb 2010 08:38 On Feb 3, 8:12 am, Luna Moon <lunamoonm...(a)gmail.com> wrote: > Of course I create the object once and used xlsread1(), but the > xlsread1 is called thousands of times... (considering 40+ iterations > for 40+ sheets).... ----------------------------------------- You didn't mention 40+ iterations before - you said that there was only "a big Excel file" which means one file with 40 sheets. So how do you now have 40+ iterations? Are you now saying that you have over 40 of these "big Excel Files"? Regardless, it shouldn't take 10 hours. I haven't used xlsread1 but I've used xlswrite1 and I can switch sheets and throw in an array of numbers quickly. I can create 3 workbooks, each with as many as 10 sheets, and hundreds of numbers in each sheet in just about 5-10 seconds or so. Tens of hours seems way out of line. Does it even complete if you wait that long? In other words, if you gave it 24 hours and came back the next day, is it done successfully? The workbooks aren't on a network are they? They're on your local hard drive, right? Maybe you can post your code with some dummy data so people can test it.
From: Luna Moon on 3 Feb 2010 08:41 On Feb 3, 8:38 am, ImageAnalyst <imageanal...(a)mailinator.com> wrote: > On Feb 3, 8:12 am, Luna Moon <lunamoonm...(a)gmail.com> wrote:> Of course I create the object once and used xlsread1(), but the > > xlsread1 is called thousands of times... (considering 40+ iterations > > for 40+ sheets).... > > ----------------------------------------- > You didn't mention 40+ iterations before - you said that there was > only "a big Excel file" which means one file with 40 sheets. So how > do you now have 40+ iterations? Are you now saying that you have over > 40 of these "big Excel Files"? Regardless, it shouldn't take 10 > hours. > > I haven't used xlsread1 but I've used xlswrite1 and I can switch > sheets and throw in an array of numbers quickly. I can create 3 > workbooks, each with as many as 10 sheets, and hundreds of numbers in > each sheet in just about 5-10 seconds or so. Tens of hours seems way > out of line. Does it even complete if you wait that long? In other > words, if you gave it 24 hours and came back the next day, is it done > successfully? > > The workbooks aren't on a network are they? They're on your local > hard drive, right? > > Maybe you can post your code with some dummy data so people can test > it. Sorry I meant I have 40+ sheets and for each sheet I need to do something, and so I put them into a loop and so that's 40+ iterations,
From: ImageAnalyst on 3 Feb 2010 09:27
That shouldn't take more than a day, or even more than a minute. Maybe it has something to do with specifying the range as being the maximum possible. With xlsread() you can omit the range and it will read in just what is there. Won't xlsread1() work the same way? Maybe if you specify the max possible size it's trying to allocate some huge block of empty cells and haul them over. This shouldn't be necessary. Try to eliminate the range and see if it will get everything that's contained in the spreadsheet automatically. And like I said, "Maybe you can post your code with some dummy data so people can test it." You didn't respond to that suggestion for some reason - maybe you're still working on simplifying your code down to something you can post. |