Is Regression Using Proc IML Faster?
in [SAS]
|
From: Claus Yeh on 24 Jan 2010 03:01 On Jan 23, 1:31 pm, art...(a)NETSCAPE.NET (Arthur Tabachneck) wrote: > Claus, > > I don't know which would be faster, but will make you an offer. > > Here is some stater code:http://www.psych.yorku.ca/lab/psy6140/ex/iml.htm > > My offer: I provide the above starter code .. you do the tests .. > then you let the list know which ended up being faster. > > Deal? > > Art > ---------- > On Jan 23, 3:34 pm, Claus Yeh <phoebe.caulfiel...(a)gmail.com> wrote: > > > Dear SAS guru's, > > > I have been using proc glm and proc logistic for a while now. these > > procedures are great - alot of options and relatively easy to use. > > > However, I always felt that they are a bit slow since they carry alot > > of calculations and outputs that are not always needed. > > > I am thinking about diving into Proc IML and write a more basic code > > for regression. > > > Has anyone tried that and got much faster run time? > > > thank you, > > claus Thank you Art. I'll give it a try. will also do some benchmark comparisons.
From: Dale McLerran on 24 Jan 2010 15:35 Mark, You would exclude the time that it takes to read the data from disk into an IML matrix as part of the time that is required for performing the regression using IML? Why? Certainly, when you look at the CPU and total time summaries that are produced by IML, those times would include the time that it takes to read the data. I really doubt that fitting regression models employing IML would save much time. I imagine that SAS has optimized many aspects of fitting a regression model. These aspects would include some features that optimize efficiency. Other efficiencies would improve accuracy of the regression results. From my perspective, I would not fit simple regression models using IML in an effort to shave time from fitting the regression. If one wants to study the equations which are used to fit a regression, then using IML has value. But for production work in fitting simple regression models, I would not use IML. Dale --------------------------------------- Dale McLerran Fred Hutchinson Cancer Research Center mailto: dmclerra(a)NO_SPAMfhcrc.org Ph: (206) 667-2926 Fax: (206) 667-5977 --------------------------------------- --- On Sun, 1/24/10, Keintz, H. Mark <mkeintz(a)WHARTON.UPENN.EDU> wrote: > From: Keintz, H. Mark <mkeintz(a)WHARTON.UPENN.EDU> > Subject: Re: Is Regression Using Proc IML Faster? > To: SAS-L(a)LISTSERV.UGA.EDU > Date: Sunday, January 24, 2010, 12:00 PM > Ceteris paribus, IML regression > SHOULD be a bit faster, since the data are already in > memory. But I doubt this advantage wold hold up with a > large dataset, or being run on a server on which your > program is competing for memory and other resources. > > Regards, > Mark >
From: "Keintz, H. Mark" on 24 Jan 2010 15:00 Ceteris paribus, IML regression SHOULD be a bit faster, since the data are already in memory. But I doubt this advantage wold hold up with a large dataset, or being run on a server on which your program is competing for memory and other resources. Regards, Mark > -----Original Message----- > From: SAS(r) Discussion [mailto:SAS-L(a)LISTSERV.UGA.EDU] On Behalf Of > Murphy Choy > Sent: Saturday, January 23, 2010 9:14 PM > To: SAS-L(a)LISTSERV.UGA.EDU > Subject: Re: Is Regression Using Proc IML Faster? > > Hi, > > I have tried simple linear regression on iml. It was slightly faster > but it ran into memory problems when I feed a large data to it. > > At the same time, you can also look at using the regression iml macro > provided in the user guide which can be very useful. > > ------Original Message------ > From: Arthur Tabachneck > Sender: SAS(r) Discussion > To: SAS-L(a)LISTSERV.UGA.EDU > ReplyTo: Arthur Tabachneck > Subject: Re: Is Regression Using Proc IML Faster? > Sent: Jan 24, 2010 10:07 AM > > Tanwan, > > Your comment reminds me of my first job as an analyst where I worked > with two systems, Statistical Analysis System (on an IBM 360) and a > memory-card-based HP electronic calculator. > > All of the correlational analyses, for whatever reason, were always > done with the calculator's stock program. I found it surprising that > all of the relationships discovered were positive ones. Of course, as > it turned out, they weren't. The software makers had made a mistake > in their programming. > > What I learned from that experience was to NEVER assume that ANY > software maker was infallible. > > Besides, in the OPs case, what a perfect way to learn IML, learn more > about regression, and simultaneously be able to compare their results > with SAS output and discover which is correct or incorrect and why. > > Art > ---------- > On Jan 23, 7:55 pm, tanwan <tanwanz...(a)yahoo.com> wrote: > > GLM and Logistic are well documented, tested, and they do what they > > are supposed and intended to do. What are the odds that you will make > > a coding error with IML that you wont even notice, trying to re- > invent > > a wheel? > > > > Besides, how much time are you going to save? A few seconds? A few > > minutes? I think on average one spends 95% of the time coding code, > > and then 5% running the code. You can use those few extra moments for > > a trip to the water cooler, call someone significant, or just browse > > the latest online news. > > > > T > > > Sent from my BlackBerry Wireless Handheld > > -- > Regards, > Murphy Choy > > Certified Advanced Programmer for SAS V9 > Certified Basic Programmer for SAS V9
From: Murphy Choy on 25 Jan 2010 10:45 Hi, At the same time, there are memory limitation for big datasets. ------Original Message------ From: tanwan Sender: SAS(r) Discussion To: SAS-L(a)LISTSERV.UGA.EDU ReplyTo: tanwan Subject: Re: Is Regression Using Proc IML Faster? Sent: Jan 24, 2010 8:55 AM GLM and Logistic are well documented, tested, and they do what they are supposed and intended to do. What are the odds that you will make a coding error with IML that you wont even notice, trying to re-invent a wheel? Besides, how much time are you going to save? A few seconds? A few minutes? I think on average one spends 95% of the time coding code, and then 5% running the code. You can use those few extra moments for a trip to the water cooler, call someone significant, or just browse the latest online news. T Sent from my BlackBerry Wireless Handheld -- Regards, Murphy Choy Certified Advanced Programmer for SAS V9 Certified Basic Programmer for SAS V9
From: Murphy Choy on 25 Jan 2010 10:46 Hi, You can also refer to the iml documentation for the linear regression and other linear models. ------Original Message------ From: Claus Yeh Sender: SAS(r) Discussion To: SAS-L(a)LISTSERV.UGA.EDU ReplyTo: Claus Yeh Subject: Re: Is Regression Using Proc IML Faster? Sent: Jan 24, 2010 4:01 PM On Jan 23, 1:31 pm, art...(a)NETSCAPE.NET (Arthur Tabachneck) wrote: > Claus, > > I don't know which would be faster, but will make you an offer. > > Here is some stater code:http://www.psych.yorku.ca/lab/psy6140/ex/iml.htm > > My offer: I provide the above starter code .. you do the tests .. > then you let the list know which ended up being faster. > > Deal? > > Art > ---------- > On Jan 23, 3:34 pm, Claus Yeh <phoebe.caulfiel...(a)gmail.com> wrote: > > > Dear SAS guru's, > > > I have been using proc glm and proc logistic for a while now. these > > procedures are great - alot of options and relatively easy to use. > > > However, I always felt that they are a bit slow since they carry alot > > of calculations and outputs that are not always needed. > > > I am thinking about diving into Proc IML and write a more basic code > > for regression. > > > Has anyone tried that and got much faster run time? > > > thank you, > > claus Thank you Art. I'll give it a try. will also do some benchmark comparisons. Sent from my BlackBerry Wireless Handheld -- Regards, Murphy Choy Certified Advanced Programmer for SAS V9 Certified Basic Programmer for SAS V9
First
|
Prev
|
Next
|
Last
Pages: 1 2 3 4 Prev: Partial Nesting in Proc Mixed Next: question asked in an Interview |