Quality of logistic regression model
in [SAS]
|
Prev: logistics model.
Next: data manipulation problem
From: Murphy Choy on 22 Oct 2009 09:38 Hi, Can I propose that you take a look at roc curve and the gini coefficient? If you see they are good, it may be just a case that one or two variables are bad. ------Original Message------ From: Alexis Lelex Sender: SAS(r) Discussion To: SAS-L(a)LISTSERV.UGA.EDU ReplyTo: Alexis Lelex Subject: Quality of logistic regression model Sent: Oct 22, 2009 8:52 PM Hi, This is my first post here and my english is not well... so i'll do my best to make me understand. I'm modelling a logistic regression on more than 120 000 individuals, and i get some very interesting results with my odds ratios, and all p-values are <0,0001. But some figures in the SAS output make look bad quality of the model: high AIC and SC, low R2 and Tau-a, Hosmer and Lemeshow telling a lack of fit... Here's some part of my output: Statistiques d'ajustement du modèle Coordonnées Coordonnées � l'origine Critères A l'origine Avec Covariables AIC 91616.461 83858.175 SC 91626.215 84043.487 -2 Log L 91614.461 83820.175 R-Square 0.0595 Max-rescaled R-Square 0.1158 Association des probabilités prédites et des réponses observées Percent Concordant 71.2 Somers' D 0.431 Percent Discordant 28.1 Gamma 0.434 Percent Tied 0.7 Tau-a 0.089 Pairs 1666460055 c 0.715 Test d'adéquation d'Hosmer et de Lemeshow Khi 2 DF Pr > Khi 2 78.2312 8 <.0001 Is it possible to make the interpretation of the odds ratios, even though there's a lack of fit and the model isn't predictive ? In other words what conclusion can we take (or not) from a model like this one ? If someone can help me on this one it'll be really great ! Thanks PS: by the way very good SAS forum, i learn a lots of things reading you peoples ! Sent from my BlackBerry Wireless Handheld -- Regards, Murphy Choy Certified Advanced Programmer for SAS V9 Certified Basic Programmer for SAS V9
From: Alexis Lelex on 22 Oct 2009 10:04 Yes i put out the roc curves like this: proc gplot data=roc1; title 'ROC Curve'; plot _sensit_*_1mspec_=1 / vaxis=0 to 1 by .1 cframe=ligr; run; But i have no idea to telle something about it. I read the Gini coefficient can be obtained like this: Gini = 2*C statistics - 1 so in my case Gini=0,43 is it good or not ? i can't tell...
From: Alexis Lelex on 22 Oct 2009 10:28 Thanks, i heard the c statistics correspond to the area under the Roc curb, and can variate between 0,5 and 1. I already try to leave some covariates out of the model, but i still got the same problem.
From: Murphy Choy on 22 Oct 2009 10:18 Hi, To understand roc curve, you can read the information from sas papers on roc. Gini looks ok. ------Original Message------ From: Alexis Lelex Sender: SAS(r) Discussion To: SAS-L(a)LISTSERV.UGA.EDU ReplyTo: Alexis Lelex Subject: Re: Quality of logistic regression model Sent: Oct 22, 2009 10:04 PM Yes i put out the roc curves like this: proc gplot data=roc1; title 'ROC Curve'; plot _sensit_*_1mspec_=1 / vaxis=0 to 1 by .1 cframe=ligr; run; But i have no idea to telle something about it. I read the Gini coefficient can be obtained like this: Gini = 2*C statistics - 1 so in my case Gini=0,43 is it good or not ? i can't tell... Sent from my BlackBerry Wireless Handheld -- Regards, Murphy Choy Certified Advanced Programmer for SAS V9 Certified Basic Programmer for SAS V9
From: Murphy Choy on 22 Oct 2009 10:47
Hi, What abt the lift? ------Original Message------ From: Alexis Lelex Sender: SAS(r) Discussion To: SAS-L(a)LISTSERV.UGA.EDU ReplyTo: Alexis Lelex Subject: Re: Quality of logistic regression model Sent: Oct 22, 2009 10:28 PM Thanks, i heard the c statistics correspond to the area under the Roc curb, and can variate between 0,5 and 1. I already try to leave some covariates out of the model, but i still got the same problem. Sent from my BlackBerry Wireless Handheld -- Regards, Murphy Choy Certified Advanced Programmer for SAS V9 Certified Basic Programmer for SAS V9 |