PROC LOGISTIC: Traps for the unwary Peter L. Flom, Independent statistical consultant, New York, NY

Transcription

1 PROC LOGISTIC: Traps for the unwary Peter L. Flom, Independent statistical consultant, New York, NY ABSTRACT Keywords: Logistic. INTRODUCTION This paper covers some gotchas in SAS R PROC LOGISTIC. A gotcha is a mistake that isn t obviously a mistake the program runs, there may be a note or a warning, but no errors. Output appears. But it s the wrong output. This is not a primer on PROC LOGISTIC, much less on logistic regression. There are many good books on logistic regression; one such is Hosmer and Lemeshow [2000]. Each section has several subsections. First, I identify the gotcha. Then I give an example. Third, I show what evidence you have that it occurs. Fourth, I show how to fix it in some cases, referring to other resources. In some cases, I offer an explanation between the evidence and the solution. GOTCHA #1 CODING 0 AND 1 PROC LOGISTIC can be used to run logistic regression on a dichotomous dependent variable. Often, these are coded 0 and 1, with 0 for no or the equivalent, and 1 for yes or the equivalent. In this case, we are usually interested in modeling the probability of a yes. However, by default, SAS models the probability of a 0 (which would be a no ). For example, we might be interested in modeling the presence of a disease, with 0 meaning the person is not infected, and 1 meaning he or she is infected. To keep it simple, I will use one independent variable: sex, code as 1 for female and 0 for male. So: d a t a t o d a y ; i n p u t d i s e a s e f emale weight ; d a t a l i n e s ; ; ; ; ; we then run PROC LOGISTIC: p roc l o g i s t i c d a t a = t o d a y ; model d i s e a s e = f emale ; and get, among other output, an odds ratio estimate of 1.39 for female, while it s clear that men are much more likely to be infected. The evidence that this is happening is one line in the output: {Probability modeled is disease=0} and several lines in the log: NOTE: PROC LOGISTIC is modeling the probability that disease=0. One way to change this to model the probability that disease=1 is to specify the response variable option EVENT= 1 There are several solutions. The simplest is not the one mentioned in the log, but rather the DESCENDING option. 1

2 p roc l o g i s t i c d a t a = t o d a y d e s c e n d i n g ; model d i s e a s e = f emale ; Another method is the one mentioned in the log, which is more general: p roc l o g i s t i c d a t a = t o d a y ; model d i s e a s e ( e v e n t = 1 ) = female ; GOTCHA #2 EFFECT CODING AND CLASS VARIABLES When you have a categorical independent variable with more than 2 levels, you need to define it with a CLASS statement. In PROC GLM the default coding for this is dummy coding. In PROC LOGISTIC, it s effect coding. To me, effect coding is quite unnatural. EXAMPLE Continuing with the same example of modeling probability of infection, suppose you now have race/ethnicity as an IV, with 6 categories, as defined by the Census Bureau: White, Black/African American, Hispanic/Latino, American Indian/Alaskan Native, Native Hawaiian or other Pacific Islander, and Asian. p roc l o g i s t i c d a t a = today2 ; c l a s s r a c e ; model d i s e a s e ( e v e n t = 1 ) = r a c e ; and get parameter estimates that include (among much else): Parameter DF Estimate Intercept race AIAN race AfrA race Asian race Lat race NHPI and OR estimates Point Effect Estimate race AIAN vs White race AfrA vs White race Asian vs White race Lat vs White race NHPI vs White but we know that the OR estimate should be e OR, and, for example, e.06 = 0.94 not 1. The design matrix. With the default, the design matrix looks like this: 2

3 race AIAN AfrA Asian Lat NHPI White and each parameter estimate estimates the difference between that level and the average of the other levels. On the other hand, with dummy (or reference) coding, it looks like race AIAN AfrA Asian Lat NHPI White and each parameter estimates the difference between that level and the reference group. Use the param = reference option on the class statement: p roc l o g i s t i c d a t a = today2 ; c l a s s r a c e / param = r e f ; model d i s e a s e ( e v e n t = 1 ) = r a c e ; For more information (and other possible parameterizations) see the SAS documentation for PROC LOGISTIC, in particular the section CLASS variable parameterization in DETAILS GOTCHA #3 QUASI-COMPLETE AND COMPLETE SEPARATION If you picture the data as a 2x2 crosstab, then quasi-complete separation occurs when one of the cells is 0. Complete separation occurs when one cell in each row and column is 0. EXAMPLE QUASI-COMPLETE SEPARATION An example of quasi-complete separation is: d a t a t o d a y 7 a ; i n p u t x $ y d a t a l i n e s ; A C A C A C A C A C B C B C B C B C B C B C B C B C B C B D B D B D B D B D ; ; ; ; B C It varies. Confidence intervals will be extremely wide. But sometimes there is a warning that there was complete or quasicomplete separation, sometimes a note. Sometimes, you get a warning that convergence was not attained. With the weight option you sometimes get a note that observations with nonpositive frequencies or weights were excluded. For example, if you run the data step creating data today7a, and then p roc l o g i s t i c d a t a = t o d a y 7 a ; c l a s s x ; model y = x ; 3

4 you get a warning in the log. But if you run this data step: d a t a today7 ; i n p u t x $ y $ d a t a l i n e s ; A C 5 A D 0 B C 10 B D 5 ; ; ; and then p roc l o g i s t i c d a t a = today7 ; c l a s s x ; model y = x ; w e i g ht weight ; you get a note in the log. Sometimes, you can delete the offending variable. In the example, there was only one independent variable, but usually, there will be more, and one of them will be the problematic one. Alternatively, if there are multiple categories, it may be sensible to combine some of them. A third possibility is to leave the offending variable in the equation, and simply report the results for the other variables these are still correct. You can then report the coefficients for the offending variables as inf. Finally, you can use exact inference with the EXACT option. REFERENCES Paul Allison s paper at SGF 2008 has a great deal of lucid explanation of this problem [Allison, 2008], although his statement that PROC LOGISTIC gives clear diagnostic messages is, as we have seen, not always true. GOTCHA #4 CONCORDANT AND DISCORDANT Part of the default output from PROC LOGISTIC is a table that has entries including percent concordant and percent discordant. To me, this implies the percent that would correctly be assigned, based on the results of the logistic regression. But that is not what it is. It looks at all possible pairs of observations. A pair is concordant if the observation with the larger value of X also has the larger value of Y. A pair is discordant if the observation with the larger value of X has the smaller value of Y; here, X and Y are the predicted value and the actual value. EXAMPLE For our first example, the output looks like this: Association of Predicted Probabilities and Observed Responses Percent Concordant 25.0 Somers D Percent Discordant 25.0 Gamma Percent Tied 50.0 Tau-a Pairs 4 c It is hard to find documentation of this. I couldn t find it explained in the LOGISTIC documentation. I found a mention of concordant and discordant in the FREQ documentation, but it was not clear what X and Y were, until I searched SAS-L and found an explanation from David Cassell. For what I was thinking of, you need the CTABLE option on the MODEL statement, which gives the proportion correctly classified, the sensitivity, the specificity, and other measures for each of a number of cutpoints of the predicted probability level. By default, it gives probability levels from 0 to 1 at intervals of.02, but if you just want a few, you can get them: p roc l o g i s t i c d a t a = today3 ; c l a s s r a c e sex / param = r e f ; model d i s e a s e ( e v e n t = 1 ) = r a c e sex / c t a b l e pprob = ( ) ; 4

5 which yields Correct Incorrect Percentages Prob Non- Non- Sensi- Speci- False False Level Event Event Event Event Correct tivity ficity POS NEG GOTCHA #5 INTERACTIONS When you add interactions to a logistic model, no odds ratios are printed. EXAMPLE data today3; input disease race $ sex $ datalines; 0 White F White M White F White M AfrA F AfrA M AfrA F 20 1 AfrA M 28 0 Lat F Lat M 90 1 Lat F 75 1 Lat M 30 0 AIAN F 25 0 AIAN M 15 1 AIAN F 10 1 AIAN M 8 0 Asian F Asian M Asian F 50 1 Asian M 40 0 NHPI F 10 0 NHPI M 8 1 NHPI F 5 1 NHPI M 3 ;;;; run; and then p roc l o g i s t i c d a t a = today3 ; c l a s s r a c e sex / param = r e f ; model d i s e a s e ( e v e n t = 1 ) = r a c e sex r a c e sex ; There are no odds ratios printed. EXPLANATION The reason no odds ratios are printed is a bit complex. Let s take a simpler case, with two independent variables, each with two levels, and, for simplicity, equal cell sizes. If the dependent variable is continuous, then we might get something like this (with DV means in each cell). Male Female Mean White Other Average Now, we can see that White males are lower than White females, but Black males are higher than Black females. The average effect of being male is to add 5, and the average effect of being White is to subtract 10. If there were no interaction, the main effects would give 5

6 Male Female White = = 45 Other = = 65 But if we put proportions in the table and estimate odds ratios, things are trickier. Start with proportions: Male Female Mean White Black Average and it still works just like means. But change to odds: Male Female Mean White 2 to 3 3 t 2 1 to 1 Black 9 to 1 1 to 1 7 to 3 Average 13 to 7 11 to 9 3 to 2 or, dividing through Male Female Mean White Black Average Notice that the average odds is not the average of the odds...e.g ( )/2. What would the main effects model predict? Some people suggest using the CONTRAST statement, but I (and many others) find these confusing and prone to error for interactions. I wasn t able to find a good reference for this method that is, one that explained clearly how to do it. But SAS-L to the rescue We can get what we want with a two step process: First, recode the race and sex variables as numbers d a t a today4 ; s e t t oday3 ; i f r a c e = White t h e n racenum = 0 ; e l s e i f r a c e = AfrA t h e n racenum = 1 ; e l s e i f r a c e = AIAN t h e n racenum = 2 ; e l s e i f r a c e = Asian t h e n racenum = 3 ; e l s e i f r a c e = NHPI t h e n racenum = 4 ; i f sex = M t h e n sexnum = 0 ; e l s e i f sex = F t h e n sexnum = 1 ; r a c e s e x = 100 racenum + sexnum ; Second, write a regular PROC LOGISTIC with the new variable racesex, defining the reference category: p roc l o g i s t i c d a t a = today4 ; c l a s s r a c e s e x ( param = r e f r e f = 100 ) ; model d i s e a s e ( e v e n t = 1 ) = r a c e s e x ; which yields, among other things: 6

7 Odds Ratio Estimates Point 95% Wald Effect Estimate Confidence Limits racesex 0 vs racesex 1 vs racesex 101 vs racesex 200 vs racesex 201 vs racesex 300 vs racesex 301 vs racesex 400 vs racesex 401 vs and we just need to remember the coding we used: 100 is African-American, 0 is male, so this is comparing each group to African-American males. An alternate solution (also gotten through SAS-L) is to use a macro created by Paul Thompson, and presented at SUGI 31: www2.sas.com/proceedings/sugi31/ pdf. SUMMARY PROC LOGISTIC offers many traps for the unwary. In addition to the above, one needs to be careful about Wald vs. likelihood ratio tests (SAS defaults to likelihood ratio tests for the confidence intervals and Wald tests for the parameter estimates. For more on these see Long [1997]. In addition to the gotchas presented here, there are the usual issues of model building, variable selection, model interpretation and so on. For more on these issues see: Hosmer and Lemeshow [2000], Long [1997], Allison [1999, 2008] REFERENCES D. W. Hosmer and S. Lemeshow. Applied logistic regression. John Wiley & Sons, New York, 2nd edition, Paul D. Allison. Convergence failures in logistic regression. In SAS Global Forum, number 360, J. S. Long. Regression models of categorical and limited dependent variables. Sage, Thousand Oaks, CA, P. D. Allison. Logistic regression using the SAS system: Theory and application. SAS Institute, Cary, NC,

8 ACKNOWLEDGMENTS CONTACT INFORMATION Peter L. Flom 515 West End Ave New York, NY Phone: (917) Personal webpage: SAS R and all other SAS Institute Inc., product or service names are registered trademarks ore trademarks of SAS Institute Inc., in the USA and other countries. R indicates USA registration. Other brand names and product names are registered trademarks or trademarks of their respective companies. 8