Unit 12 Logistic Regression Supplementary Chapter 14 in IPS On CD (Chap 16, 5th ed.)

Transcription

1 Unit 12 Logistic Regression Supplementary Chapter 14 in IPS On CD (Chap 16, 5th ed.) Logistic regression generalizes methods for 2-way tables Adds capability studying several predictors, but Limited to binary response variables Similar in intent to linear regression, but details are different Method for estimating joint association between several predictors and a response variable Typically useful in some class projects 1

2 2

3 Betting in a fair game An American roulette wheel has 38 slots: 1,2,3,, 36, 0, 00 If you place a $1 bet on 00 for a single spin of the wheel, you have 1/38 chance of winning in a single spin 1 way to win, 37 ways to lose, or The casino has 37 ways to win, 1 way to lose The odds of winning for the house are 37 to 1, 1 to 37 for you 3

4 Betting in roulette... For the game to be fair, Casino keeps your $1 if 00 does not come up Casino pays $37 if 00 comes up, and you keep your bet If X is your winnings from a $1 bet, E(X) = -1 (37/38) + 37 (1/38) = 0 Casinos stay in business by paying out 35 to 1, the casinos insure that roulette is not a fair game. In this case E(X) = -1 (37/38) + 35 (1/38) = -(2/38) =

5 Converting probabilities to odds and log(odds) In a game of chance, the odds of winning is the same as the ratio of money that should be bet by the two players. In roulette, the odds of your winning is the ratio of the probability of your winning to the probability of losing p/(1-p) = (1/38) / (37/38) = 1/37 Typically, odds are given to show the ratio of the payout: 37 to 1 in this case The values of an `odds range from 0 to Think of probabilities 0.01, 0.001, , 0.99, 0.999, ,etc We will use a transformation of odds to log(odds) The values of log(odds) range from - to. 5

6 Odds vs log(odds) Transformation of p Why consider such a transformation? Answer: it transforms a 0 < p < 1 variable to a quantitative variable from - to + It is a simple algebraic operation to go back and forth between probabilities and log(odds) Odds Log odds p p/(1-p) log(p/(1-p))

7 Computing odds in data, an example The example on the next slide is very similar to IPS Example 8.1 (5th and 6th edition), but the numbers are from the 5th ed. Be careful when reading the example in the 6th ed. 7

8 Example: binge drinking survey Binger Men Women Total Yes % % 3314 No % Total % % %

9 Logistic Regression Idea behind logistic regression Let ˆp M be the proportion of men who are binge drinkers; log(ˆp M /(1 ˆp M )) is the log odds. Let ˆp F be the proportion of women who are binge drinkers; log(ˆp F /(1 ˆp F )) is the log odds. The ratio of the odds (called the odds ratio) of men to women being binge drinkers is ˆp M 1 ˆp M = ˆp F 1 ˆp F ( ˆpM )( 1 ˆpF 1 ˆp M ˆp F ) = = Now recall log(x/y) = log(x) log(y). 9

10 New page Idea behind logistic regression In the binge drinking table, log [( ˆpM )( 1 ˆpF 1 ˆp M ˆp F )] = log(0.294) log(0.205) = ( 1.587) = The log odds for males differs from the log odds for females by a constant. Logistic regresson is a model in which predictors induce changes in log(odds), similar to linear regression, where Predictors induce changes in mean of response variable. 10

11 Model for Logistic Regression Set the log odds to be a linear combination of the predictor variables This is the Logistic Regression Model Sometimes equivalently written as: 11

12 Logistic regression The Logistic Regression Model Predictor variables (x s) can be quantitative or binary More complex formulas for estimates than least squares Omnibus test of the model now a χ 2, not an F test We can test each predictor variable (x i ) for its contribution but now this is a z test, not a t test Assumptions of this model are quite complex and are not often checked Logistic regression model is widely used Coefficients can be derived directly in some 2-way tables Back to binge drinking example 12

13 Example: binge drinking survey Binger Men Women Total Yes % % % No % Total % % % % % 13

14 The logistic model - binge drinking From the previous slide (odds of being a binge drinker) For men: Log odds = For women: Log odds = The logistic model for one predictor (gender) is Log (p /(1-p)) = Log odds = b0 + b1x1 where Y = 1 if a binge drinker; 0 otherwise and X1 = 1 if male; 0 if female So the logistic model is For men: Log odds = b0 + b1 = For women: Log odds = b0 = Solving b0 = and b1 = b0 = Thus the fitted logistic model for this example is Log odds = X1 14

15 The logistic model - binge drinking Working backwards to confirm this fitted model Log odds = log (p/(1-p)) = X 1 where X 1 = 1 if male and X 1 = 0 if female So for men Log odds = log (p/(1-p)) = (1) = and odds = e = Thus the proportion of binge drinkers is odds / (odds +1) = / = For women Log odds = log (p/(1-p)) = and odds = e = Thus the proportion of binge drinkers is odds / (odds +1) = / =

16 Comparing two proportions Relative risk and odds ratio S F Total Group 1 a b a + b Group 2 c d c + d Total a + c b + d 16

17 Odds ratio As with RR, an odds ratio of 1 indicates the proportion of successes (events) is the same in both groups RR is easier to interpret (ratio of sample proportions) When successes are rare, RR and OR are very similar When successes are common, RR and OR are similar only if they are close to 1 OR tends to overstate differences Example: binge drinking 17

18 Odds ratio and logistic regression Odds ratio is the key output from a logistic regression An OR is calculated for each predictor variable OR measures the strength of the effect on p (probability of `success ) Example: binge drinking Log odds = X 1 where X 1 = 1 if M, 0 if F For men: Log odds = b 0 + b 1 = For women: Log odds = b 0 = Let OR be the odds ratio men to women Log (OR) = log (odds for men) log (odds for women) = (b 0 + b 1 ) (b 0 ) = b 1 So OR = e b 1 For the binge drinking example OR = e =

19 Inference for logistic regression parameters A 95% confidence interval for the coefficient β 1 is given by b 1 ± 1.96 s.e.(b 1 ) A 95% confidence interval for the odds ratio e β 1 is given by e (b 1± 1.96 s.e.(b 1 )) To test the null hypothesis H 0 : β 1 =0(i.e., no association between the response variable and the predictor variable X 1 ), use Z = b 1 s.e.(b 1 ) Z has (approximately) a N(0, 1) distribution when H 0 is true. 19

20 Binge drinking: expanding data from a 2x2 table to a rectangular data file The 2x2 table Let Binge = 1 if binger Let Sex = 1 if male 0 otherwise Stata commands input Binge Sex Count end expand Count Binger Men Women Total Yes No Total This creates a rectangular data file with 17,096 rows: etc. 20

21 Binge drinking logistic regression Stata has 2 commands - logistic and logit logistic displays ORs; logit displays model coefficients Note: b 0 = and b 1 = as earlier 21

22 Binge drinking logistic regression Logistic command displays the odds ratio Notes: OR = as earlier 95% CI for OR (1.33, 1.55) excludes OR = 1 z for the Wald test = 9.31 (P < 0.001) 22

23 Multiple logistic regression Example: Intensive Care Unit (ICU) Study of 200 patients admitted to the adult ICU at Baystate Medical Center in Springfield, MA Response variable Survival until hospital discharge (Surv) Surv = 1 if died, 0 if survived Predictor variables Age, in years (Age) Sex = 1 if female, 0 if male (Sex) Race = 1 if white, 0 otherwise (Race) Heart rate at ICU admission, beats/min. (HRate) Level of consciousness at ICU admission (LOC) LOC = 1 if deep stupor of coma, 0 otherwise Source: Hosmer & Lemeshow (2000) Wiley & Sons 23

24 Density Heart Rate at ICU Admission Density Age Density 0 Density Age Heart Rate at ICU Admission 200. table LOC Level of Consciousness at ICU Admission Freq. No Coma or Deep Stupor 185 Deep Stupor 5 Coma 10. table surv surv Freq

25 Example ICU, summary of the data Response variable - survival to hospital discharge Surv N = 200, 20% died Predictor variables Sex 38% female Age Average is 57.5 yrs., range from 16 to 92 yrs. Race 87.5% white HRate Average is 99, range from 39 to 192 beats/min. LOC 7.5% in deep stupor or coma 25

26 ICU - predictors of death before discharge A logistic regression with all 5 predictors Age and level of consciousness (LOC) both significant Re-estimate the model keeping only the significant terms 26

27 ICU - predictors of death before discharge The final logistic model (using logistic command) Age and LOC are significant Odds ratio For Age (95% CI is to 1.064) P = For LOC (95% CI is 7.63 to ) P <

28 ICU - predictors of death before discharge The final logistic model (using logit command) Shows the logistic model coefficients Log odds = Log (p/1-p) = Age LOC Note: e = (OR) and e 3.59 = (OR for LOC) 28

29 Interpretation of coefficients of the logistic regression model The sign of the β i term indicates whether p increases or decreases as x increases ICU Example: both β i terms were positive so risk of death increases with age and presence of deep stupor or coma The magnitude of the β i term gives the additive change in log odds when there is +1 unit change in the predictor variable, holding other predictors fixed 29

30 Interpretation of coefficients The magnitude of the odds ratio term (= e βi ) gives the multiplicative change in odds for +1 change in predictor ICU Example: the odds of death increases multiplicatively by 2.8% (OR = 1.028) for each year increase in age To see this, exponentiate both sides of logistic model and note (p/1-p) = e β0 + β1[x + 1] = (e β0 )(e β1x )(e β1 ), where e β1 = OR 30

31 Final Thoughts on Logistic Regression Some of you will find logistic regression useful in a project, so last p-set has logistic regression problem. Not covered on final exam, because we have not had time to digest it. Logistic regression extends the analysis of two-way tables Response variable must still be binary. Predictors can now be categorical or quantitative. Logistic regression is an example of a class of regression models much more general than linear regression. These models are covered in detail in Stat 138 and Stat