Logistic and Poisson Regression: Modeling Binary and Count Data. Statistics Workshop Mark Seiss, Dept. of Statistics

Transcription

1 Logistic and Poisson Regression: Modeling Binary and Count Data Statistics Workshop Mark Seiss, Dept. of Statistics March 3, 2009

2 Presentation Outline 1. Introduction to Generalized Linear Models 2. Binary Response Data - Logistic Regression Model 3. Count Response Data - Poisson Regression Model 4. Variable Significance Likelihood Ratio Test

3 Reference Material Short Course Presentation and Data from Examples Categorical Data Analysis Alan Agresti Examples found with SAS Code at UCLA Statistical Consulting Website Detailed examples of statistical analysis of data using SAS, SPSS, Stata, R, etc.

4 Generalized Linear Models Generalized linear models (GLM) extend ordinary regression to non-normal response distributions. Model for i = 1 to n Why do we use GLM s? Linear regression assumes that the response is distributed normally GLM s allow for analysis when it is not reasonable to assume the data is distributed normally.

5 Generalized Linear Models Predictor Variables Two Types: Continuous and Categorical Continuous Predictor Variables Examples Time, Grade Point Average, Test Score, etc. Coded with one parameter Categorical Predictor Variables Examples Sex, Political Affiliation, Marital Status, etc. Actual value assigned to Category not important Ex) Sex - Male/Female, M/F, 1/2, 0/1, etc. Coded Differently than continuous variables

6 Generalized Linear Models Categorical Predictor Variables cont. Consider a categorical predictor variable with L categories One category selected as reference category Assignment of Reference Category is arbitrary Variable represented by L-1 dummy variables Model Identifiability Two types of coding Dummy and Effect

7 Generalized Linear Models Summary Generalized Linear Models Continuous and Categorical Predictor Variables

8 Generalized Linear Models Questions/Comments

9 Logistic Regression Consider a binary response variable. Variable with two outcomes One outcome represented by a 1 and the other represented by a 0 Examples: Does the person have a disease? Who is the person voting for? Outcome of a baseball game? Yes or No McCain or Obama Win or loss

10 Logistic Regression Logistic Regression Example Data Set Response Variable > Admission to Grad School (Admit) 0 if admitted, 1 if not admitted Predictor Variables GRE Score (gre) Continuous University Prestige (topnotch) 1 if prestigious, 0 otherwise Grade Point Average (gpa) Continuous

11 Logistic Regression First 10 Observations of the Data Set ADMIT GRE TOPNOTCH GPA

12 Logistic Regression Consider the logistic regression model GLM with binomial random component and logit link g(µ) = logit(µ) Range of values for π(x i ) is 0 to 1

13 Logistic Regression Interpretation of Coefficient β Odds Ratio The odds ratio is a statistic that measures the odds of an event compared to the odds of another event. Say the probability of Event 1 is π 1 and the probability of Event 2 is π 2. Then the odds ratio of Event 1 to Event 2 is: Value of Odds Ratio range from 0 to Infinity Value between 0 and 1 indicate the odds of Event 2 are greater Value between 1 and infinity indicate odds of Event 1 are greater Value equal to 1 indicates events are equally likely

14 Logistic Regression Interpretation of Coefficient β Odds Ratio cont. From our logistic regression model with a single continuous variable, the ratio of the odds of Y=0 for X+1 and X is From our logistic regression model with a single two category variable with effect coding, the ratio of the odds of Y=0 from one category to another is

15 Logistic Regression Single Continuous Predictor Variable - GPA Generalized Linear Model Fit Response: Admit Modeling P(Admit=0) Distribution: Binomial Link: Logit Observations (or Sum Wgts) = 400 Whole Model Test Model -LogLikelihood L-R ChiSquare DF Prob>ChiSq Difference Full Reduced Goodness Of Fit Statistic ChiSquare DF Prob>ChiSq Pearson Deviance

16 Logistic Regression Single Continuous Predictor Variable GPA cont. Effect Tests Source DF L-R ChiSquare Prob>ChiSq GPA Parameter Estimates Term Estimate Std Error L-R ChiSquare Prob>ChiSq Lower CL Upper CL Intercept < GPA Interpretation of the Parameter Estimate: Exp{ } = 2.86 = odds ratio between the odds at x+1 and odds at x for all x The ratio of the odds of being admitted between a person with a 3.0 gpa and 2.0 gpa is equal to 2.86 or equivalently the odds of the person with the 3.0 is 2.86 times the odds of the person with the 2.0.

17 Logistic Regression Single Categorical Predictor Variable Top Notch Generalized Linear Model Fit Response: Admit Modeling P(Admit=0) Distribution: Binomial Link: Logit Observations (or Sum Wgts) = 400 Whole Model Test Model -LogLikelihood L-R ChiSquare DF Prob>ChiSq Difference Full Reduced Goodness Of Fit Statistic ChiSquare DF Prob>ChiSq Pearson Deviance I

18 Logistic Regression Single Categorical Predictor Variable Top Notch cont. Effect Tests Source DF L-R ChiSquare Prob>ChiSq TOPNOTCH Parameter Estimates Term Estimate Std Error L-R ChiSquare Prob>ChiSq Lower CL Upper CL Intercept TOPNOTCH[0] Interpretation of the Parameter Estimate: Exp{2* } = = odds ratio between the odds of admittance for a student at a less prestigous university and the odds of admittance for a student from a more prestigous university. The odds of being admitted from a less prestigous university is.48 times the odds of being admitted from a more prestigous university.

19 Logistic Regression Summary Introduction to the Logistic Regression Model Interpretation of the Parameter Estimates β Odds Ratio

20 Logistic Regression Questions/Comments

21 Poisson Regression Consider a count response variable. Response variable is the number of occurrences in a given time frame. Outcomes equal to 0, 1, 2,. Examples: Number of penalties during a football game. Number of customers shop at a store on a given day. Number of car accidents at an intersection.

22 Poisson Regression Poisson Regression Example Data Set Response Variable > Number of Days Absent Integer Predictor Variables Gender- 1 if Female, 2 if Male Ethnicity 6 Ethnic Categories School 1 if School, 2 if School 2 Math Test Score Continuous Language Test Score Continuous Bilingual Status 4 Bilingual Categories

23 Poisson Regression First 10 Observations from the Poisson Regression Example Data Set GENDER Ethnicity School Math Score Lang. Score Bilingual.status Days Absent

24 Poisson Regression Consider the Poisson log-linear model GLM with Poisson random component and log link g(µ) = log(µ) Predicted response values fall between 0 and +

25 Poisson Regression Interpretation of Coefficient β From our Poisson regression model with a single continuous variable, the relationship between the predicted response at value x and value x+1 is From our Poisson regression model with a single two category variable with effect coding, the relationship between the predicted response from one category to another is

26 Poisson Regression Single Continuous Predictor Variable Math Score Generalized Linear Model Fit Response: number days absent Distribution: Poisson Link: Log Observations (or Sum Wgts) = 316 Whole Model Test Model -LogLikelihood L-R ChiSquare DF Prob>ChiSq Difference <.0001 Full Reduced Goodness Of Fit Statistic ChiSquare DF Prob>ChiSq Pearson Deviance <.0001

27 Poisson Regression Single Continuous Predictor Variable Math Score Effect Tests Source DF L-R ChiSquare Prob>ChiSq ctbs math nce <.0001 Parameter Estimates Term Estimate Std Error L-R ChiSquare Prob>ChiSq Lower CL Upper CL Intercept < ctbs math nce < Interpretation of the parameter estimate: Exp{ } =.98 = multiplicative effect on the expected number of days absent for an increase of 1 in the Math Score Fabricated Example If a student is expected to miss 5 days with a math score of 50, then another student with a math score of 51 is expected to miss 5*.98 = 4.9 days

28 Poisson Regression Single Continuous Predictor Variable Gender Generalized Linear Model Fit Response: number days absent Distribution: Poisson Link: Log Observations (or Sum Wgts) = 316 Whole Model Test Model -LogLikelihood L-R ChiSquare DF Prob>ChiSq Difference <.0001 Full Reduced Goodness Of Fit Statistic ChiSquare DF Prob>ChiSq Pearson Deviance <.0001

29 Poisson Regression Single Continuous Predictor Variable Gender Effect Tests Source DF L-R ChiSquare Prob>ChiSq GENDER <.0001 Parameter Estimates Term Estimate Std Error L-R ChiSquare Prob>ChiSq Lower CL Upper CL Intercept GENDER[1] < Interpretation of the parameter estimate: Exp{2*0.1586} = = multiplicative effect on the expected number of days absent of being female rather than male If a male student is expected to miss X days, then a female student is expected to miss *X.

30 Poisson Regression Summary Introduction to the Poisson Regression Model Interpretation of β

31 Likelihood Ratio Test Deviance Let L(µ y) = maximum of the log likelihood for the model L(y y) = maximum of the log likelihood for the saturated model Deviance = D(y µ) = -2 [L(µ y) - L(y y) ] Tests the null hypothesis that the model is a good alternative to the observed values Deviance has an asymptotic chi-squared distribution with N p degrees of freedom, where p is the number of parameters in the model.

32 Likelihood Ratio Test Nested Models Model 1 - model with p predictor variables {X 1, X 2, X 3,.,X p } and vector of fitted values µ 1 Model 2 - model with q<p predictor variables {X 1, X 2, X 3,.,X q } and vector of fitted values µ 2 Model 2 is nested within Model 1 if all predictor variables found in Model 2 are included in Model 1. i.e. the set of predictor variables in Model 2 are a subset of the set of predictor variables in Model 1 Model 2 is a special case of Model 1 - all the coefficients associated with X p+1, X p+2, X p+3,.,x q are equal to zero

33 Likelihood Ratio Test Likelihood Ratio Test Null Hypothesis: There is not a significant difference between the fit of two models. Null Hypothesis for Nested Models: The predictor variables in Model 1 that are not found in Model 2 are not significant to the model fit. Alternate Hypothesis for Nested Models - The predictor variables in Model 1 that are not found in Model 2 are significant to the model fit. Likelihood Ratio Statistic = -2* [L(y,u 2 )-L(y,u 1 )] = D(y,µ 2 ) - D(y, µ 1 ) Difference of the deviances of the two models Always D(y,µ 2 ) > D(y,µ 1 ) implies LRT > 0 LRT is distributed Chi-Squared with p-q degrees of freedom

34 Likelihood Ratio Test Theoretical Example of Likelihood Ratio Test 3 predictor variables 1 Continuous (X 1 ), 1 Categorical with 4 Categories (X 2, X 3, X 4 ), 1 Categorical with 1 Category (X 5 ) Model 1 - predictor variables {X 1, X 2, X 3, X 4, X 5 } Model 2 - predictor variables {X 1, X 5 } Null Hypothesis Variables with 4 categories is not significant to the model (β 2 = β 3 = β 4 = 0) Alternate Hypothesis - Variable with 4 categories is significant Likelihood Ratio Statistic = D(y,µ 2 ) - D(y, µ 1 ) Difference of the deviance statistics from the two models Chi-Squared Distribution with 5-2=3 degrees of freedom

35 Likelihood Ratio Test Likelihood Ratio Test Consider the model with GPA, GRE, and Top Notch as predictor variables Generalized Linear Model Fit Response: Admit Modeling P(Admit=0) Distribution: Binomial Link: Logit Observations (or Sum Wgts) = 400 Whole Model Test Model -LogLikelihood L-R ChiSquare DF Prob>ChiSq Difference <.0001 Full Reduced Goodness Of Fit Statistic ChiSquare DF Prob>ChiSq Pearson Deviance

36 Likelihood Ratio Test Variable Selection Likelihood Ratio Test cont. Effect Tests Source DF L-R ChiSquare Prob>ChiSq TOPNOTCH GPA GRE Parameter Estimates Term Estimate Std Error L-R ChiSquare Prob>ChiSq Lower CL Upper CL Intercept < TOPNOTCH[0] GPA GRE

37 Likelihood Ratio Test Questions/Comments