Statistical Modelling with Stata: Binary Outcomes

Transcription

1 Statistical Modelling with Stata: Binary Outcomes Mark Lunt Arthritis Research UK Centre for Excellence in Epidemiology University of Manchester 22/11/2016

2 Cross-tabulation Exposed Unexposed Total Cases a b a + b Controls c d c + d Total a + c b + d a + b + c + d Simple random sample: fix a + b + c + d Exposure-based sampling: fix a + c and b + d Outcome-based sampling: fix a + b and c + d

3 The χ 2 Test Compares observed to expected numbers in each cell Expected under null hypothesis: no association Works for any of the sampling schemes

4 Measures of Association Relative Risk = a a+c b b+d Risk Difference = Odds Ratio = == a(b + d) b(a + c) a a + c b b + d a c b d == ad cb All obtained with cs disease exposure[, or] Only Odds ratio valid with outcome based sampling

5 Crosstabulation in stata. cs back_p sex, or sex Exposed Unexposed Total Cases Noncases Total Risk Point estimate [95% Conf. Interval] Risk difference Risk ratio Attr. frac. ex Attr. frac. pop Odds ratio (Cornfield) chi2(1) = Pr>chi2 =

6 Limitations of Tabulation No continuous predictors Limited numbers of categorical predictors

7 Introduction Logistic Other GLM s for Binary Outcomes Linear and Binary Outcomes Can t use linear regression with binary outcomes Distribution is not normal Limited range of sensible predicted values Changing parameter estimation to allow for non-normal distribution is straightforward Need to limit range of predicted values

8 Example: CHD and Age Introduction Logistic Other GLM s for Binary Outcomes chd age

9 Example: CHD by Age group Introduction Logistic Other GLM s for Binary Outcomes Proportion of subjects with CHD Mean age

10 Example: CHD by Age - Linear Fit Introduction Logistic Other GLM s for Binary Outcomes Proportion of subjects with CHD Fitted values

11 Introduction Logistic Other GLM s for Binary Outcomes Linear Model Y = β 0 + β 1 x β p x p + ε ε is normally distributed Generalized Linear Model g(y ) = β 0 + β 1 x β p x p + ε ε has a known distribution

12 Probabilities and Odds Introduction Logistic Other GLM s for Binary Outcomes Probability Odds p Ω = p/(1 p) 0.1 = 1/10 0.1/0.9 = 1:9 = = 1/2 0.5/0.5 = 1:1 = = 9/10 0.1/0.9 = 9:1 = 9

13 Probabilities and Odds Introduction Logistic Other GLM s for Binary Outcomes Proportion Log odds

14 Advantage of the Odds Scale Introduction Logistic Other GLM s for Binary Outcomes Just a different scale for measuring probabilities Any odds from 0 to corresponds to a probability Any log odds from to corresponds to a probability Shape of curve commonly fits data

15 The binomial distribution Introduction Logistic Other GLM s for Binary Outcomes Outcome can be either 0 or 1 Has one parameter: the probability that the outcome is 1 Assumes observations are independent

16 The Logistic Equation Introduction Logistic Other GLM s for Binary Outcomes ( ) ˆπ log 1 ˆπ = β 0 + β 1 x β p x p Y Binomial(ˆπ) Y has a binomial distribution with parameter π ˆπ is the predicted probability that Y = 1

17 Parameter Interpretation Introduction Logistic Other GLM s for Binary Outcomes When x i increases by 1, log (ˆπ/(1 ˆπ)) increases by β i Therefore ˆπ/(1 ˆπ) increases by a factor e β i For a dichotomous predictor, this is exactly the odds ratio we met earlier. For a continuous predictor, the odds increase by a factor of e β i for each unit increase in the predictor

18 Odds Ratios and Relative Risks Introduction Logistic Other GLM s for Binary Outcomes Proportion Odds Proportion

19 Logistic in Stata Introduction Logistic Other GLM s for Binary Outcomes. logistic chd age Logistic regression Number of obs = 100 LR chi2(1) = Prob > chi2 = Log likelihood = Pseudo R2 = chd Odds Ratio Std. Err. z P> z [95% Conf. Interval] age

20 Predict Cross-tabulation Introduction Logistic Other GLM s for Binary Outcomes Lots of options for the predict command p gives the predicted probability for each subject xb gives the linear predictor (i.e. the log of the odds) for each subject

21 Plot of probability against age Introduction Logistic Other GLM s for Binary Outcomes Pr(chd) Proportion of subject in each ageband with CHD

22 Plot of log-odds against age Introduction Logistic Other GLM s for Binary Outcomes Linear prediction age

23 Other Models for Binary Outcomes Introduction Logistic Other GLM s for Binary Outcomes Can use any function that maps (, ) to (0, 1) Probit Model Complementary log-log Parameters lack interpretation

24 The Log-Binomial Model Introduction Logistic Other GLM s for Binary Outcomes Models log(π) rather than log(π/(1 π)) Gives relative risk rather than odds ratio Can produce predicted values greater than 1 May not fit the data as well Stata command: glm varlist, family(binomal) link(log) If association between log(π) and predictor non-linear, lose simple interpretation.

25 Log-binomial model example Introduction Logistic Other GLM s for Binary Outcomes logistic predictions Proportion of subjects with CHD log binomial predictions

26 Logistic Goodness of Fit Influential Observations Poorly fitted observations Separation Goodness of Fit Influential Observations Poorly fitted Observations

27 Problems with R 2 Cross-tabulation Goodness of Fit Influential Observations Poorly fitted observations Separation Multiple definitions Lack of interpretability Low values Can predict P(Y = 1) perfectly, not predict Y well at all if P(Y = 1) 0.5.

28 Hosmer-Lemeshow test Goodness of Fit Influential Observations Poorly fitted observations Separation Very like χ 2 test Divide subjects into groups Compare observed and expected numbers in each group Want to see a non-significant result Command used is estat gof

29 Hosmer-Lemeshow test example Goodness of Fit Influential Observations Poorly fitted observations Separation. estat gof, group(5) table Logistic model for chd, goodness-of-fit test (Table collapsed on quantiles of estimated probabilities) Group Prob Obs_1 Exp_1 Obs_0 Exp_0 Total number of observations = 100 number of groups = 5 Hosmer-Lemeshow chi2(3) = 0.05 Prob > chi2 =

30 Sensitivity and Specificity Goodness of Fit Influential Observations Poorly fitted observations Separation Test +ve Test -ve Total Cases a b a + b Controls c d c + d Total a + c b + d a + b + c + d Sensitivity: Probability that a case classified as positive a/(a + b) Specificity: Probability that a non-case classified as negative d/(c + d)

31 Goodness of Fit Influential Observations Poorly fitted observations Separation Sensitivity and Specificity in Logistic Sensitivity and specificity can only be used with a single dichotomous classification. Logistic regression gives a probability, not a classification Can define your own threshold for use with logistic regression Commonly choose 50% probability of being a case Can choose any probability: sensitivity and specificity will vary Why not try every possible threshold and compare results: ROC curve

32 ROC Curves Cross-tabulation Goodness of Fit Influential Observations Poorly fitted observations Separation Shows how sensitivity varies with changing specificity Larger area under the curve = better Maximum = 1 Tossing a coin would give 0.5 Command used is lroc

33 ROC Example Cross-tabulation Goodness of Fit Influential Observations Poorly fitted observations Separation Sensitivity Specificity Area under ROC curve =

34 Influential Observations Goodness of Fit Influential Observations Poorly fitted observations Separation Residuals less useful in logistic regression than linear Can only take the values 1 ˆπ or ˆπ. Leverage does not translate to logistic regression model ˆβ i measures effect of i th observation on parameters Obtained from dbeta option to predict command Plot against ˆπ to reveal influential observations

35 Plot of ˆβ i against ˆπ Cross-tabulation Goodness of Fit Influential Observations Poorly fitted observations Separation Pregibon s dbeta Pr(chd)

36 Goodness of Fit Influential Observations Poorly fitted observations Separation Effect of removing influential observation. logistic chd age if dbeta < 0.2 Logistic regression Number of obs = 98 LR chi2(1) = Prob > chi2 = Log likelihood = Pseudo R2 = chd Odds Ratio Std. Err. z P> z [95% Conf. Interval] age

37 Poorly fitted observations Goodness of Fit Influential Observations Poorly fitted observations Separation Can be identified by residuals Deviance residuals: predict varname, ddeviance χ 2 residuals: predict varname, dx2 Not influential: omitting them will not change conclusions May need to explain fit is poor in particular area Plot residuals against predicted probability, look for outliers

38 Separation Cross-tabulation Goodness of Fit Influential Observations Poorly fitted observations Separation Need at least one case and one control in each subgroup If you have lots of subgroups, this may not be true In which case, log(or) for that group is or Stata will drop all subjects from that group (unless you use the option asis) Not a problem with continuous predictors