Basic Biostatistics in Medical Research. Lecture 3: Introduction to Linear Regression and Logistic Regression

Transcription

1 Basic Biostatistics in Medical Research Lecture 3: Introduction to Linear Regression and Logistic Regression Jungwha Julia Lee, PhD, MPH Biostatistics Collaboration Center (BCC) Department of Preventive Medicine NU Feinberg School of Medicine

2 Outline Multiple linear regression estimating parameters testing hypotheses about parameters confidence and prediction interval Logistic regression odds ratio vs. relative risk interpretation of coefficient etensions of logistic regression 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 2

3 Data Analysis Overview All variables you encounter in data analysis can be classified according as [independent dependent] and [categorical continuous]. 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 3

4 Data Analysis Overview The classification of the data dictates the tool you can use: Categorical Dependent Variable Continuous Independent Variable Categorical Chi-squared test ANOVA Continuous Logistic Regression Linear Regression 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 4

5 Scatter Plots and Correlation A scatter plot is used to show the relationship between two variables. Correlation analysis is used to measure the strength of the association (linear relationship) between two variables. Only concerned with the strength of the relationship No causal effect is implied 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 5

6 Scatter Plot Eamples Linear relationships y Curvilinear relationships y y y

7 Scatter Plot Eamples Strong relationships y Weak relationships y y y

8 Scatter Plot Eamples y No relationships y

9 Correlation Coefficient The population correlation coefficient ρ (rho) measures the strength of the association between the variables The sample correlation coefficient r is an estimate of ρ and is used to measure the strength of the linear relationship in the sample observations 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 9

10 Features of ρ and r Unit free Range: -1 ~ +1 The closer to 1 the stronger negative linear relationship The closer to +1 the stronger positive linear relationship The closer to 0 the weaker the linear relationship 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 10

11 Eamples of r Values y y y r = -1 r = -.6 r = 0 y y r = +.3 r = +1

12 Sample Correlation Coefficient ( )( y y) r [ ( ) 2 ][ ( y y) 2 ] where: r = Sample correlation coefficient n = Sample size = Value of the independent variable y = Value of the dependent variable

13 Sample Correlation Coefficient Eample y r = relatively strong positive linear association between and y

14 Significance Test for Correlation Hypotheses H 0 : ρ = 0 (no correlation) H A : ρ 0 (correlation eists) Test statistic (df=n-2) r t 2 1 r n 2 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 14

15 Regression Analysis Regression analysis is used to: Predict the value of a dependent variable based on the value of at least one independent variable Eplain the impact of changes in an independent variable on the dependent variable 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 15

16 Simple Linear Regression Only one independent variable, Relationship between and y is described by a linear function Changes in y are assumed to be caused by changes in 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 16

17 Types of Regression Models Positive Linear Relationship Relationship NOT Linear Negative Linear Relationship No Relationship

18 Population Linear Regression Model Dependent Variable Population y intercept Population Slope Coefficient Independent Variable y β β 0 1 ε Random Error term, or residual Linear component Random Error component

19 Linear Regression Assumptions y β β 0 1 Error values (ε) are statistically independent Error values are normally distributed for any given value of The probability distribution of the errors is normal The probability distribution of the errors has constant variance The underlying relationship between the variable and the y variable is linear ε

20 Population Linear Regression y Observed Value of y for i y β β 0 1 ε Predicted Value of y for i ε i Random Error for this value Slope = β 1 Intercept = β 0 i

21 Estimated Regression Model The sample regression line provides an estimate of the population regression line Estimated (or predicted) y value Estimate of the regression intercept Estimate of the regression slope ŷ b b i 0 1 Independent variable The individual random error terms e i have a mean of zero

22 Least Squares Criterion b 0 and b 1 are obtained by finding the values of b 0 and b 1 that minimize the sum of the squared residuals e 2 (y ŷ) 2 (y (b 0 b 1 )) 2

23 The Least Squares Equation The formulas for b 1 and b 0 are: b 1 ( 2 ( )( y ) y) algebraic equivalent: y y b n ( ) n b0 y b1

24 Interpretation of Slope and Intercept b 1 is the estimated change in the average value of y as a result of a one-unit change in b 0 is the estimated average value of y when the value of is zero

25 Finding the Least Squares Equation The coefficients b 0 and b 1 will usually be found using computer software, such as Stata or SAS. Other regression measures will also be computed as part of computerbased regression analysis

26 Simple Linear Regression Table 1. Age and systolic blood pressure (SBP) among 33 adult women Age SBP Age SBP Age SBP Source: Colton. Statistics in Medicine (1974) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 26

27 Simple Linear Regression SBP (mmhg) Age (years) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 27

28 Least Squares Regression Properties The sum of the residuals from the least squares regression line is 0 ( ) The sum of the squared residuals is a 2 minimum (minimized ( y yˆ) ) The simple regression line always passes through the mean of the y variable and the mean of the variable The least squares coefficients are unbiased estimates of β 0 and β 1 ( y yˆ) 0

29 Eplained and Uneplained Variation Total variation is made up of two parts: SST SSR SSE Total Sum of Squares Sum of Squares Regression Sum of Squares Error 2 SST (y y) 2 SSR (ŷ y) SSE (y 2 ŷ) where: y = Average value of the dependent variable y = Observed values of the dependent variable ŷ = Estimated value of y for the given value

30 Eplained and Uneplained Variation SST = total sum of squares Measures the variation of the y i values around their mean y SSR = regression sum of squares Eplained variation attributable to the relationship between and y SSE = error sum of squares Variation attributable to factors other than the relationship between and y

31 Eplained and Uneplained Variation y y i y _ y _ SST = (y i - y) 2 SSE = (y i - y i ) 2 _ SSR = (y i - y) 2 y _ y X i

32 Coefficient of Determination, R 2 The coefficient of determination is the portion of the total variation in the dependent variable that is eplained by variation in the independent variable The coefficient of determination is also called R-squared and is denoted as R 2 R 2 SSR SST where 0 R 2 1

33 Coefficient of Determination, R 2 R 2 SSR SST sum of squares eplained by regression total sum of squares Note: In the single independent variable case, the coefficient of determination is where: 2 R r R 2 = Coefficient of determination r = Simple correlation coefficient 2

34 Eamples of R 2 Values y R 2 = 1 y R 2 = 1 Perfect linear relationship between and y: 100% of the variation in y is eplained by variation in R 2 = +1

35 Eamples of R 2 Values y 0 < R 2 < 1 Weaker linear relationship between and y: y Some but not all of the variation in y is eplained by variation in

36 Eamples of R 2 Values y R 2 = 0 No linear relationship between and y: R 2 = 0 The value of Y does not depend on. (None of the variation in y is eplained by variation in )

37 Standard Error of Estimate The standard deviation of the variation of observations around the regression line is estimated by s n SSE k 1 Where SSE = Sum of squares error n = Sample size k = number of independent variables in the model

38 The Standard Deviation of the Regression Slope The standard error of the regression slope coefficient (b 1 ) is estimated by s where: s b1 b 1 s ε ( ) 2 2 s ε ( n ) = Estimate of the standard error of the least squares slope 2 s ε SSE n 2 = Sample standard error of the estimate

39 Comparing Standard Errors y Variation of observed y values from the regression line y Variation in the slope of regression lines from different possible samples small s small s b1 y y large s large s b1

40 Inference about the Slope: t test for a population slope Is there a linear relationship between and y? Null and alternative hypotheses H 0 : β 1 = 0 (no linear relationship) H 1 : β 1 0 (linear relationship does eist) Test statistic (df=n-2) t b1 β s b 1 1 where: t Test b 1 = Sample regression slope coefficient β 1 = Hypothesized slope s b1 = Estimator of the standard error of the slope

41 Inference about the Slope: t Test Estimated Regression Equation: The slope of this model is 1.22 Does age of the house affect SBP?

42 Inference about the Slope: t Test Estimated Regression Equation: Hypothesis: H 0 : 1 0 (slope) H 1 : 1 0 Estimate of Test Statistic t=5.74 P-value < Conclude: 1 0 There is a statistically significant linear trend between age and SBP. 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 42

43 Regression Analysis for Description Confidence Interval Estimate of the Slope (95% CI of ) : b t s 1 /2 d.f. = n - 2 b 1 At 95% level of confidence, the confidence interval for the slope is (0.79, 1.66)

44 Confidence Interval for the Average y, Given Confidence interval estimate for the mean of y given a particular p Size of interval varies according to distance away from mean, ŷ 2 1 (p ) t /2sε 2 n ( )

45 Confidence Interval for an Individual y, Given Confidence interval estimate for an Individual value of y given a particular p ŷ 1 ( ) t s 1 p /2 ε n ( ) 2 2 This etra term adds to the interval width to reflect the added uncertainty for an individual case

46 Interval Estimates for Different Values of y Prediction Interval for an individual y, given p Confidence Interval for the mean of y, given p p

47 Residual Analysis Purposes Eamine for linearity assumption Eamine for constant variance for all levels of Evaluate normal distribution assumption Graphical Analysis of Residuals Can plot residuals vs. Can create histogram of residuals to check for normality

48 Residual Analysis for Linearity y y residuals residuals Not Linear Linear

49 Residual Analysis for Constant Variance y y residuals residuals Non-constant variance Constant variance

50 Multiple Linear Regression y = p p Predicted Response variable Outcome variable Dependent variable Predictor variables Eplanatory variables Covariates Independent variables 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 50

51 Multiple Linear Regression Relation between a continuous variable and a set of k continuous variables y= p p Partial regression coefficients p Amount by which y changes on average when p changes by one unit and all the other p s remain constant Measures association between p and y adjusted for all other p Eample SBP vs. age, weight, height, etc 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 51

52 Logistic Regression

53 Logistic Regression Table 2. Age and signs of coronary heart disease (CHD) Age CHD Age CHD Age CHD Source: Colton. Statistics in Medicine (1974) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 53

54 How to analyze these data? Compare mean age of diseased and nondiseased. Diseased 57.7 years Non-diseased 38.6 years Hypothesis: H 0 : 1 2 vs. H 1 : 1 2 t-test: p< Linear regression? 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 54

55 Scatter plot: Data from Table Signs of coronary heart disease Age (years) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 55

56 Logistic regression Table 3. Prevalence (%) of signs of CHD according to age group Diseased Age group No. in group No. % /25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 56

57 Scatter plot: Data from Table 3 Diseased % Age group 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 57

58 Objective of the Analysis How is a risk factor related to disease or death? To model the relationship between the risk of the outcome and the predictors of interest, where The outcome is binary (Yes vs No): Y=1 or 0 The predictors are X=(X1, X2,, Xp) The risk of the outcome is Pr(Y=1) or E(Y) The statistical model is Pr(Y=1 X) = E(Y X) = ƒ(x) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 58

59 Construction of f(x) Linear regression: E(Y)=ƒ(X)=+X Can we use the same model? e.g., Pr(CHD)= +BMI Pr(CHD) is a probability and thus will always be between 0 and 1 Need to select f(x) so that it is always between 0 and 1 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 59

60 Logistic Regression Model Pr( Y 1 X ) f ( X ) e 1 X e X 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 60

61 Relate a binary outcome to one or more predictors. Describes the magnitude of effects using odds ratios. Let Y=1 for disease, Y=0 for non-disease, Race=1 for white, Race=0 for minority, Age. Logistic Regression 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 61

62 Review : Odds and Odds Ratio White Non-White Disease a b No Disease c d Risk a/(a+c) b/(b+d) Odds a/c b/d Relative Risk (RR) = a b d a c b Odds Ratio (OR ) = Odds for White / Odds for Non-White 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 62

63 Review : Odds and Odds Ratio White Non-White Disease No Disease 12,079 12,890 Risk 130/12, /12,991 Odds 130/12, /12,890 Relative Risk (RR) , ,991 Odds Ratio (OR ) = Odds for White / Odds for Non-White 130 /12, /12, /25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 63

64 More on OR Ranges from 0 to infinity Tends to be skewed (i.e., not symmetric): OR>1: Increased risk when eposed OR=1: no difference in risk btw the two groups OR<1: Lower risk (protective) when eposed Log of OR tends to be symmetric and normally distributed. Log (OR) >0: increased risk Log (OR) =0: no difference in risk Log (OR) <0: decreased risk 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 64

65 Logistic Regression Model Pr( Y e 1 X ) 1 Pr( Y 1 ln Pr( Y 0 X ) X ) X 0 e X ln (Odds Ratio) 1 X ep( β 1) = Odds Ratio 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 65

66 Interpretation of Coefficient Let X=1 if eposed and X=0 if not eposed Let denote the regression coefficient for X, then e is the OR for the outcome, comparing the eposed to the not eposed. e Pr( Y Pr( Y 1 X 1 X 1) / 0) / Pr( Y Pr( Y 0 0 X X 1) 0) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 66

67 Interpretation of Coefficient For continuous eposure measurements, e is the OR for the outcome, corresponding to one unit increase from the previous level. e Pr( Y 1 X 1) / Pr( Y 1 X ) / Pr( Y Pr( Y 0 0 X X 1) ) Which previous level? Any level! 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 67

68 Logistic Regression Eample Applied Logistic Regression (Hosmer & Lemeshow) Outcome: CHD recurrence, Yes vs No Predictor: Age The logistic regression model is Pr (CHD Age) e Age 0 1 e Age /25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 68

69 Logistic Regression Eample id age chd /25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 69

70 Logistic Regression Eample Pˆ r (CHD Age) The estimate of b 1 =0.111 with s.e.=0.024 The p-value for H 0: b 1 =0 is < The 95% conf. interval for b 1 is (0.064, 0.158) OR corresponding to one year increase of age: e b e Age Age with the 95% confidence interval of (e 0.064,e ) = (1.066, 1.171). e b 0 e b b 0 1 b 1 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 70

71 Multiple Logistic Regression How to make adjustment for potential confounders? In linear regression: E(Y X 1,X 2 ) = b 0 +b 1 X+b 2 X 2 In logistic regression: or, 1 ), ( X X X X e e X X Y P ) Pr( 1) Pr( ln logit(p) X X Y Y 10/25/ Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression

72 Multiple Logistic Regression The eponential of the regression coefficient, e b1 can be interpreted as the OR corresponding to one unit increase in X 1, while the level of X 2 is fied at 2. e b 1 Pr( Y 1 X1 Pr( Y 1 X 1 1,, 1 1 X X ) / ) / Pr( Y Pr( Y 0 0 X X ,, X 2 X 2 ) 2 2 ) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 72

73 Multiple Logistic Regression Eample Dependent variable whether the patient has had a second heart attack within 1 year (binary: 1= Yes v.s. 0 = No). Two independent variables: whether the patient completed a treatment consisting of anger control practice (binary: 1=Yes v.s. 0=No). score on a trait aniety scale (a higher score means more anious). 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 73

74 Multiple Logistic Regression Eample: Data ID Heart Attack Treatment Aniety /25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 74

75 Univariate Analysis Result The parameter estimate is with a 95% confidence interval of ( , ). The OR of the second heart attack comparing those received treatment to those did not is with a 95% CI of (0.0448, ). 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 75

76 Multiple Analysis Result The OR for the second heart attack comparing those received treatment to those did not, while controlling for the aniety level, is 0.36 with a 95% CI of [0.04, 3.56] and a p-value of Every unit increase in Aniety score increases the odds of the second heart attack by 12.6% (95% CI: [1.1%, 25.5%], p=0.03) while treatment status remains the same. 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 76

77 Obtaining Risk, Risk Difference, and Relative Risk What if we want to know the risk difference or relative risk since odds ratio is too difficult to eplain? You can still use logistic regression! 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 77

78 Obtaining Risk, Risk Difference, and Relative Risk Pr (Heart Attack) e 1 ( Treatment0.12Aniety) e ( Treatment0.12Aniety) 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 78

79 Obtaining Risk, Risk Difference, and Relative Risk Aniety level = 50: Pr(Heart attack Treatment, 50)=0.19 Pr(Heart attack No Treatment, 50)=0.40 Risk Difference (RD) = = Relative Risk (RR) = 0.19 / 0.40 = 48% Aniety level = 80: Pr(Heart attack Treatment, 80)=0.89 Pr(Heart attack No Treatment, 80)=0.96 RD = = RR = 0.89 / 0.96 = 93% 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 79

80 How about OR? What is the OR for treatment vs. no treatment? Aniety level = 50 Pr(Heart attack Treatment, 50)=0.19 Pr(Heart attack No Treatment, 50)=0.40 OR = (0.19/0.81) / (0.40/0.60) = 0.36 Aniety level = 80 Pr(Heart attack Treatment, 80)=0.89 Pr(Heart attack No Treatment, 80)=0.96 OR = (0.89/0.11) / (0.96/0.04) = 0.36 What a surprise: the same OR??? 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 80

81 Etensions of Logistic Regression Polytomous response Eact method Conditional logistic regression Longitudinal data 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 81

82 Co Proportional Hazards Regression Outcome of interest is amount of time from an initial observation until occurrence of a subsequent event Time from baseline eamination until death Goal of Co proportional hazards regression analysis is to eamine predictors of time to an event 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 82

83 Net Time Statistical Genetics: Classical to Modern 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 83

84 Useful References Intuitive Biostatistics, Harvey Motulsky, Oford University Press, 1995 Highly readable, minimally technical Practical Statistics for Medical Research, Douglas G. Altman, Chapman & Hall, 1991 Readable, not too technical Fundamentals of Biostatistics, Bernard Rosner, Dubury, 2000 Useful reference, somewhat technical 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 84

85 Contact BCC Please go to our web site at: Fill out the online request form for further collaboration. 10/25/2011 Biostatistics Collaboration Center (BCC) Lecture 3: Intro to Linear and Logistic Regression 85