Lab 11: Simple Linear Regression

Transcription

1 Lab 11: Simple Linear Regression Objective: In this lab, you will examine relationships between two quantitative variables using a graphical tool called a scatterplot. You will interpret scatterplots in terms of form, direction, and strength of the relationship, and use it to assess the appropriateness of using a simple linear regression model to describe the relationship between the two variables. If appropriate, you can then perform a simple linear regression analysis to produce an estimated model that can be used to predict the value of the response y for a given value of the predictor x. You will learn how to perform hypothesis tests and compute confidence intervals in regression; including learning how to check the assumptions needed for these inference procedures to be valid Application: Pam believes that the number of Facebook friends a person has could interfere with the number of hours they spends socializing offline. She wants to be able to predict the number of hours a person spends socializing offline for a given number of Facebook friends. She will collect data from her friends and examine the relationship between these two variables. Pam will be able to fit a simple linear regression model if the relationship between these variables is linear. Overview: A regression model describes how the mean of one variable is thought to depend on the value of one or more other variables. If we think the number of Facebook friends may explain changes in the amount of time spent socializing online, we call the number of Facebook friends an explanatory variable (or predictor variable or independent variable) and the amount of time spent socializing offline is called the response variable (or dependent variable). To start, we use a scatterplot to display the relationship between two quantitative variables, plotting the number of Facebook friends on the x- axis and the amount of time spend socializing offline on the y- axis. In examining the relationship, Pam looks at the overall pattern showing the form, which appears to be linear. She notes that the direction of the form is negative and strength of the relationship is moderate. She notes that there are no apparent outliers. One of the many misconceptions about regression arises from the concept of association. Scatterplots can show the association between variables, but Pam should remember that correlation does not imply causation. For example: weekly flu medication sales and weekly sweater sales for an area with extreme seasons would exhibit a positive association because both tend to go up in winter and down in summer. However, neither causes the other. The observed association between two variables is sometimes due to other factors, such as confounding variables. This correlation value, r, explains the strength of the linear relationship between x and y. The correlation can take on values between - 1 and 1. The sign of the correlation also describes the direction of the linear relationship. The correlation between number of Facebook friends and time spent socializing offline is reported to be , which confirms what Pam observed in her scatterplot. The square of this correlation is known as the coefficient of determination and also has an important

2 interpretation. Pam calculates this coefficient to be and she can interpret this value by stating that 34.8% of the variation in the amount of time spent socializing online can be explained by the linear relationship between number of Facebook friends and time spent socializing online. Since Pam s scatterplot suggested that the dependence of amount of time spent socializing online on the number of Facebook friends can be summarized by a straight line, the least squares regression line can be calculated. The least squares regression line is the line that minimizes the sum of the squared vertical distances of the data points to the line hence the name least squares. This fitted line can be used to describe the linear relationship between the amount of time spent socializing offline and the number of Facebook friends and to predict the amount of time spent socializing online for a given number of Facebook friends. The distances from the observed amount of time spend socializing offline to the predicted amount of time spent socializing offline are known as the residuals. These residuals are estimates of the true error terms associated with Pam s model. Pam has fit a simple linear regression model, which is what will also be fit during the lab. Pam s regression is based on a linear model relating the number of Facebook friends to the mean amount of time spend socializing offline as follows: E(time spent socializing offline) = β 0 + β 1 (number of Facebook friends). Here β 0 and β 1 are parameters fixed but unknown constants. Specifically, β 0 is the population y- intercept (the amount of time spent socializing offline when the number of Facebook friends is zero) and β 1 is the population slope (the change in the mean time spent socializing offline for every additional Facebook friend). These two values are unknown, but can be estimated using the least squares criterion. The resulting estimated regression line is generally written as: y ˆ = b0 + b1( x). The estimates, b 0 and b 1 are referred to as the least squares estimates of β 0 and β 1. There are also several assumptions that Pam must check in order for inferences to be valid. First, the time spent socializing offline must have a normal distribution with a mean that varies linearly with the number of Facebook friends and a standard deviation that does not depend on the number of Facebook friends. To check her assumption Pam would create a residual plot: a plot with residuals on the y axis and the number of Facebook friends on the x axis. For this assumption to hold, the residuals be randomly scattered. Pam must also assume that the error terms are normally distributed and are identically distributed. To check this assumption about the normal distribution she will create a QQ plot of the residuals. To check if the errors are identically distributed, she will create a time plot of the residuals and look for stability.

3 Formula Card: Warm- Up: Check Your Understanding For which scatterplot(s) does a linear regression analysis seem appropriate? Plot 1 Plot 2 Plot 3

4 ILP: Using a Scatterplot to Display the Relationship Background: The local library is interested in determining if the time spent watching TV is taking over a young person s reading time. They are interested in predicting the number of books read the past year from the number of minutes spent watching TV on a typical week night. The library took a sample of young patrons and recorded this information. The data set TVandBooks.sav contains the number of books a patron has read yearly (Books) as well as the number of minutes the patron spends watching television on a weekly basis (TV). Task: Is there a relationship between the amount of time spent watching television and the number of books a person has read? Produce a scatterplot to help examine the plausibility of a linear relationship between minutes spent watching television and number of books read. Note: You are interested in predicting the number of books read from the number of minutes spent watching TV. Procedure: We have two variables being measured the number of books read (which is quantitative and playing the role of the response) and the number of minutes spent watching TV (which is also quantitative and plays the role of the explanatory variable). The goal is to assess the relationship between these two quantitative variables. The appropriate inference procedure for this scenario is simple linear regression. 1. Open the data set and produce a scatterplot: Graphs> Legacy Dialogs> Scatter/Dot. Note that your explanatory variable should be on the x- axis (independent), and your response variable should be on the y- axis (dependent). Sketch the general pattern of the plot below. Make sure to label your axes. 2. Based on the scatterplot, does there appear to be a linear relationship between minutes of television watched and the number of books read? Explain. From the scatterplot, we see that the overall pattern is approximately linear with a negative association. A straight line seems like it could approximate the data fairly well. 3. Are there any unusual observations or outliers present? There do not appear to be any unusual observations or outliers present, as almost all of the data points seem to be close to another point and follow a similar pattern. 4. Based on the scatterplot, which direction is the association between these two variables? The association is negative between amount of TV watched and number of books read. 5. Does there appear to be a strong relationship between the minutes of television watched and the number of books read? The scatterplot shows the points clustered around a negative, linear form. Because the points are relatively close to a linear form but still show some variability, the relationship between TV watched and books read appears to be fairly strong.

5 ILP: Describing a Linear Relationship with a Regression Line If we are satisfied that a linear model seems to describe the relationship between the minutes of television watched and the number of books read, we are ready to estimate that model and use it to predict Books on the basis of TV. Task: Fit a linear model to the data. If you have questions about the regression output after the activity, refer to Supplement 8 in this workbook for more details. 1. Obtain the linear regression output using Analyze> Regression> Linear. Make sure to enter the appropriate dependent and independent variables. Report the estimated regression line (predicting equation): _Predicted Books Read Last Year = *(Week Night TV Time (min)) 2. Interpret the estimated slope b 1 in terms of change in number of books read. The estimated slope of 0.15 means that we expect the number of books read last year to decrease by 0.15 books for each unit (1 min) increase in time spent watching TV on a week night, on average. 4. Report the coefficient of correlation, r, between Books and TV: r = _ Report the coefficient of determination, r 2, and interpret it: r 2 = Interpretation: The squared correlation, r 2, is 0.599, which means that 59.9% of the variation in the number of books read last year by a young patron of a library can be accounted for by the linear relationship between number of books read and the time spent watching TV on a weeknight. 6. Use the regression line to predict the yearly number of books read for a person who watches 100 minutes of television weekly. How does it compare to the observed yearly number of books read for the patron who has watched 100 minutes of television weekly? Predicted Number Books = *(TV time) = *(100) = 6.58 books The predicted number of books, 6.58, is fairly close to the actual number of books read by the patron who watches 100 minutes of television weekly, 6 books. However, the predicted and observed values are not the same.

6 ILP: Is There a Significant (Non- Zero) Linear Relationship Between TV Time and Books Read? Is the explanatory variable, time spent watching TV, a useful linear predictor for the number of books read? Is there a significant, non- zero linear relationship between time spent watching television (TV) and number of books read (Books)? Remember that another way to make inferences about the significance of the linear relationship is through a confidence interval for the population slope. Further, recall the basic form of a confidence interval: point estimate ± (a few) standard errors. Most standard computer regression output provides the slope estimate and its standard error, and the few will correspond to a t* value for the corresponding confidence level with degrees of freedom for regression of n 2. Since a confidence interval provides a range of reasonable values for the parameter, it can be used to perform two- sided hypothesis tests by seeing whether the hypothesized value falls in the interval or not. Task: Assess if the explanatory variable TV is significant in the linear model. Hypothesis Test: 1. State the Hypotheses: H 0 :!! = 0 and H a :!! 0, where!! represents: the population slope in the true regression equation for books read from TV time Determine Alpha: We were told to use a significance level of 5%. Remember: Your hypotheses and parameter definition should always be a statement about the population(s) under study. 2. Checking the Assumptions Assumptions: Covered in the next activity. 3. Compute the Test- Statistic and calculate the p- value: Test- Statistic a. Using the computer output, which two test statistics could you use to test these hypotheses? Give the value for each test statistic. t = b 1 /[s.e.(b 1 )] = 6.925; F = MSRegression / MSError = b. Which would not be appropriate for conducting a one- sided version of the alternative hypothesis? The F test statistic Calculate the p- Value: c. What is the reported p- value (for both test statistics)? 0.001

7 4. Evaluate the p- value and Conclusion Evaluate the p- value: What is your decision at a 5% significance level? Reject H 0 Fail to Reject H 0 Remember: Reject H 0 Results statistically significant Fail to Reject H 0 Results not statistically significant Conclusion: What is your conclusion in the context of the problem? We have sufficient evidence that week night TV time is a significant linear predictor of number of books read, i.e. that!! 0. Note: Conclusions should always include a reference to the population parameter(s) of interest. They should not be too strong; you can say that you have sufficient evidence, but do NOT say that we have proven anything true or false. 5. Confidence Intervals (CI): a. Generate the confidence interval for estimating the population slope using Analyze> Regression> Linear. Under Statistics, choose the Confidence intervals, which has a default level of 95%. Give the 95% confidence interval for the population slope. ( , ) b. Provide an interpretation of the resulting interval in context. This interval provides a range of reasonable values for the population slope. We are 95% confident that the population slope value for predicting number of books read using minutes of TV watched lies in the interval ( , ). c. Based on the confidence interval, would you reject the null hypothesis at a 5% significance level? Circle one: Yes No Explain. The 95% confidence interval for!! does not include 0. Did your conclusion here match the one you made in part 4? Yes ILP: Is the Linear Model Appropriate? Are the Assumptions Met for Inference? You will produce and examine the residuals from the regression line as well as create some plots to assess the fit of the linear model. This will also serve to evaluate the validity of the testing and confidence intervals performed in the earlier activities. Regression assumptions may be stated in terms of the response variable or in terms of the error terms. The statistical model for simple linear regression assumes that for each value of x, the observed values of the response are normally distributed with some mean (that may depend on x in a linear way) and a standard deviation σ that does not depend on x. For each x, Y is N(E(Y), σ), where E(Y) = β 0 + β 1 x. Thinking about the error terms, we can say the true error terms (those that we do not observe) are the difference of the response and the true mean (for a given x). These errors are to have a normal distribution with a mean of 0 and a standard deviation of σ (that does not depend on x).

8 Task: Obtain the residuals for the regression of Books on TV, and generate the appropriate plots to check the assumptions of the linear model. 1. Obtain the residuals in SPSS using Analyze> Regression> Linear, but instead of running it right away, click on the Save button. In the box that will open, select Unstandardized under the Residuals heading, and click Continue. This saves the residuals as a new variable. Now, construct a scatterplot of the residuals against the explanatory variable, TV. Sketch the general pattern of the plot. What assumption of the error terms does this plot help assess? The scatterplot of the residuals against the explanatory variable (or residual plot) is useful for checking if the standard deviation does not change with the explanatory variable. We would expect that the residuals scatter around the horizontal line around 0 with no apparent pattern. What conclusion can you draw from this plot? From this plot, we can see that there might be a decrease of standard deviation with larger TV times, but we cannot be certain about this with the small number of observations that we have. There do not appear to be any serious problems, so we can cautiously make this assumption. 2. Construct a Q- Q plot of the residuals. Sketch the general pattern of the plot. What assumption of the error terms does this plot help assess? With the Q- Q plot of the residuals, we are checking the normally distributed errors assumption of the linear model. Based on the plot what is your conclusion about this assumption? From this Q- Q plot, we can say that the normal model seems to be appropriate for the distribution of the errors in the linear model, since the points fall roughly along the 45- degree line.

9 Cool- Down: Think About It 1. Why would you not want to use this model to predict the yearly number of books read for a patron that watches 15 minutes of television weekly? It is usually not a good idea to use a regression equation to predict values far outside the range where the original data fell (extrapolation). All of the patrons observed watched at least 30 minutes of television on a typical weeknight, while this patron would watch 15 minutes. There is no guarantee that the relationship described by the regression will continue beyond the range for which we have observed data. 2. The library plans to report a few prediction intervals to predict the number of books read in a year for patrons who watch a given amount of television per week. What amount of television watched weekly would produce a prediction interval with the narrowest margin of error? The narrowest margin of error for a prediction interval will be found when the television watched weekly is minutes. This is because the margin of error for a prediction interval will only change based on the (!!)! term of the s.e.(fit) calculation. To achieve the narrowest margin of error, we want the smallest s.e.(fit) possible. This will occur when! =!, so that the term is 0. Because the mean TV watched weekly is minutes, we want x=71.43 minutes. 3. Which is the only test statistic you can use if you want to test if the explanatory variable television is a significant negative linear predictor of the number of books read per week? We can only use the t- test statistic, because the F- test statistic produces positive values with no way of separating the contributions to the p- value from the negative or positive linear predictors. Example Exam Question on Regression A doctor wanted to study the relationship between a male s age and HDL (so- called good) cholesterol. She randomly selects 18 male adult patients and records their x = age and y = HDL. A scatterplot of the data indicated an approximately linear relationship overall, so she performs a linear regression analysis using SPSS. Model Summary Model 1 Model 1 Model 1 Regression Residual Total (Constant) Age Adjusted Std. Error of R R Square R Square the Estimate ANOVA Sum of Squares df Mean Square F Sig Unstandardized Coefficients Coefficients Standardized Coefficients B Std. Error Beta t Sig We also have: = = 2883 x and S xx = ( x).131 x.

10 a. What is the correlation between age and HDL cholesterol levels? final answer: r = b. Provide the least square regression line for predicting HDL from age: final answer:! = ! - - OR predicted HDL = (age) c. Based on this model, predict HDL cholesterol for a male who is 30 years old. Predicted y = * 30 = final answer: d. Use a 1% significance level to assess if there is a significant linear relationship between the age and HDL cholesterol for male adults. State the hypotheses to be tested, the observed value of the test statistic, the corresponding p- value, and your decision. Hypotheses: H 0 :!! = 0 H a :!! 0 Test Statistic Value: _t = or F = 8.91 p- value: Decision: (circle) Fail to reject H 0 Reject H 0 e. Calculate a 95% confidence interval for the mean HDL cholesterol level for all 30- year- old male adults. s.e.(fit) = s 1 + n ( x x) 2 ( x x) i 2 = ( ) = 2.61! ±!!.!. (!"#) ± (2.12)(2.61) ± 5.53 (44.12, 55.18) final answer: (44.12, 55.18) f. Consider the residual plot provided below. Does this plot support the conclusion that the linear regression model is appropriate? Yes No Explain: The plot shows a random scatter of points in a horizontal band around 0 with no apparent pattern. This supports that the variance of the of the residuals/the response variable does not change with the explanatory variable Residual Age