Residuals. Residuals = ª Department of ISM, University of Alabama, ST 260, M23 Residuals & Minitab. ^ e i = y i - y i

Transcription

1 A continuation of regression analysis Lesson Objectives Continue to build on regression analysis. Learn how residual plots help identify problems with the analysis. M23-1 M23-2 Example 1: continued Case Y Sample of n = 5 students, Y = Weight in pounds, = Height in inches. Prediction equation: Wt ^ = Ht r-square =? Std. error =? To be found later. M23-3 Example 1, continued WEIGHT ^ Y = = distance from point to line, measured parallel to Y- axis HEIGHT M23-4 Calculation: For each case, residual = observed value For the i th case, ^ e i = y i - y i estimated mean M23-5 Example 1, continued Compute the fitted value and residual for the 4 th person in the sample; i.e., = 72 inches, Y = 207 lbs. fitted value = ^y 4 = ( ) residual = e 4 = = = ^ y 4 - y 4 = M23-6 1

2 Residual Plots Scatterplot of residuals vs. the predicted means of Y, Y; ^ or an -variable. M23-7 Example 1, continued WEIGHT ^ Y = e 4 = = distance from point to line, measured parallel to Y- axis HEIGHT M23-8 Example 1, continued Residual Plot Regression line from previous plot is rotated to horizontal. e 4 is the residual for the 4 th case, = HEIGHT M23-9 Residual Plot Scatterplot of residuals versus the predicted means of Y, Y; ^ or an -variable, or Time. Expect random dispersion around a horizontal line at zero. Problems occur if: Unusual patterns Unusual cases M versus Good random pattern, or time M versus Outliers? Next step: to determine if a recording error has occurred., or time M

3 0 versus Nonlinear relationship Next step: Add a quadratic term, or use., or time M versus Next step: Stabilize variance by using. Variance is increasing, or time M23-14 Residual Plots help identify Unusual patterns: Possible curvature in the data. Variances that are not constant as changes. Unusual cases: Outliers High leverage cases Influential cases Three properties of illustrated with some computations. M23-15 M23-16 Y = Weight = Height Property 1. ^ Y = Y Y^ e = Y Y^ Find the sum of the residuals. round-off error Properties of Least Squares Line 1. always sum to zero. Se i = 0. M

4 Y = Weight = Height Y Y^ ^ Y = Property 2. e = Y Y^ e Properties of Least Squares Line 1. always sum to zero. 2. This least squares line produces a smaller Sum of squared residuals than any other straight line can. Se 2 SSE for Find the sum of squares i = SSE = < any other line. of the residuals. M23-20 WEIGHT Y = 68.4, Y = 159 Property HEIGHT M23-21 Properties of Least Squares Line 1. always sum to zero. 2. This least squares line produces a smaller Sum of squared residuals than any other straight line can. 3. Line always passes through the point ( x, y ). M23-22 Illustration of unusual cases: Outliers Leverage Influential Y outlier Unusual point does not follow pattern. It s near the -mean; the entire line pulled toward it. M23-23 M

5 Y outlier Unusual point does not follow pattern. The line is pulled down and twisted slightly. M23-25 Y Unusual point is far from the -mean, but still follows the pattern. High leverage M23-26 Y Unusual point is far from the -mean, but does not follow the pattern. Line really twists! leverage & outlier, influential M23-27 Definitions: Outlier: An unusual y-value relative to the pattern of the other cases. Usually has a large residual. High Leverage Case: An extreme value value relative to the other values. M23-28 Definitions: continued Influential Case has an unusually largeeffect on the slope of the least squares line. M23-29 Definitions: continued Conclusion: High leverage potentially influential. High leverage & Outlier influential!! M

6 Why do we care about identifying unusual cases? The least squares regression line is not resistant to unusual cases. Regression Analysis in Minitab M23-31 M23-32 Lesson Objectives Learn two ways to use Minitabto run a regression analysis. Learn how to read output from Minitab. Can height be predicted using shoe size? Step 1? DTDP M23-33 M

7 Graph Plot Height Scatterplot The scatter for each subpopulation is about the same; i.e., there is constant variance Shoe Size 13 Jitter added in -direction. 14 Female Male 15 M23-35 Stat Method 1 Regression Regression Y = a + b M

8 Copied from Session Window. Regression Analysis: Height versus Shoe Size The regression equation is Height = Shoe Size Predictor Coef SE Coef T P Constant Shoe Siz S = R-Sq = 79.1% R-Sq(adj) = 79.0% Analysis of Variance Source DF SS MS F P Regression Error Total M23-37 Regression Analysis: Height versus Shoe Size The regression equation is Height = Shoe Size Predictor Coef SE Coef T P Constant Shoe Siz S = R-Sq = 79.1% R-Sq(adj) = 79.0% Analysis of Variance Source DF SS MS F P Regression Error Total Least squares estimated coefficients. Total Degrees of Freedom = Number of cases - 1 M

9 Regression Analysis: Height versus Shoe Size The regression equation is Height = Shoe Size R-Sq = SSR Predictor Coef SE Coef T P Constant TSS0.000 Shoe Siz = S = R-Sq = 79.1% R-Sq(adj) = 79.0% Analysis of Variance Source DF SS MS F P Regression Error Total M23-39 Regression Analysis: Height versus Shoe Size The regression equation is Height Standard = 50.5 Error + of 1.87 Regression. Shoe Size Measure of variation around Predictor Coef SE Coef T P Constant the regression line Shoe Siz S = R-Sq = 79.1% R-Sq(adj) = 79.0% Sum of squared residuals Analysis of Variance Source DF SS MS F P Regression Error Error Total MSE S = MSE = 3.8 Mean Squared M

10 5 Versus Shoe Siz (response is Height) Are there any problems visible in this plot? Residual Shoe Siz 15 No Jitter added. M23-41 Least squares regression equation: Height = Shoe r-square = 79.1%, Std. error = inches The two summary measures that should always be given with the equation. M

11 Stat Method 2 Regression Fitted Line Plot Y = a + b This program gives a scatterplot with the regression superimposed on it. M23-43 Regression Plot Height = Shoe Size S = R-Sq = 79.1 % R-Sq(adj) = 79.0 % 80 The fit looks Height Shoe Size M

12 Regression Analysis: Height versus Shoe Size The regression equation is Height = Shoe Size What information do these values provide? Predictor Coef SE Coef T P Constant Shoe Siz S = R-Sq = 79.1% R-Sq(adj) = 79.0% Analysis of Variance Source DF SS MS F P Regression Error Total M23-45 How do you determine if the -variable is a useful predictor? Use the t-statistic or the F-stat. t measures how many standard errors the estimated coefficient is from zero. F = t 2 for simple regression. 1 M

13 How do you determine if the -variable is a useful predictor? 2 A P-value is associated with t and F. The further t and F are from zero, in either direction, the smaller the corresponding P-value will be. P-value: a measure of the likelihood that the true coefficient IS ZERO. M23-47 If the P-value IS SMALL (typically < 0.10 ), then conclude: 3 1. It is unlikely that the true coefficient is really zero, and therefore, 2. The variable IS a useful predictor for the Y variable. Keep the variable! If the P-value is NOT SMALL (i.e., > 0.10 ), then conclude: 1. For all practical purposes the true coefficient MAY BE ZERO; therefore 2. The variable IS NOT a useful predictor of the Y variable. Don t use it. M

14 Could shoe size have a true coefficient that is actually zero? Regression Analysis: Height versus Shoe Size t measures how many standard The regression equation errors is the estimated coefficient Height = Shoe is from Size zero. Predictor Coef SE Coef T P Constant Shoe Siz S = R-Sq = P-value: 79.1% a measure R-Sq(adj) of the = likelihood 79.0% that the true coefficient is zero. Analysis of Variance The P-value for Shoe Size IS SMALL (< 0.10). Conclusion: Source DF SS MS F P Regression Error Total Shoe 256 size IS a useful predictor The shoe size coefficient is NOT zero! of the mean of height. M23-49 The logic just explained is statistical inference. This will be covered in more detail during the last three weeks of the course. M