Chapter 8 Regression Wisdom

Size: px

Start display at page:

Download "Chapter 8 Regression Wisdom"

Barnard Grant
7 years ago
Views:

1 Chapter 8 Regression Wisdom 1

2 8.1 Examining Residuals 2

3 Why Examine the Residuals? Looking at residuals is easiest way to check for linearity: the Straight Enough Condition Residuals reveal subtleties not apparent from the scatterplot. Residuals help us focus on the appropriateness of the regression line. 3

4 Without Looking at the Residuals Analysis R 2 = 71.5% Moderately strong negative association ŷ = x On average, for each minute the penguin is under water, its heart rate declines by 5.47 bpm. The predicted heart rate before the dive is 96.9 bpm. 4

5 Examining the Residuals What the Residuals Tell Us There is a clear bend in the residual plot. It is more spread out for low durations and less for high durations. The residual plot suggests a re-expression may be needed to straighten out the data. 5

6 Examining the Histogram of the Residuals What Does the Histogram Tell Us? Close to Normal but: Cluster of outliers on the left Cluster of outliers on the right 6

7 Re-examining the Residual Plot High Residual Cereals Just Right Fruit & Nut Muesli Raisins, Dates and Almonds Mueslix Crispy Blend Nutri-Grain Almond Raisin All healthy cereals It might be better to put healthy cereals in a separate group and analyze them separately. Similarly, the lower group may need separating out. 7

8 Subsets This scatterplot shows the cereal boxes separated out by shelf: One line for each shelf. 8

9 8.2 Extrapolation: Reaching Beyond the Data 9

10 Predicting Gas Prices The data clearly follows a linear model. Can we predict the price of oil for the year 18 (1988)? (18) $ How good is this prediction? 10

11 Beware of Long Term Predictions Actual vs. Predicted Predicted for 1988: $ Actual $24 What went wrong? The regression line may not be a good predictor for values that are far from the collected data values, especially when x represents the date. 11

Age at First Marriage Discuss the Model for 120 (2010) 25.7 0.04(120) = 20.

12 Age at First Marriage Discuss the Model for 120 (2010) (120) = 20.9 years old The actual median age at first marriage was What went wrong? The regression model only works until year 50 (1940). 12

13 Caution: Predicting Changes Can be Very Wrong Very Wrong: Based on the regression equation above that relates grams of sugar in a serving of cereal, adding a gram of sugar to your cereal leads to an additional 2.50 calories. Fact: Sugar contains 3.90 calories per gram. Better: Cereals that have one more gram of sugar tend to have about 2.50 more calories per serving. 13

14 8.3 Outliers, Leverage and Influence 14

15 The Election of 2000 Buchanan and Nader in Each County of Florida R 2 = 42.8% The regression seems reasonable?? Always draw a scatterplot! Investigate the outlier: In Palm Beach County, the ballot was confusing. 15

16 With and Without the Outlier The blue line shows the regression without the outlier. With the outlier: R 2 = 43% Without the outlier: R 2 = 82% Is Palm Beach a Buchanan stronghold? Or did the ballots trick the electorate from Gore to Buchanan? The latter means Gore should have been president. 16

17 Leverage and Influential Points A data point whose x-value is far from the mean of the rest of the x-values is said to have high leverage. Leverage points have the potential to pull strongly on the regression line. A point is influential if omitting it from the analysis changes the model enough to make a meaningful difference. Influence is determined by 1. The residual 2. The leverage 17

18 Shoe Size and IQ: Bozo the Genius Clown Model that Includes Bozo Almost all the variation accounted for by the model is from one point. After removing the outlier R 2 = 0.7% Bozo is an influential point. What if Bozo had an IQ of 50? The slope would go from 0.96 IQ point/shoe size to

19 8.4 Lurking Variables and Causation 19

20 Two Examples of Regression Doctors and Life Expectancy There is a linear relationship between the square root of Doctors per person and Life Expectancy. Should we send doctors to countries with low life expectancies? 20

21 Two Examples of Regression TV and Life Expectancy There is a stronger linear relationship between the square root of TVs per person and Life Expectancy. Should we send even more TVs to countries with low life expectancies? 21

22 Lurking Variables Although both examples showed a positive linear relationship, this does not mean that if we send doctors or TVs to countries with low life expectancies, then their life expectancies will increase. There could be a lurking variable such as standard of living. Don t confuse correlation with causation. 22

23 8.5 Working with Summary Values 23

24 The Effect of Taking the Mean Scatterplots of the mean show less variability. Plotting the mean could can give the false impression of a stronger correlation. There is no standard correction for this. 24

There is a strong positive association between Displacement and

25 Using Regression Relationship Between Engine Displacement and Price What does the scatterplot show? There is a strong positive association between Displacement and MSRP. There is an outlier: the high-end hand-made Husqvarna TE 510 dirt bike at $19,

26 How to Make Sense of Information How will removing the outlier affect the slope, R 2, and s e? The TE 510 was an influential point pulling the model upward. Without the TE 510: The slope will decrease. R 2 will increase. s e will decrease. 26

27 Residuals Without the Outlier What does the residual plot tell us? The bikes with high displacement engines do not fit in with the rest of the data. There is a bend in the relationship. The linear model may not be appropriate. Try a re-expression. 27

28 Conclusions A cube root re-expression Displacement units are cubic centimeters. Try a cube root transformation. Also, separate liquid-cooled from air-cooled engines. Positive linear relationship between the cube root of displacement and MSRP 28

29 Conclusions Continued Larger bikes tend to be pricier. Liquid-cooled engines tend to be pricier than aircooled engines with the same displacement. A few liquid-cooled bikes appear to be less expensive. 29

30 What Can Go Wrong? Make Sure the Relationship is Straight: Examine the residuals, especially the most extreme ones. Look for Different Groups: Consider fitting a different linear model to each group. Beware of Extrapolating: Don t predict y-values for x-values that are far from the other x-values, especially when x represents time. Look for Unusual Points: Use the scatterplot to find high leverage and influential points. Use a residual scatterplot to find large residual points. 30

31 What Can Go Wrong? Beware of High Leverage Points, Especially Influential Points: Influential points can dominate. Consider Comparing Two Regressions: Study the regression with and without the influential point. Treat Unusual Points Honestly: Don t just keep removing points until you are satisfied with the model. Just admit failure. 31

32 What Can Go Wrong? Beware of Lurking Variables: A strong linear relationship does not mean that x causes y. Use good reasoning and science to determine the true cause. It may be a lurking variable. Be Cautious of Summary Data: Statistics such as the mean and median tend to inflate the impression of the strength of the linear relationship. 32

Chapter 10. Key Ideas Correlation, Correlation Coefficient (r),

Chapter 10. Key Ideas Correlation, Correlation Coefficient (r), Chapter 0 Key Ideas Correlation, Correlation Coefficient (r), Section 0-: Overview We have already explored the basics of describing single variable data sets. However, when two quantitative variables