1. ε is normally distributed with a mean of 0 2. the variance, σ 2, is constant 3. All pairs of error terms are uncorrelated

Transcription

1 STAT E-150 Statistical Methods Residual Analysis; Data Transformations The validity of the inference methods (hypothesis testing, confidence intervals, and prediction intervals) depends on the error term, ε, satisfying these assumptions: 1. ε is normally distributed with a mean of 0 2. the variance, σ 2, is constant 3. All pairs of error terms are uncorrelated How can we determine if the assumptions are met? And, what can be done if they are not met? We can estimate the value of the regression residuals for each value of y: y yˆ which is the observed value - the predicted (or expected) value i i Properties of Regression Residuals 1. The mean of the residuals is 0, because (y y) ˆ 0 2. The standard deviation of the residuals = the standard deviation of the fitted regression model, s. Consider this data for the level of cholesterol (in mg/l) and average daily intake of saturated fat (in mg) for a sample of 20 Olympic athletes: Fat (mg) Cholesterol (mg/l)

2 The scatter diagram for this data is shown below. Which would be better, a linear model or a quadratic model? First we can look at the regression models. Here is the linear regression model for this data: ANOVA b Model Sum of Squares df Mean Square F Sig. 1 Regression a Residual Total a. Predictors: (Constant), FAT b. Dependent Variable: CHOLES Coefficients a Unstandardized Coefficients Standardized Coefficients Model B Std. Error Beta t Sig. 1 (Constant) FAT a. Dependent Variable: CHOLES Model Summary b Model R R Square Adjusted R Square Std. Error of the Estimate a

3 ANOVA b Model Sum of Squares df Mean Square F Sig. 1 Regression a Residual Total a. Predictors: (Constant), FAT b. Dependent Variable: CHOLES What is the least-squares model, based on this data? For the observation where x = 1200, What is the expected value of y? What is the observed value of y? What is the value of the residual, ε? 3

4 Here is the table with the residuals included: Fat (mg) Cholesterol (mg/l) Residual and the descriptive statistics for the residuals: Descriptive Statistics N Minimum Maximum Mean Std. Deviation Unstandardized Residual Valid N (listwise) 20 Note that the sum of the residuals is 0. 4

5 How does the quadratic model compare with the linear model? ANOVA Sum of Squares df Mean Square F Sig. Regression Residual Total The independent variable is FAT. Coefficients Unstandardized Coefficients Standardized Coefficients B Std. Error Beta t Sig. FAT FAT ** (Constant) Model Summary Adjusted R Std. Error of the R R Square Square Estimate The independent variable is FAT. Note that there is a problem: the coefficient of the quadratic term is shown as.000 To fix this, double-click on any table that shows the coefficients and then double-click on the coefficient to see more precision: The regression equation is 2 ŷ x x 5

6 Here are the residuals for this model, and the descriptive statistics for the residuals. Note that once again the sum of the residuals is zero. Fat (mg) Cholesterol (mg/l) Residual Residuals Statistics a Minimum Maximum Mean Std. Deviation N Predicted Value Residual Std. Predicted Value Std. Residual a. Dependent Variable: CHOLES 6

7 Here are the residual plots for these two models: Linear Model: Quadratic Model: What do they suggest? Checking the assumption of equal variance We want the error term to have constant variance for all values of the predictor variable(s). This is called homoscedasticity. (Variances without this property are called heteroscedastic.) Look at the residual plots to see if this assumption is satisfied. 7

8 Checking the normality assumption Construct a histogram of the residuals to see if they are normally distributed. Check to see if the histogram is unimodal and symmetric, and also check the Normal Probability Plot. However, regression is robust with regard to nonnormal errors. That is, the regression inference can be considered to be valid even if this assumption is not exactly satisfied. However, if the distribution of the residuals is highly skewed, you may try transforming the data. What if the assumptions for this analysis are not met? For example, what if the scatterplot does not show a linear relationship between the variables? The United Nations Development Programme (UNDP) collects data in the developing world to help countries solve global and national development challenges. One summary measure used by the agency is the Human Development Index (HDI) which attempts to summarize in a single number the progress in health, education, and economics of a country. In 2006 the HDI was as high as for Norway and as low as for Niger. The gross domestic product per capita (GDPPC), by contrast, is often used to summarize the overall economic strength of a country. Is there a relationship between the HDI and the GDPPC? Here is a scatterplot of GDPPC against HDI. Is it appropriate to fit a linear model to this data? Why or why not? 8

9 GDPPC is measured in dollars. Incomes and other economic measures tend to be highly right-skewed. Taking logs often makes the distribution more unimodal and symmetric. Here are histograms of the GDPPC values and the log of those values. How would you describe these distributions? How would you describe the relationship between the HDI and log(gdppc)? 9

10 Here is the SPSS output for this relationship: Model Summary Model R R Square Adjusted R Square Std. Error of the Estimate a a. Predictors: (Constant), HDI ANOVA b Model Sum of Squares df Mean Square F Sig. 1 Regression a Residual Total a. Predictors: (Constant), HDI b. Dependent Variable: loggdppc Coefficients a Unstandardized Coefficients Standardized Coefficients Model B Std. Error Beta t Sig. 1 (Constant) HDI a. Dependent Variable: loggdppc Assess this relationship and write the appropriate regression equation. The new regression equation is 10

11 What do these graphs tell you about this model? Outliers and Influential Observations An outlier is an observation that lies outside the overall pattern for the data. Points that are outliers in the y-direction have large residuals (greater than 3s ), but other outliers may not. An observation is influential if removing it would remarkably change the overall pattern. Points that are outliers in the x-direction are often influential. Influential points draw the regression line toward themselves, and so they cannot be identified by looking for large residuals. It should be noted that not all outliers are influential. 11

12 Assignment 6 Read Chapter 8 through page 407 Hand in the solutions to the following questions. Use SPSS for your analyses; if Minitab or SAS output is shown, use SPSS to reproduce the results. Be sure to paste the SPSS output into your solutions. Ex. 7.20, 8.2*, 8.3, 8.10, 8.13 (try transforming y to y ), 8.19, 8.25, 8.28 * if you decide that this model is not appropriate, explain why, and suggest and test a model that you think would be better. 12