3:3 LEC - NONLINEAR REGRESSION

Transcription

1 3:3 LEC - NONLINEAR REGRESSION Not all relationships between predictor and criterion variables are strictly linear. For a linear relationship, a unit change on X produces exactly the same amount of change on Y no matter what the value of X. This defines a straight line (i.e., an equation with a single slope across the entire range of X). But sometimes the change in Y varies (i.e., grows steeper or less steep) across values of X. In studies of forgetting, for example, the amount of information lost may be greatest during the early part of the retention interval and less during later parts of the retention interval. This pattern is represented by the bottom curve (diamonds) in the top figure to the right. Note that the loss between X = 20 and X = 30 (about 10 units on Y) is much greater than the loss between X = 60 and X = 70 (less than 5 units on Y). The top curve (circles) shows a pattern in which more positive change occurs early and less positive change occurs later along the values of X. A study of learning, for example, may show more learning across the first 10 trials than across the last 10 trials. In other situations, the amount of change (increase or decrease) may be greater at higher values of X than at lower values of X. Idealized patterns of this sort are shown in the bottom figure to the right. In the top curve (circles), a negative relationship becomes even more negative. This might occur, for example, in a study of fatigue over time; there may not be much deterioration initially, and more marked loss in performance later. The bottom curve (diamonds) shows a positive relationship that becomes even more positive. In learning how to solve insight problems, for example, people may not improve very much initially, but eventually improve more rapidly with greater experience. Such nonlinear relationships can occur in any area of psychology. There are a number of ways to accommodate nonlinear relationships; we here consider two approaches: polynomial regression and transformations. A third approach based on interaction is presented in the next chapter. Identifying Nonlinear Relationships with Graphs To illustrate analysis of nonlinear relationships, we explore the prediction of crime rates as a function of pctwhite in the USA dataset. The basic relationship is shown in the bottom figure to the right. Considering first the linear relation shown by the solid line, note that the linear fit is quite good (r 2 =.468, recall that this value decreased markedly when other factors were controlled) but produces deviations from predicted values that vary systematically across values of pctwhite. Specifically, residuals for low and high values of pctwhite tend to be negative (i.e., the linear equation over-predicts), whereas residuals for moderate values of pctwhite tend to be positive (i.e., the linear equation under-predicts). For purely linear relationships, deviations above and below the best-fit straight line tend to be evenly distributed across the

2 range of X. We return to the dashed line of fit in a moment. Although plots of Y against X often reveal the nonlinear nature of a relationship, an even stronger visual test results from a plot of residual Y scores from a linear regression of Y on X (i.e., REGRE /DEP = crime /ENTER pctwhite /SAVE RESI(resc.w). Residual crime scores are shown in the plot to the right. A dashed line has been inserted at 0, the mean of the residual scores to make more obvious positive and negative values for the residuals. It is even clearer now, that negative residuals are more common at the extremes of pctwhite and positive residuals more common for intermediate values of the predictor. Adding a nonlinear prediction in the chart editor would strengthen this impression. Polynomial Regression One approach to the analysis of nonlinear relationships is polynomial regression. In polynomial regression, the predictor X is used to generate X 2 (and occasionally, X 3, X 4, and so on) values. The newly generated predictor is included along with X in a multiple regression equation. This allows for a nonlinear relationship (the dashed line in the bottom figure on the previous page is a polynomial fit, called quadratic because it only includes X and X 2 ). The quadratic regression for the crime study is reported below. Note that including pctwhite2 (i.e., X 2 ) along with pctwhite (i.e., X) has markedly increased SS Regression and R 2, resulting in a substantial part r 2 =.149 and a highly significant effect for the pctwhite2 predictor, F = or t = , p =.000. If the relationship was purely linear, none of these values would be substantial. COMPUTE pctwhite2 = pctwhite**2. REGRE /STAT = DEFAU CHANGE /DEP = crime /ENTER pctwhite /ENTER pctwhite2. Model R R Adjusted Std. Error of Change Statistics Square R Square the Estimate R Square Change F Change df1 df2 Sig. F Change 1.684(a) (b) Model Sum of Squares df Mean Square F Sig. 1 Regression (a) Residual Total Regression (b) Residual Total Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) pctwhite (Constant) pctwhite pctwhite Essentially the fit is better because the quadratic equation in model 2 generates predicted scores that follow a curve rather than a straight line. The resulting curve is the dashed line shown previously along with the best-fit straight line. The curve was computed by SPSS in the chart editor, but could also have been generated by saving the predicted values from the best-fit equation in model 2. SPSS also provides a second way to carry out polynomial regression using the CURVE ESTIMATION procedure, as shown below.

3 CURVEFIT /VARIABLES=crime WITH pctwhite /CONSTANT /MODEL=LINEAR QUADRATIC /PRINT ANOVA /PLOT FIT. Dependent variable.. crime Method.. LINEAR Multiple R R Square Adjusted R Square Standard Error DF Sum of Squares Mean Square Regression Residuals F = Signif F =.0000 Variable B SE B Beta T Sig T pctwhite (Constant) Dependent variable.. crime Method.. QUADRATI Multiple R R Square Adjusted R Square Standard Error DF Sum of Squares Mean Square Regression Residuals F = Signif F =.0000 Variable B SE B Beta T Sig T pctwhite pctwhi_ (Constant) Abbreviated Extended Name Name pctwhi_1 pctwhite**2 The CURVEFIT procedure produces two analyses, one for the linear and one for the quadratic equation. Two lines are shown fitted to the data in the requested graph. Transformations of Predictor The second approach to nonlinear relationships that we consider is transformations. In essence the predictor can be transformed (and/or the criterion variable) to make the relationship more linear. This involves stretching out or compressing the predictor and/or criterion variable. We focus on transforming the predictor, as transformations of the dependent variable can be complex with multiple predictors because the specific transformation required for the dependent variable could vary across predictors, yet one can only have one dependent variable. Compressing or stretching the predictor is done by raising the predictor to some power. Powers less than 1 compress X and powers greater than 1 stretch it out. Consider for example the numbers 1, 4, and 9. On the original scale, the distance between 9 and 4 (5 units) is greater than the distance between 4 and 1 (3 units). But if we compress the scale by raising the numbers to a power less than 1 (.5 = square root), we obtain 1.5 = 1, 4.5 = 2, and 9.5 = 3 with equal distances between the previous differences. The upper end has been compressed. Values less than.5 (e.g., ~0 = logarithm, -1 = reciprocal) would compress the upper values even further. Consider next starting with the values 1, 2, and 3. Raising these numbers to a power greater than 1 (e.g.,

4 squaring them) stretches out the upper end so that what were equal distances become larger for the upper values; that is, 1 2 = 1, 2 2 = 4, and 3 2 = 9. The equal difference between 1-2 (1 unit) and 2-3 (1 unit) on the original scale is now 1-4 (3 units) and 4-9 (5 units). X is more spread out at the upper end. Examine the first two graphs in this supplement on nonlinear relationships and try to determine why the first (top) graph requires X to be compressed, whereas the second graph requires X to be stretched out. The crime and pctwhite relationship requires that pctwhite be stretched out. The following analyses examine the effect of different transformations on R 2 (recall that R 2 =.617 for the quadratic regression and R 2 =.468 for the linear regression). Note in the following, that ONLY the transformed variable is used as a predictor, and not the original X (as in quadratic regression). Note also that transformations less than one produce a poorer fit than the linear, while transformations greater than one produce a better fit, although never as good as the quadratic, at least up to the fourth power. The CURVE ESTIMATION procedure also performs and analyzes some transformations. COMPUTE wreciprocal = pctwhite**-1. COMPUTE wlogarithm = lg10(pctwhite). COMPUTE wsquareroot = pctwhite**.5. COMPUTE w2 = pctwhite**2. COMPUTE w3 = pctwhite**3. COMPUTE w4 = pctwhite**4. Model R R Square Adjusted R Std. Error of Square the Estimate REGRE /STAT = R /DEP = crime /ENTER wreciprocal (a) REGRE /STAT = R /DEP = crime /ENTER wlogarithm (a) REGRE /STAT = R /DEP = crime /ENTER wsquareroot (a) REGRE /STAT = R /DEP = crime /ENTER pctwhite (a) <<<<< Original predictor REGRE /STAT = R /DEP = crime /ENTER w (a) REGRE /STAT = R /DEP = crime /ENTER w (a) REGRE /STAT = R /DEP = crime /ENTER w (a) One challenge students sometimes have with transformations is the incorrect belief that there is something sacred or unchangeable about the numerical values of our measures. That is, isn t there something fixed about IQ scores, measures of depression, a person s age that makes it inappropriate to stretch out or compress the scales? But psychologically the answer is no. Consider age, for example. Is it really the case that the aging of mental functions is necessarily the same between 40 and 50 and between 70 and 80 (i.e., 10 years of aging in each case). No, because 10 years might represent greater loss of cognitive functioning between 70 and 80 than between 40 and 50. Indeed, that defines a nonlinear relationship. But given this, it would be perfectly sensible to transform the age variable so that the interval between 70 and 80 was larger than the interval between 40 and 50. Squaring the numbers produces an interval of 1500 between 70 2 and 80 2 versus an interval of 900 between 40 2 and Or consider measuring performance on some cognitive task (e.g., arithmetic). Imagine three people who took 10, 20, and 30 minutes to solve 60 problems. It would appear that the difference between persons 1 and 2 (i.e., 10 minutes) is the same as the difference between persons 2 and 3 (i.e., 10 minutes). But what if we instead use a problems-per-minute (ppm) measure; now we have scores of 60/10 = 6 ppm for person 1, 60/20 = 3 ppm for person 2, and 60/30 = 2 ppm for person 3. Now the difference in scores between persons 1 and 2 (i.e., 3 ppms) is greater than the difference in scores between persons 2 and 3 (1 ppm). Clearly there is nothing to indicate one of these measures is better than the other despite the fact they are not linear equivalents. Similar logic applies to many variables.

5 11/12/2013 Cardiovascular Deaths (CVD) Life Expectancy & Wealth Vehicle Weight & Mileage IQ & National Income Driver Age & Legibility Distance Antagonism & Play in Lemurs 1

6 11/12/2013 Shuttle O-Ring Damage & Temperature Income & Internet Use Age, Kids, & Stress Religion & Attitudes toward Atheists Age & Depression Accidents & Time of Day 2