Example: Quadratic Linear Model

Transcription

1 1 of 24 Example: Quadratic Linear Model Gebotys and Roberts (1989) were interested in examining the effects of one variable (i.e. age ) on the seriousness rating of the crime (y); however, they wanted to fit a quadratic curve to the data. The variables to be examined within this example are age and age squared (i.e., agesq ). The model is: E(y x)= B 0 + B 1 age + B 2 age 2 Complete the following steps in the Linear regression utilizing SPSS in order to follow the output explanation starting on page thirteen. 1. Pull up Crime data set. (or enter data below)

2 Before proceeding to any analysis, the new variable agesq needs to be created. To create agesq do the following: 2 of Click Transform on the main menu bar. 2. Click Compute on the Transform menu. You should now see the following box:

3 3 of Type agesq in the Target Variable text box, located at the top left corner of the Compute Variable dialogue box. This is to specify the new variable that you are creating, agesq. 4. To specify the numeric expression of agesq (that is, the squaring of age ), click age in the variable source list and click the arrow button to the right of the variable source list box. Then click the ** button on the lower right-hand side of the onscreen key pad followed by clicking the 2 button on the keypad. As a result of the above clicks, the expression age**2 will be entered into the Numeric Expression box. 5. Click the OK command pushbutton of the dialogue box. You will immediately see that a new variable, agesq, has been entered into the Data Editor. (Note that the variable agesq is set to two decimal points. If you would like to re-set this variable to zero decimal points to be congruent

4 4 of 24 with other variables do the following: (i) Double click on agesq variable label to Define label box ; (ii) See section called Change settings and click Type button; (iii) Change decimal places from 2 to 0 ; (iv) Click button Continue; and finally (v) Click the OK button. Your agesq variable should now contain no decimal places).

5 6. At this stage, you may save the data matrix on a diskette, for example, under the file name crimeage2.sav. 5 of 24 Fitting a Quadratic Regression Curve on the Scatterplot 1. Start by creating a scatterplot for the set of data on age and crime seriousness. 2. To fit a quadratic regression curve, start by clicking on the scatterplot as it appears in the Results pane of the SPSS viewer window so that a thin black line surrounds the scatterplot. Next, click on Edit on the menu bar and select SPSS Chart Object from the Edit menu, followed by clicking Open on the submenu. This series of clicks will open a SPSS Chart Editor window. The scatterplot you have produced will appear in the Chart Editor window.

6 6 of Click Chart on the Chart Editor window menu bar, followed by clicking Options in the Chart menu. When the Scatterplot Options dialogue box is activated, click the check box to the left of the Total in the Fit Line box. Then click the Fit Options button in the Fit Line box. This will open a Scatterplot Options: Fit Line dialogue box as shown below.

7 7 of Click the Quadratic regression box. Click the Continue pushbutton. When the Scatterplot Options dialogue box reappears, click the OK command pushbutton. After a few seconds, you should see that a curve has already been fit to the scatterplot. A copy of the scatterplot is reproduced below. At this stage you can edit, save, and print the scatterplot. 100 Scatterplot of "Serious" vs "Age" Quadratic Line Fit SERIOUS AGE

8 8 of 24 Specifying the Regression Procedure for a Polynomial of Degree 2 Follow the Regression Procedure in Example: The Linear Model with Normal Error. The one exception is that both age and agesq should be defined as the independent variables in this model. This is done by clicking both age and agesq in the variable source list and then clicking the arrow button to the left of the Independent[s: text box. The dialogue box should now resemble the one below.

9 9 of 24 After completing the procedures as specified in the Example: Linear Model with Normal Error (i.e. designate Statistics, Plots and Save selections for this analysis), then click the OK command pushbutton in the Linear Regression dialogue box. This will instruct SPSS to produce a set of output similar to that to be discussed in the next section. Alternate Method to Perform Linear Regression This is performed by utilizing the Run command in a SPSS Syntax window. That is, the selections you have made in the Linear Regression dialogue boxes are actually commands for the regression procedure. You can click the Paste command pushbutton in the Linear Regression dialogue box to paste this underlying command syntax into a Syntax window (i.e., a syntax window will be opened when the Paste command is clicked and the selections that you have made are reproduced in this window in the format of a command syntax or SPSS programming language).

10 10 of 24 There is another way of specifying and running the regression procedure for fitting the quadratic model. If you have already saved the syntax commands in a file (e.g., crime.sps), as outlined in Example: Linear Model with Normal Error, you can specify and run the regression procedure by following the steps described below: 1. Insert the diskette containing the relevant file (e.g., crime.sps) into drive A. 2. Click File on the main menu bar. 3. Click Open in the File menu to activate and open File dialogue box. 4. Check if the current position of the drive (i.e. the Look in text box) is drive A. If not, change it to drive A by clicking on the arrow to the right of the text box and selecting 3.5 Floppy [A:].

11 11 of Check if the File type text box contains SPSS syntax files (*.sps). If not, change it by clicking on the arrow to the right of the text box and selecting that option using a single click. 6. The relevant file (e.g., crime.sps) should now be listed in the large text box in the centre of the Open File dialogue box. Select the file with a single click and then proceed to that file by clicking the Open command pushbutton. This will open a window titled crime SPSS Syntax Editor. Sometimes the only problem is the solution to the problem.that s it! Focus on entering the syntax correctly..then there is no problem, only solutions or in SPSS lingo, output.

12 12 of Go to the end of the line /METHOD=ENTER age and click once to move your cursor/pointer to that location. Add a space after age and then type agesq. Your command syntax should now look like that listed below: If you want to save or print this syntax window, you may do so now. 8. Click Run on the Syntax Editor menu bar, and then click All on the Run menu. This will instruct the computer to perform the required regression procedure.

13 13 of 24 Quadratic Linear Model SPSS Output Explanation The REGRESSION command fits linear models by least squares. The METHOD subcommand tells SPSS what variables are in the model. The DEPENDENT subcommand tells SPSS which is the y variable. The ENTER command defines the x variables. In this case we have asked SPSS to fit the model. E(y x)=b 0 + B 1 x + B 2 x 2 The output, using SPSS windows or syntax, is interpreted as follows: Model 1 Variables Entered/Removed b Variables Variables Entered Removed Method AGESQ, AGE a. Enter a. All requested variables entered. b. Dependent Variable: SERIOUS

14 14 of 24 Model 1 Model Summary b Std. Error Adjusted R of the R R Square Square Estimate Durbin-Watson.988 a a. Predictors: (Constant), AGESQ, AGE b. Dependent Variable: SERIOUS Model 1 Regression Residual Total ANOVA b Sum of Mean Squares df Square F Sig a a. Predictors: (Constant), AGESQ, AGE b. Dependent Variable: SERIOUS Model 1 (Constant) AGE AGESQ Unstandardized Coefficients a. Dependent Variable: SERIOUS Coefficients a Standardi zed Coefficien ts 95% Confidence Interval for B Lower Upper B Std. Error Beta t Sig. Bound Bound E E

15 15 of 24 In order to determine if the model is adequate we examine the ANOVA table. Note the degrees of freedom and F-statistic values. F = which has an F distribution with 2 (number of parameters [3] intercept B 0 [1] = 3 1 = 2) and 7 (number of observations number of parameters = 10 3 = 7) degrees of freedom. We reject H o : B 1 = B 2 = 0 H a : B 1 B 2 0 with p-value less than.0001, the SIGNIF value on the output. The REGRESSION row refers to the model and the RESIDUAL row refers to the error component. The mean square of the residual is equal to s 2, our estimate of sigma 2, note s is also printed in the STD ERROR OF THE ESTIMATE column. s 2 = MSE= s= 3.74

16 16 of 24 In the same area we also have R 2, R SQUARE printed where: R 2 = SSM/SST = 3896/3994 = In other words % of the variance in seriousness is accounted for by the model ( age, agesq ). In the Coefficients section the column model variable lists the variable age, agesq and constant these refer to the variables associated with the parameters B 0, B 1, and B 2 in the model. The column labeled B given the least squares (b 0 = , B 1 = -.085, b 2 =.0126) estimator for B 0, B 1, and B 2. The equation is therefore E(y x) = x +.012x 2.

17 17 of 24 The STD ERROR column is the standard error column for each of the parameters for example s(b 0 ) = s(b 1 ) =.410 s(b 2 ) =.004 the T column gives the corresponding t statistic for testing the hypothesis H 0 : B 1 = 0 H a : B 1 0 T = b 1 /s(b 1 ) = H 0 : B 2 = 0 H a : B 2 0 T = For B 1 the t statistic has the value -.207, and for B 2 the t statistic has the value The column SIG gives the OLS or p-value for the test above. In this case we have p=.842 for B 1 (not significant, therefore we cannot reject H 0 ) and p=.016 for B 2 (significant, therefore we can reject H 0 ); however, p=.016 for B 2, therefore there is strong evidence against H 0. Both are with 7 degrees of freedom.

18 18 of 24 Casewise diagnostics gives a STD RESIDUAL COL with STANDARDIZED VALUES between 3. standard deviations which is reasonable. The Durbin-Watson Statistic is about 2 which indicates zero correlation. The leverage (LEVER) and Cook s distance (COOK D) values for the 10 th observation are relatively large (h 10 =.8977, D = ) indicating this is an influential observation. If we compare h 10 to 2meanh where meanh =.2 (found in the summary statistics section) our suspicion that the 10 th observation (a person 80 year old with a high seriousness rating) is influential is confirmed. Notice that the residual for this observation is small and therefore not an outlier. Casewise Diagnostics a Case Number Std. Residual SERIOUS Predicted Value Residual E a. Dependent Variable: SERIOUS

19 19 of 24 Predicted Value Std. Predicted Value Standard Error of Predicted Value Adjusted Predicted Value Residual Std. Residual Stud. Residual Deleted Residual Stud. Deleted Residual Mahal. Distance Cook's Distance Centered Leverage Value a. Dependent Variable: SERIOUS Residuals Statistics a Std. Minimum Maximum Mean Deviation N E Look we got studentized and other residual statistics without asking for them. Bonus!

20 20 of 24 The histogram of residuals looks reasonable, although with 10 observations, this is difficult to judge. Histogram 3.5 Dependent Variable: SERIOUS Frequency Std. Dev =.88 Mean = 0.00 N = Regression Standardized Residual The probability plot has improved, from the previous Example: Linear Model with Normal Error, in the sense that the residuals more closely approximate a normal distribution. The large bulge present in the

21 21 of 24 normal probability plot of residuals in the Example: Linear Model with Normal Error is no longer present in the polynomial of degree 2 model. Normal P-P Plot of Regression Standardized Residual 1.00 Dependent Variable: SERIOUS Ex pe cte d Cu m Pr ob Observed Cum Prob

22 22 of 24 The plot of y predicted vs e, displays a reasonable band shape as well. Scatterplot Dependent Variable: SERIOUS 1.0 Regression Standardized Residual Regression Standardized Predicted Value Yes! A reasonable band shape for the residuals and predicted values.

23 23 of 24 In a future example, we will learn how to compare the polynomial model discussed in this example, and the linear model (previous example), utilizing the ANOVA technique. Although the polynomial model has a higher R 2 than the line we do not know whether this improvement is statistically significant. In a future example we will learn how to compare these types of nested models. See next page for data table following analysis. Now let s see how the additional variables in the data matrix compare to the syntax command for this analysis?

24 24 of 24 age serious agesq pre_1 res_1 zpr_1 zre_1 coo_1 lev_1 lmci_1 umci_1 lici_1 uici_ January 2000