SPSS Regressions Social Science Research Lab American University, Washington, D.C. Web. www.american.edu/provost/ctrl/pclabs.cfm Tel. x3862 Email. SSRL@American.edu Course Objective This course is designed to give you a basic understanding of how to run regressions in SPSS. Learning Outcomes Learn how to make a scatter plot Learn how to run and interpret simple regressions Learn how to run a multiple regression Note: This tutorial uses the World95.sav dataset, which can be found at our website. Simple Regressions and the Scatter Plot A Simple regression analyzes the relationship between two variables. Scatter plot The relationship between two variables can be portrayed visually using a scatter plot. To create a scatter plot Go to Graphs>Legacy Dialogues>Scatter/dot then choose simple. Plug in your variables. For your dependent variable choose Infant mortality (babymort), and for your independent variable choose Women s literacy (lit_fema). Tip: The dependent variable is displayed on the (Y) vertical axis and the independent variable on the (X) horizontal axis. 1
An SPSS Scatter plot displaying the relationship between infant mortality and women s literacy The relationship between the two variables is estimated as a linear or straight line relationship, defined by the equation: Y = ax + b b is the intercept or the constant and a, the slope. The line is mathematically calculated such that the sum of distances from each observation to the line is minimized. By definition, the slope indicates the change in Y as a result of aunit change in X. The straight line is also called the regression line or the fit line and a is referred to as the regression coefficient. Tip: The method of calculating the regression coefficient (the slope) is called ordinary least squares, or OLS. OLS estimates the slope by minimizing the sum of squared differences between each predicted (ax + b) and the actual value of Y. One reason for squaring these distances is to ensure that all distances are positive. A positive regression coefficient indicates a positive relationship between the variables; the fit line will be upward sloping (see the example on the previous 2
page). A negative regression coefficient indicates a negative relationship between the variables; the fit line will be downward sloping. Testing for Statistical Significance The test of significance of the regression slope is a key test of hypothesis regression analysis that tells us whether the slope a is statistically different from 0. To determine whether the slope equals zero, a t-test is performed (For more information on t-tests see the SPSS bivariate statistics tutorial). As a general rule when observations on the scatter plot lie closely around the fit line, the regression line is more likely to be statistically significant: in other words, it will be more likely that the two variables are positively or negatively related. Fortunately, SPSS calculates the slope the intercept standard error of the slope, and the level at which the slope is statically significant. Let s look at an example: determine whether gas mileage and horsepower are related. Go to Analyze>Regression>Linear SPSS Linear Regression Window Infant Mortality (babymort) is our dependent variable and Female Literacy (lit_fema), our independent variable. Click OK. SPSS will produce a series of 3
tables which tell us about the nature of the relationship between these two variables. In the first table, the R 2, also called the coefficient of determination is very useful. It measures the proportion of the total variation in Y about its mean explained by the regression of Y on X. In this case, our regression explains 70.8 % of the variation of infant mortality. Typically, values of R 2 below 0.2 are considered weak, between 0.2 and 0.4, moderate, and above 0.4, strong. In the second table, we will focus on the F-statistic. By computing this statistic, we test the hypothesis that none of the explanatory variables help explain variation in Y about its mean. The information to pay attention to here is the probability shown as Sig. in the table. If this probability is below 0.05, we conclude that the F-statistic is large enough so that we can reject the hypothesis that none of the explanatory variables help explain variation in Y. This test is like a test of significance of the R 2. 4
Finally, the last table will help us determine whether infant mortality and women s literacy are significantly related, and the direction and strength of their relationship. The first important thing to note is that the sign of the coefficient of Females who read (%) is negative. It confirms our assumption (infant mortality decreases as women s education increases) and our visual analysis of the scatter plot (see page one for the scatter plot). Furthermore, the probability reported in the right column is very low. This implies that the slope a is statistically significant. To be less abstract, let us recall what those coefficients mean: they are the slope and the intercept of the regression line, i.e. Y = -1.129 X + 127.203. What does this mean? It means that when literacy increases by one unit (i.e. 1%), infant mortality on average fall by 1.13 per thousand. In sum, R 2 is high, probabilities are low: WE ARE HAPPY! Multiple Regressions A multiple regression takes what we ve just done and adds several more variables to the mix. It is the right tool whenever you think that your dependent variable is explained by more than one independent variable. In our empirical work, we reasonably assume that gas mileage (Y) is not only explained by horsepower (X1) but also the weight (X2) of the car and its engine displacement (X3). Therefore, we will test the following equation: Y = a0 + a1x1 + a2x2 + a3x3 + error Y is the dependent variable, a0 is the constant, X1, X2, and X3 are the independent variables and their respective coefficients a1, a2, and a3, and the error term reflects all other factors that are not in the model. Scatter Plot Matrix Before performing any statistical work, do not forget to draw scatter plots of every independent variable against the dependent variable. Go to Graphs>Legacy Dialogues>Scatter/dot then choose Matrix Scatter 5
SPSS Scatter plot Matrix Window Lets plug in these variables: babymort, lit_fema and gdp_cap This will produce a matrix of scatter plots which looks like this: 6
A scatter plot like this is helpful for visually detecting relationships between your variables. For instance, you might observe a non-linear relationship between two variables, in which case, you should use different techniques (GDP per capita in general exhibits non-linear relationships with other variables). Correlation Matrix Before starting our multiple regression analysis, it is important to compute the correlation matrix. Go to Analyze>Correlate>Bivariate We select the three independent variables. Then click OK. The following table will show up in the output window. This preliminary step is important because independent variables should NOT be correlated with one another. If independent variables are correlated, this might affect the robustness of our results. In the fascinating world of statistics, this is referred to as the issue of multicollinearity. When we use multiple regression analysis, we attach a weight to any of the independent variables in order to explain the variation in Y. If two independent variables are strongly correlated, it becomes very hard to attach a weight to those variables because they basically convey the same information. As a result, the validity of our empirical work will be greatly affected. In general, 7
assuming two of the independent variables are correlated, the easiest solution is to ignore one of them variables one and to use simply the other one. In this particular case, all variables are strongly correlated with one another. I would recommend you should use only the women s literacy (which we did in the previous section) and not use GDP per capita or daily calorie intake. Let s conduct a multiple regression to explain its basic philosophy, even though our results will be sloppy. Running your Regression To run this regression, go to Analyze>Regression>Linear Select babymort as the dependent variable and calories, lit_fema, and gdp_cap as the independent variables. Click OK. The following tables show up in the output window. The R 2 for our new model is slightly greater than we obtained earlier (.711 vs..811). However, the variation is too large large considering that we added two new variables. Clearly, this is due to the fact that the new independent variables are strongly correlated and ultimately, do not bring much extra information. One of the flaw of the R 2 is that it is sensitive to the number of included independent variables. Specifically, addition of additional independent variables can only increase the R 2. In contrast, the Adjusted R 2 accounts for the number of independent variables. It may rise or fall with the addition of more variables. The Adjusted R 2 is greater than the one obtained in the former section. Therefore, the extra information brought by the new variables is greater than the penalty of adding variables (assuming that we did not encounter the issue of multicollinearity). 8
Finally, as expected from our correlation matrix, we find out that our independent variables are negatively correlated with infant mortality. If we look to the right column we find that our puzzling and strange result for GDP per capita is not statistically significant. This result is an illustration of what happens when there is extensive multicolinearity amongst your independent variables 9