Regression PSYC 381 Statistics Arlo Clark-Foos Regression: Predicting the Future Correlation Regression Examples: Car Insurance Age, Male, Car, Driving History WHO & Avian Flu Spread, Poverty Regression vs. Correlation Regression: Prediction Correlation: Relationship Simple Linear Regression Statistical tool that predicts an individual s score on the DV from the score on one IV Uses a straight line if we know x, we can find y 1
Linear Regression Using z Scores A student who knows they will miss X days What can I tell them about their probable exam grade? Linear Regression Using z Scores z ( r yˆ xy )( z x ) ŷ = y hat (predicted score on variable y) r xy = Correlation between x and y z x = z score for a raw score on variable x Linear Regression Using z Scores Note: Predicted z scores for Y are smaller (i.e., closer to the mean) than the actual z scores for X they are regressing to the mean. 2
Regression to the Mean The tendency of scores that are particularly high or low to drift toward the mean over time Teaching Air Force Training Good and Bad Days Flying Operant Conditioning Reward vs. Punishment Linear Regression Using z Scores Regression to the mean The tendency of scores that are particularly high or low to drift toward the mean over time Predicted z score to predicted raw score z X X z( ) Creating a Regression Line y m( x) b Yˆ a b( X ) a = intercept the value of Y when X = 0 b = slope, the amount of increase in Y for every increase of 1 in X 3
Calculating Intercept (a) 1. Calculate a z score for X = 0 z x 2. Calculate predicted z score for Y z ( r yˆ ( 0 M x) SD )( z 3. Calculate predicted raw score from predicted z score Y xy Yˆ z ( SD ) M x Y x ) Y Calculating Slope (b) Repeat steps for X = 1 Rise y2 y1 Slope Run x x 2 1 How does Y-hat change as X goes from 0 to 1? If positive, then the line goes up to the right. If negative, then the line goes down to the right. Drawing a regression line Calculate several pairs of Y-hat and X, then plot them on your scatter plot and draw a straight line through the points. Standardized Slope (β) When comparing regression equations for variables measured on different scales. β = standardized version of slope in a regression equation (st. deviation (σ)units). β = b SS X SS Y 4
Errors in Prediction Predicting the cost of moving to MI from GA Truck Rental, Gas, Hotels Oops pet fee at hotels, food on the way up, furniture pads for truck Standard Error of the Estimate A statistic indicating the typical distance between regression line and actual data points Effect Size of Regression Proportionate Reduction in Error (r 2 ) AKA: Coefficient of determination Statistic that quantifies how much more accurate our predictions are when we use the regression line instead of the mean as a prediction tool. Goal: How accurate is our regression equation at predicting the future? SS Total Total error we have if we use only the mean to predict SS Total ( Y MY 2 ) 5
SS Total Total error we have if we use only the mean to predict SS Error Total error we have if we use Y-hat from regression equation. SS Error 2 ˆ) ( Y Y SS Error Total error we have if we use Y-hat from regression equation. 6
r 2 ( SSTotal SS SS Total Error ) The amount of variance in DV that is explained by the IV Proportion of variance accounted for Multiple Regression & R 2 Y' i = b 0 + b 1 X 1i + b 2 X 2i Using several variables to predict future scores Orthogonal Variable An IV that makes a separate and distinct contribution in the prediction of a DV Stepwise Multiple Regression Software determines the order in which IVs are included in the regression equation Largest significant r 2 comes first Pros: Good if we have no good theory about our predictions Cons: May ignore nonorthogonal, overlapping, variables implying they are unimportant 7
Hierarchical Multiple Regression Researcher uses theory to determine the order in which IVs are included in the regression equation PSYC 465: Age, Gender, Sleep, Depression Pros: Based on theory so it is less likely to identify bad predictors on accident Cons: Sometimes our theory is lacking 8