Correlations & Linear Regressions. Block 3

Transcription

1 Correlations & Linear Regressions Block 3

2 Question You can ask other questions besides Are two conditions different? What relationship or association exists between two or more variables? Positively related: as x goes, so goes y. Negatively related: whatever x does, y does the opposite. No relationship.

3 Example of linear correlation 3000 Advertsing Budget (thousands of pounds) Record S ales (thousand s)

4 Covariance An association is indexed by covariance. Are changes in one variable met with a similar or opposite change in another variable? Variance (s 2 ) = SS/N-1 SS = #( x i " x) 2 We squared the error scores when looking for variance within one variable. If interested in the association between two variables, we multiply the error scores together.

5 Calculating Covariance If deviations from the mean go in the same directions for both variables, you ll get a positive number. If deviations from the mean go in opposite directions (one negative, one positive) you ll get a negative number. cov(x, y) = # ( x i " x) ( y i " y) N "1

6 Interpreting linear relations Correlation coefficient [r] = linear relationship between two variables. r 2 = proportion of common variation in the two variables (strength or magnitude of the relationship. Outliers? A single outlier can greatly influence the strength of a correlation.

7 Effect of outliers One approach to dealing with outliers is to see if they are non-representative (i.e., at the far end of the normal distribution). If so, they should be removed.

8 Conducting the analysis Each variable gets separate column. Create a scatter plot to get visual impression of data. Direction of relationship Strength of relationship Extreme values (outliers), which can greatly influence correlation coefficient.

9 When doing T- tests or anovas, especially repeated measures, each row was data from 1 person. Each observation is its own data row. No collapsing of data

10 Types of correlations Bivariate correlation: between two variable Pearson s correlation coefficient for parametric data (interval or ratio data) Partial correlation: relationship between two variables while controlling the effect of one or more additional variables.

11

12

13

14 Partial Correlations

15

16 Drawing conclusions Correlations only inform us about a relationship between two or more variables. Not able to talk about directionality or causality. An increase in X does not CAUSE an increase in Y or vise versa. Cause could be from unmeasured third variable. We don t know which variable is influencing and which is being influenced.

17 R 2 By squaring our test statistic, we can tell how much of total variance in the data for variable x is accounted for by the relationship with variable y. R 2 = =.056 = 5.6% of variance. (94% of variability still unaccounted for!) For height x age: = = 58%

18 Non-parametric correlations Spearman s Rho Ranks the data and then applies Pearson s equation to ranks. Kendall s Tau Preferred for small data sets with many tied rankings. Biserial correlations: When one variable is dichotomous

19

20 Regressions Correlations detect associations between two variables. Say nothing of causal relationships or directionality Can t predict behavior on one variable given a value behavior for another variable With Regression models we can predict variable Y based on variable X.

21 Simple Linear Regressions A line is fit to the data (similar to the correlations line). Best line is one that produces the smallest sum of squares from regression line to data points. Evaluation based on improvement of prediction relative to using the mean or some other model.

22 Hypothetical Data Predictor variable outcome variable Mean

23 Error from Mean Predictor variable outcome variable Mean

24 Predictor variable Error from regression line outcome variable Regression line Mean

25 Regression Results The best regression line has the lowest sum of squared errors Evaluation of the regression model is achieved via R 2 = tells you % of variance accounted for by the regression line (as with correlations) F = Evaluates improvement of regression line compared to the mean as a model of the data.

26 Simple Linear Regression in SPSS Data input in SPSS as for correlation Only one predictor (IV) and one outcome (DV) allowed. Coefficient table allows you to predict DV for new values of IV.

27

28

29 Correlation between expected y and y Adjusted R 2 = adjusts for bias R 2 proportion of variance accounted for by the regression, biased Is the model an improvement over the mean or over a prior model? β = Change in outcome resulting in change in predictor Tests null hypothesis for relationship between IV an DV

30 Predicting New Values Equation for line: Y - output value X = predictor value β0 = intercept (constant in table. Value of Y without predictors) β1 = slope of line (value for predictor) ε = residual (error) Y = " 0 + " 1 X i + # i

31 Multiple Regression Extends principles of simple linear regression to situation with multiple predictor variables. We seek to find the linear combination of predictors that correlate maximally with the outcome variable. Y = " 0 + " 1 X i + " n X n + # i Predictor 1 Predictor 2

32 Multiple Regression, con t R 2 gives the % variance accounted for by the model consisting of the multiple predictors. T-test tell you independent contribution of each predictor in capturing data.

33 Descriptives and other stats from here

34 R 2 was.33, now.44 Evaluation of mod as a whole Linear relationship of each factor to the dependent variable

35 Logistic Regression If you are instead interested in predicting class membership, or seeing how well your variables predict class membership, then you can do a Logistic Regression. You can use multiple factors as predictors, just like in multiple regression. You can also enter interactions between factors. Categorical and continuous factors can be combined, but you must tell SPSS which factors are continuous.

36 Summary Correlations tell you about the relationship between 2 variables Regressions allow you to predict an outcome variable and to make causal inferences. Logistic regressions good for predicting group membership Next, I ll tell you about some new developments in my world of stats including more sophisticated regression techniques and a method to compare changes to data over time.

37 Informative Websites for spss videos! Or, just google the test you are interested in.