Correlation and Regression

Size: px

Start display at page:

Download "Correlation and Regression"

Maximilian Parks
7 years ago
Views:

1 Correlation and PS 360 Introduction to Statistics for the Behavioral Sciences Correlation and Both examine linear (straight line) relationships Both work with a pair of scores, one on each of two variables, and Correlation: Defined as the degree of linear relationship between and Is measured/described by the statistic r : Describes the form or function of the linear relationship between & Is concerned with the prediction of from Forms a prediction equation to predict from Why do we care? Knowing the values of the one most representative score (central tendency) and a measure of spread or dispersion (variability) is critical for DESCRIBING the characteristics of a distribution However, sometimes we are interested in the relationship between variables or to be more precise, how the value of one variable changes when the value of another variable changes. Correlation and regression help us understand these relationships 1

2 Correlation The aspect of the data that we want to describe/measure is the degree of linear relationship between and The statistic r describes/measures the degree of linear relationship between and r=σz z /N, the average product of z scores for and Works with two variables, and -1<r<1, r measures positive or negative relationships Measures only the degree of linear relationship r 2 =proportion of variability in that is explained by r is undefined if or has zero spread Correlation: -1<r<1 The sign of r shows the type of linear relationship between and. We can use the definitional formula for r and these scatterplots to see positive, negative and zero relationships r = 1 r = -1 r = Correlation: -1<r<1 What happens to Variable increases decreases increases decreases What happens to Variable increases decreases decreases increases Type of Correlation Positive Positive Negative Negative Value Positive, ranging from 0 to +1 Positive, ranging from 0 to +1 Negative, ranging from 0 to 1 Negative, ranging from 0 to 1 Example The more time you spend studying, the higher your test score will be The less money you put in the bank, the less interest you will earn The more you exercise, the less you will weigh The less time you take to complete a test, the more you ll get wrong 2

3 Interpreting Correlation Coefficients Size of the correlation coefficient General Interpretation.8 to very strong relationship.6 to.8 strong relationship.4 to.6 moderate relationship.2 to.4 weak relationship.0 to.2 weak or no relationship Correlation: Linear If there is a curvilinear relationship between and, then r will not detect it The value of r will be zero if there is no linear relationship between and r = 0 r = 0 Correlation: r 2 r 2 =proportion of variability in that is explained by If r=, r 2 =.25, so the proportion of variability in that is explained by is.25 (as a percentage, this shows 25% explained by, 75% unexplained) Scatterplots: r=, r 2 =.25 r=.7, r 2 =.49 r=.9, r 2 =.81 Venn Diagrams: r 2 is represented by the proportion of overlap. 3

4 Correlation: Undefined If there is no spread in or, then r is undefined. Note that any z is undefined if the standard deviation is zero, and r=σz z /N. s* =0 s* =0 r is undefined r is undefined Correlation Example of correlation: Murder rates and ice cream sales are positively correlated As murder rates increase, ice cream sales also increase Why? CORRELATION DOES NOT MEAN CAUSATION Murders may cause increases in ice cream sales Ice cream sales may cause more murders Some other variable may cause both murders and ice cream sales Correlation Things to remember: Correlations can range from -1 to +1 The absolute value of the correlation coefficient reflects the strength of the correlation. So a correlation of -.7 is stronger than a correlation of + Do not assign a value judgment to the sign of the correlation. Many students assume that negative correlations are bad and positive correlations are good. This is not true! Population correlation coefficient, ρ (rho) Impact on r: Restriction of range Extreme scores (outliers) 4

5 Not only can we compute the degree to which two variables are related (correlation coefficient), but we can use these correlations as the basis for predicting the value of one variable from the value of another Prediction is an activity that computes future outcomes from present ones. When we want to predict one variable from another, we need to first compute the correlation between the two variables 3.3 Total and First-ear are Correlated First-ear r =.68 for these two variables

6 is concerned with forming a prediction equation to predict from Uses the formula for a straight line, =b+a is the predicted score on the criterion variable b is the slope, b= / =rise/run is a score on the predictor variable a is the -intercept, where the line crosses the axis, the value of when =0 Example: if b=.695, a=.739, and =, then =.695()+.739 = line of on 1 4 Prediction of, given =

7 Linear only Generalize only for values in your sample Actual observed is different from by an amount called error, e, that is, = +e Error in regression is e=- Many different potential regression lines : Best Line There are many different potential regression lines, but only one best-fitting line The statistics b and a are computed so as to minimize the sum of squared errors, Σe 2 =Σ(- ) 2 is a minimum which is called the Least Squares Criterion This means that it minimizes the distance between each individual point and the regression line 1 4 Error in prediction: the distance between each individual data point and the regression line (a direct reflection of the correlation between two variables) Error in prediction = 3.3, =

8 : s y.x Standard error of estimate is a statistic that measures/describes spread of errors or scores in regression s y x is the standard deviation of errors in regression s y x = Σe 2 /(N-2)= Σ(- ) 2 /(N-2). s y x = [(N-1)/(N-2)](s y ) (1-r 2 ) As r 2 increases, s y x decreases. For example, if N=100 and s y =4 r 2 s y x s y x is the standard deviation of around the regression line : Partitioning Partitioning total variability Total = Explained + Not Explained This is true for proportion of spread and amount of spread Proportion: 1 = r 2 + (1-r 2 ) Amount: s 2 y = r2 *s 2 y + (1-r2 )s 2 y Total Explained Not Explained Proportion 1 r 2 1-r 2 Amount s 2 y r 2 *s 2 y (1-r 2 )s 2 y : Partitioning Example: r=.7, s 2 y=150 Total Explained Not Explained Proportion 1 r 2 =.49 1-r 2 = 1 Amount S 2 y = 150 r 2 *s 2 y = 7 (1-r 2 )s 2 y = 76 8

9 Homework Quiz 1 Quiz Compute the correlation between Quiz 1 and Quiz 2 scores using SPSS. Run a regression model predicting Quiz 2 scores using Quiz 1 scores. What is the slope? What is the y-intercept? Based on the regression results, estimate the Quiz 2 score of someone who earned a 9.0 on Quiz 1. 9

Chapter 7: Simple linear regression Learning Objectives

Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) -