Correlations. MSc Module 6: Introduction to Quantitative Research Methods Kenneth Benoit. March 18, 2010

Transcription

1 Correlations MSc Module 6: Introduction to Quantitative Research Methods Kenneth Benoit March 18, 2010

2 Relationships between variables In previous weeks, we have been concerned with describing variables the same as making descriptive inferences about population parameters Last week we focused on how to do this by comparing two groups For the next two weeks we will focus on a different aspect: estimating the nature of the relationship between two variables Typically these are continuous variables, but not necessarily By convention, we note these as X and Y and the variables have a specific association that has to do with which one is a dependent and which one an independent variable

3 Independent and Dependent variables A dependent variable represents the quantity we wish to explain variation in, or the thing we are trying to explain Typical examples of a dependent variable in political science: votes received by a governing party support for a referendum result like Lisbon party support for European integration An independent variable represents a quantity whose variation will be used to explain variation in the dependent variable Typical examples of independent variables in political science: democraphic: gender, national background, age economic: socioeconomic status, income, national wealth political: party affiliation of one s parents institutional: district magnitude, electoral system, presidential v. parliamentary behavioural: campaign spending levels Using this language implies causality: X Y

4 Correlations The central idea behind correlation is that two variables have a relationship such that there is a systematic relationship between the their values This relationship can be positive or negative This relationship varies in strength Bivariate associations are usually depicted graphically by use of a scatterplot

5 Scatterplot example Example: correlation between education and income, where the strength of the relationship differs for males versus females

6 Correlations cont. Correlations can be either positive or negative A positive correlation indicates that high X scores tend to be associated with high Y scores A negative correlation exists when high X scores tend to be associated with low Y scores (and low X scores with high Y scores)

7 Different functional relationships Linear A linear relationship, also known as a straight-line relationship, exists if a line drawn through the central tendency of the points is a straight-line Curvilinear Exists if the relationship between variables is not a straight-line function, but is instead curved Example: Television viewing (Y ) as a function of age (X )

8 The correlation coefficient The strength of a correlation can be summarized as a statistic known as a correlation coefficient The correlation coefficient ranges in value from -1.0 to 1.0 Value Correlation perfect negative correlation -.60 strong negative correlation -.30 moderate correlation -.10 weak negative correlation.00 no correlation +.10 weak positive correlation +.30 moderate positive correlation +.60 strong positive correlation perfect positive correlation Two correlation coefficients of the same size but opposite directions have the same strength (e.g , +0.60) The most common correlation coefficient is one called Pearson s r

9 Pearson s correlation coefficient r Question: what direction and strength of association exists between height and weight in this sample? Pearson s r is an exact measure that adds the products of the deviations of each variable from its mean: (X i X ) and (Y i Ȳ ). Positive products will indicate a positive relationship (++), while negative products will indicate a negative relationship (-+) or (+-)

10 Height-weight example D Weight in pounds H B C F G E A Height in inches # scatterplots from p350 plot(x, y, xlab="height in inches", ylab="weight in pounds", pch=19) text(x, y, child, pos=4) # add mean lines abline(v=mean(x), h=mean(y), lty="dashed")

11 Formulas for Pearson s r Basic formula: r = = = i (X i X )(Y i Ȳ ) i (X i X ) 2 (Y i Ȳ )2 Sum of Products Sum of Squaresx Sum of Squares y SP SSx SS y Computational formula (simpler): r = i X iy i N X Ȳ ( i X i 2 N X 2 )( i Y i 2 NȲ 2 )

12 (SSx <- sum(xminusxbar2)) Example from (SSy <- p353 sum(yminusybar2)) LF&F data.frame(child=child, X=x, Y=y, xminusxbar, yminusybar, devproduct, xminusxbar2, yminusybar2) ## define function to compute r from p246 pearsonr <- function(x, y) { xdev <- x - mean(x) ydev <- y - mean(y) sum(xdev*ydev) / sqrt(sum(xdev^2)*sum(ydev^2)) } pearsonr(x,y) pearsonrcomp <- function(x, y) { N <- length(x) (sum(x*y) - N*mean(x)*mean(y)) / sqrt((sum(x^2)-n*mean(x)^2) * (sum(y^2)-n*mean(y)^2)) } pearsonrcomp(x,y) [and show full computations in R]

13 Example from LF&F p353 > ## Example L&F p353 > x <- c(12,10,6,16,8,9,12,11) > y <- c(12,8,12,11,10,8,16,15) > # using our created function > pearsonr(x,y) [1] > # using R s built-in correlation command > cor(x,y) [1]

14 Testing the significance of Pearson s r Pearson s r measures correlation in the sample but the association we are interested in exists in the population Question: what is the probability that any correlation we measure in a sample really exists in the population, and is not merely due to sampling error H 0 : ρ x,y = 0 No correlation exists in the population H A : ρ x,y 0 Significance can be tested by the t-ratio: t = r N 2 1 r 2

15 Pearson s r significance testing: Example From previous example: r =.24, N = 8 So computation of t is: t = (.24) 2 = =.24(2.45) = =.61 From table (or R), critical value for t with df=6, α =.05 is so we do not reject H 0 In R there is a test for this called cor.test(x,y) There is a also a simplified method described in L&F which provides significance tables directly for r with given df, but we will always prefer the R test

16 Pearson s r significance testing: Example in R > # compute the empirical t-value > (t.calc <- (.24*sqrt(8-2) / sqrt(1-.24^2))) [1] > # compute the critical t-value > (t.crit <- qt(1-.05/2, 6)) [1] > # compare > t.calc > t.crit [1] FALSE > # using R s built-in correlation significance test > cor.test(x,y) Pearson s product-moment correlation data: x and y t = , df = 6, p-value = alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: sample estimates: cor

17 Pearson s r significance testing: L&F example pp > ## Final example from LF&F from pp > x <- c(10,3,12,11,6,8,14,9,10,2) > y <- c(1,7,2,3,5,4,1,2,3,10) > cor.test(x,y) Pearson s product-moment correlation data: x and y t = , df = 8, p-value = alternative hypothesis: true correlation is not equal to 0 95 percent confidence interval: sample estimates: cor

18 Requirements for using Pearson s r 1. Linear relationship between X and Y 2. Interval data for X and Y. When only ordinal data exists, there is an alternative called Spearman s ρ. 3. Random sampling of X and Y from the population (in other words, sample must be representative 4. Normally distributed X and Y in the population, at least when N 30 (at N > 30 it becomes of minor importance)

19 Partial correlation Question: Does our correlation hold up when we control for an additional variable? Control here means to consider the influence of an additional variable Example: Relationship between salary and height should not be considered without controlling for the variable of gender

20 Partial correlation example Height and salary only appear related when gender is ignored

21 Relationships given a third variable

22 Partial correlation coefficient The partial correlation coefficient is the correlation between two variables, after removing (or partialling out ) the common effects of a third variable. r XY.Z = r XY r XZ r YZ 1 rxz 2 1 ryz 2 The following is a correlation matrix:

23 Revisiting hypothesis testing and errors