Overview: Correlational & Ex Post Facto (aka causal-comparative ) Designs

Transcription

1 Overview: Correlational & Ex Post Facto (aka causal-comparative ) Designs Y520 Strategies for Educational Inquiry Correlation & Ex Post Facto designs-1 What is Correlational Research? Researchers want to know if there is a relationship between the number of science courses students take and score on the National Assessment of Educational Progress science test. What does relationship mean? What data should they gather to decide whether a relationship exists? Correlation & Ex Post Facto designs-2 1

2 Describe the Relationship (a) Correlation & Ex Post Facto designs-3 Describe the Relationship (b) Correlation & Ex Post Facto designs-4 2

3 What is a Relationship? We often have questions about relationships. Scores on the NAEP are related to: Geographic region Amount of time watching TV Type of computer use Teacher s area of undergraduate preparation. Type of science class taken. Correlation & Ex Post Facto designs-5 Scattergrams Reveal Relationships A Scatterplot is a graph that uses a coordinate plane to show the relationship between two variables. The convention is to place the exogenous (antecedent or predictor) variable on the X- axis (horizontal) and the endogenous (predicted or outcome) variable on the Y-axis. Each data point shows the xy pair of values for acase. Correlation & Ex Post Facto designs-6 3

4 Displaying Relationships Scattergram: X-Y pairs for each case Strong Positive Relationship: IQ and Spelling Test Scores Correlation & Ex Post Facto designs-7 Displaying Relationships Scattergram: X-Y pairs for each case Strong Positive Relationship with line of best fit. Correlation & Ex Post Facto designs-8 4

5 Displaying Relationships Scattergram: X-Y pairs for each case Strong Negative Relationship Correlation & Ex Post Facto designs-9 Another Example: Per Capita Income & Charitable Contributions Is there a relationship between the amount of money earned and the amount of charitable giving? We can list the per capita income for each state and the average itemized charitable contribution (1998). Each state has values for two variables. A good way to see whether a relationship exists is to use a scatter plot. Correlation & Ex Post Facto designs-10 5

6 Scattergram: X-Y pairs for each case $8,000 State Per Capita Income vs Per Capita Charitable Contriubtions, 1998 Avg Itemized Contributions by State $7,000 $6,000 $5,000 $4,000 $3,000 $2,000 Moderate Negative Relationship (r = ) $1,000 $19,000 $24,000 $29,000 $34,000 $39,000 Per Capita Income by State Correlation & Ex Post Facto designs-11 Scattergram: X-Y pairs for each case 160 Weight and IQ No Relationship 100 IQ Score Weignt (lbs) Correlation & Ex Post Facto designs-12 6

7 Summary: Relationships vary along two dimensions: Strength Strong Weak None Direction Positive Negative Correlation & Ex Post Facto designs-13 We can calculate the degree of relationship mathematically: One formula is the Pearson Product Moment Correlation Coefficient -- ( Pearson r ) Another is the Spearman Rho, used to calculate correlation for rank ordered data. Several other formulas exist for special situations. Correlation & Ex Post Facto designs-14 7

8 Example: Pearson r The table shows each student s test score and number of absences. First, look at the scatterplot and guess the direction and magnitude. Student Absences Score Alicia 1 92 Bobby 4 73 Carlos 5 86 Donna 8 58 Eddie 2 98 Frea 0 97 Gloria 4 70 Helena 7 65 Ingrid 0 88 John 2 82 mean std dev Correlation & Ex Post Facto designs-15 Example: Pearson r 110 Relationship between Absences and Test Score Test Score The relationship is negative and likely strong. The relationship appears to be linear Absences Correlation & Ex Post Facto designs-16 8

9 Example: Pearson r Second, calculate the mean and standard deviation for each column of values. The standard deviation is the square root of the average of the squared deviations of the raw scores from the mean. Student Absences Score Alicia 1 92 Bobby 4 73 Carlos 5 86 Donna 8 58 Eddie 2 98 Frea 0 97 Gloria 4 70 Helena 7 65 Ingrid 0 88 John 2 82 mean std dev Correlation & Ex Post Facto designs-17 Example: Pearson r Third, convert each raw score to its corresponding standard (z) score. (Eliminates the problem of different units for different variables). Standard scores have a mean of zero and standard deviation of 1. Each person s standard z- score shows their distance above or below the mean. Student zabsences zscore Alicia Bobby Carlos Donna Eddie Frea Gloria Helena Ingrid John Correlation & Ex Post Facto designs-18 9

10 Example: Pearson r Fourth, for each student, multiply the standardized absences times the standardized test score. The result shown in the right hand column is known as the crossproduct Sum the crossproducts and divide by the number of cases, minus one. Student zabsences zscore zabsences * zscore Alicia Bobby Carlos Donna Eddie Frea Gloria Helena Ingrid John < -- Sum <-- Pearson Correlation & Ex Post Facto designs-19 Example: Pearson r Formula for converting a raw score to a z-score: Conceptual formulas for Pearson r: Spearman s Rho : For large data sets, use built-in functions in Excel or statistical program. z r = = r spearman = 1 X raw M raw - SD raw z x z y - n 1 6 D 2 NN Correlation & Ex Post Facto designs-20 10

11 Correlational Study: Data Gathering Decide on the variables you suspect may be related. Decide how to measure those variables Select (preferably with a probability method) a sample Measure the variables You have ONE GROUP of subjects but TWO or more variables. Correlation & Ex Post Facto designs-21 Correlational Study: Data Analysis Use scatter plots Be sure the relationship is linear Look for trends and outliers Calculate the Pearson r or use other appropriate correlation formula Note the direction and strength of relationship Correlation & Ex Post Facto designs-22 11

12 Factors that distort our interpretation of Correlation Linearity The line of best fit for a linear relationship is a straight line. A curvilinear relationship yields an r value that underestimates the true strength of the relationship Restricted or truncate range of values The sample exhibits limited values for one or both variables. In either case, the value obtained for r is an underestimate. Extreme Groups r is overestimated. E.g., if the sample contains only poor and excellent readers, the correlation is overestimated. Extreme Scores Inflates r Correlation & Ex Post Facto designs-23 Factors that distort : Curvilinearity Linear and curvilinear (or nonmonotonic) relationships Correlation & Ex Post Facto designs-24 12

13 Factors that distort : Restricted Range Restricted Range: In this example, the correlation between foot size and age is much lower if we restrict age to one year rather than consider multiple years. Can range be artificially restricted without the investigator s awareness? Correlation & Ex Post Facto designs-25 Factors that distort : Truncated Range Truncated Range: The correlation between grades and ACT scores is likely underestimated because only students at the high end of the grading scale take the ACT. Correlation & Ex Post Facto designs-26 13

14 Factors that distort : Extreme Groups Extreme Groups: In this example only poor and excellent readers were selected. The correlation with IQ is optimistic. Inclusion of middle range readers would provide a more representative sample and lower the correlation. Correlation & Ex Post Facto designs-27 Factors that distort : Extreme Score Extreme Scores have a greater effect when thesamplesizeis small. In this example, asamplewithno obvious relationship appears to exhibit a linear relationship, due to a single extreme score. Correlation & Ex Post Facto designs-28 14

15 Summary: Factors that Distort Note that with the exception of linearity, all of the conditions that can distort the correlation are the result of a sample that is not representative of the population: Restricted range Truncated range Extreme groups Extreme scores Correlation & Ex Post Facto designs-29 Correlational Study: Results Report obtained correlations Discuss strength and direction Determine statistical significance (Fisher s r to z transformation) Evaluate what it means in the context of your study. Causality Correlation & Ex Post Facto designs-30 15

16 Correlational Study: Causality? Correlation indicates little or nothing about causality. Inferring causality from correlation is a common error. Possibility a. b. c. d. Symbols X Y X Y X A Y B C X B Y Explanation X causes Y Y causes X A causes both X and Y Correlation & Ex Post Facto designs-31 Interpreting Correlational Correlation indicates little or nothing about causality. Inferring causality from correlation is a common error. How much of the change in one variable is related to the change in the other variable? This is known as common variance or the coefficient of determination (r 2 ). Unexplained variance = 1 - r 2 Correlation & Ex Post Facto designs-32 16

17 Causal Comparative Research We now shift from describing two or more variables in one group, to Comparison (two or more groups). Correlation & Ex Post Facto designs-33 Causal Comparative Research Involves comparison of two or more groups on a single endogenous variables. The characteristic that differentiates these groups is the exogenous variable. Causal comparative studies are also called ex post facto because the investigator has no control over the exogenous variable. Whatever happened occurred before the researcher arrived. We can never know with certainty that the two groups were exactly equal before the difference occurred. Correlation & Ex Post Facto designs-34 17

18 Causal Comparative: Data Collection You select two groups that differ on the (exogenous) variable of interest. Next, compare the two groups by looking at an endogenous variable that you think might be influenced by the exogenous variable. Define clearly and operationally the exogenous variable. Be sure the groups are similar on all other important variables. Correlation & Ex Post Facto designs-35 Causal Comparative: Equating groups Use subject matching Use change scores; i.e., each subject as own control Compare homogeneous groups Use analysis of covariance Correlation & Ex Post Facto designs-36 18

19 Causal Comparative: Data Analysis Because we usually are dealing with samples, we use inferential statistical testing techniques: T-test (two groups) Analysis of variance Chi-square for frequency data Correlation & Ex Post Facto designs-37 Causal Comparative: Conclusions Researchers often infer cause and effect relationships based on such studies. Conditions necessary, but not necessarily sufficient, to infer a causal relationship: A statistical relationship exists that is unlikely attributable to chance variation You have reason to believe the supposed exogenous variable preceded the endogenous. You can, with some degree of certainty, rule out other possible explanations. Correlation & Ex Post Facto designs-38 19

20 Discuss the relationship (correlation) Correlation & Ex Post Facto designs-39 Types of Quasi - Experimental Designs (a) Time-series designs. Equivalent time-series samples Equivalent samples, materials design Non-equivalent control group Counterbalanced designs Correlation & Ex Post Facto designs-40 20

21 Begin here Correlation & Ex Post Facto designs-41 21