Quasi-Experimental Designs Chapter 13
Quasi-Experimental Designs In a quasi-experimental design, the researcher lacks control over the assignment to conditions and/or does not manipulate the causal variable of interest. A quasi-independent variable is not manipulated by the researcher but rather is an event that occurred for other reasons.
Examples Does smoking cause cancer? Did 9/11 cause an increase in prejudice against people of middle-eastern decent? Do Republican vs. Democratic presidents affect the economy? Do extreme events (i.e., winning the lottery or being paralyzed) affect day-to-day happiness? Does campus crime affect applicants to a university?
Questionable Internal Validity No control over assignment of participants to the independent variable, so internal validity of quasiexperiments is always questionable. However, some quasi-experimental designs are more internally valid than others. The extent to which a quasi-experimental design can eliminate possible threats to internal validity determines its usefulness.
Quasi-Experiments and Internal Validity To infer that X causes Y 1. X must precede Y in time 2. X <-> Y must be related to each other 3. all other alternative explanations of the results are eliminated through random assignment or experimental control With quasi-experimental designs, you can t rule out ALL alternative explanations, but you can try to minimize them.
One-group pretest-posttest design Time 1 Measure Treatment Time 2 Measure O1 X O2 Example: Sample of violent adolescent women Treatment: anger management class Measures: aggressive behavior 1 year pre/post If aggressive behavior is lower at T2 than T1, can we conclude this decrease is caused by the treatment?
One-group pretest-posttest design Time 1 Measure Treatment Time 2 Measure O1 X O2 Internal validity problems? Maturation effect: between observations, participants could have grown out of aggressive behavior History effect: something else in the school environment caused a decrease less overcrowding Testing effect: the act of assessing aggression led to awareness of their own aggression
One-group pretest-posttest design Time 1 Measure Treatment Time 2 Measure O1 X O2 For these reasons, simple pretest-posttest design should never be used without a control condition and other efforts to control third variables.
Nonequivalent Control Group Designs Researcher obtains a groups of participants who are similar to the group that receives the quasiindependent variable. Example: Do extreme events (winning lottery, being paralyzed) have a long-term effect on day-to-day happiness?
Nonequivalent groups: posttest only design Measure both groups after one receives the quasiindependent variable. Time 1: Event X Event (X): winning lottery, paralyzing accident Advantage: unlike pre-post design, not subject to testing effect. Threat to Internal Validity: Selection bias, you can not be sure the groups were the same before the treatment. How could we try to address this? Time 2: Measure O -- O
Time Series Designs Measure the dependent variable on several occasions before and after the quasi-independent variable occurs.
Simple interrupted time series design T1 T2 T3 T4 T5 T6 T7 T8 T9 O1 O2 O3 O4 X O5 O6 O7 O8 8 Observations 4.0 4.3 3.9 4.2 2.9 3.1 3.0 3.3 2 Observations 4.2 2.9 Example: Does a well publicized crime spree (X) around Howard campus impact on number of observed applicants (O1-O8) to the university? What if we only had the two observations?
Simple interrupted time series design T1 T2 T3 T4 T5 T6 T7 T8 T9 O1 O2 O3 O4 X O5 O6 O7 O8 Advantage: Can rule out maturation. Possible Threat to Internal Validity: Contemporary history could be confounded with some other event that occurred at the same time.
Interrupted Time Series with a Reversal T1 2 3 4 5 6 7 8 9 10 11 12 13 O1 O2 O3 O4 X O6 O7 O8 -X O9 O10 O11 O12 Reputation 5 4 6 5 10 10 9 6 7 5 4 Example: Howard graduates earn 3 metals in one Olympics, and earn 0 metals in the next Olympics. What impact does that have on Howard s reputation for athletics? Shows the effect of adding AND removing quasiindependent variable. This pattern of results rules out maturation and history.
Time Series Designs Comparative Time Series Design Examines two or more variables over time in order to understand how changes in one variable are related to changes in another variable Provides indirect evidence that the change in one variable may be causing the change in the other variable
Example: Comparative Time Series Design (Friedberg, 1998)
Time Series Designs Control Group Interrupted Time Series Design nonequivalent control group included that does not receive the quasi-independent variable helps rule out certain history effects but could still get local history effects Experimental:O1 O2 O3 O4 X O5 O6 O7 O8 Control :O1 O2 O3 O4 -- O5 O6 O7 O8
Control Group Interrupted Time Series Example: Economic Output of Cities FEMA went into New Orleans 120 100 80 60 40 New Orleans Tampa FL 20 0 2001 2002 2003 2004 2005 2006 2007 2008 2009
Longitudinal vs. Cross-sectional Designs Design Participant # Longitudinal 1990 1995 2000 2005 P s 1-25 Age 5 Age 10 Age 15 Age 20 Cross-sectional P s 1-25 Age 5 P s 26-50 Age 10 P s 51-74 Age 15 P s 75-100 Age 20
Longitudinal Designs The quasi-independent variable is time; nothing has occurred from one observation to the next except for the passage of time O1 O2 O3 O4 O5 Mostly used by developmental psychologists to study age-related changes in how people think, feel, and behave. Developmental changes in memory, emotions, selfesteem, personality
Marital Satisfaction Longitudinal Design Example: Marital Satisfaction (Kurdek, 1999) Wife Husband 1 2 3 4 5 6 7 8 9 10 Years of Marriage Note: Years is within-participants. Would you offer to collect these data as a graduate student?
Longitudinal Designs Three drawbacks: Difficult to obtain participants who agree to be in a study over a long period of time Difficult to keep track of participants once they are in the study Repeatedly testing a sample requires time, effort, and money
Cross-Sectional Designs Drawback: Age is confounded with generation. Examples: 1. Texting ability by age (10, 20, 30, 40, 50, and 60 year old Ps) Younger people are faster texting Does texting speed decrease with age? Or are there generational effects with experience with texting? 2. Prejudice as a function of age. 3. Attitudes toward sex in the media as a function of age.
Evaluating Quasi-Experimental Designs Quasi-experimental designs can show that: the presumed causal variable preceded the effect in time cause and effect covary Quasi-experimental designs do not: eliminate all other alternative explanations for the results reason: no random assignment and no experimental control
Increasing Confidence in Quasi- Experimental Results 1. Measure both the effects of the quasi-iv and the processes assumed to explain the relationship. 2. Include a noncomparable control condition to rule out global history effects. 3 4
3. Find a complex pattern of results that cannot be explained by anything except for your quasi-experimental factor.
Complex patterns of data are not sufficient by themseves
Increasing Confidence in Quasi- Experimental Results 4. Critical multiplism employ multiple approaches that may converge to yield conclusions that are as concrete as those obtained in experimental research. Longitudinal and cross-sectional quasi and full experiments different quasi-experiments for the same factor replications in different populations...
Design your own Quasi-Experiment Points to consider -Pick an event or set of events & one DV variable -Is it longitudinal, cross-sectional, time-series, pre-post test, or post-test? -Is a non-equivalent control group or reversal included? -What is you hypothesis (prediction)? -What confounds threaten internal validity? -Can you improve the design to address the confounds?