# Lee A. Becker. Effect Size (ES) 2000 Lee A. Becker

Save this PDF as:

Size: px
Start display at page:

Download "Lee A. Becker. Effect Size (ES) 2000 Lee A. Becker"

## Transcription

1 Lee A. Becker <http://web.uccs.edu/lbecker/psy590/es.htm> Effect Size (ES) 2000 Lee A. Becker I. Overview II. Effect Size Measures for Two Independent Groups 1. Standardized difference between two groups. 2. Correlation measures of effect size. 3. Computational examples III. Effect Size Measures for Two Dependent Groups. IV. Meta Analysis V. Effect Size Measures in Analysis of Variance VI. References Effect Size Calculators Answers to the Effect Size Computation Questions I. Overview Effect size (ES) is a name given to a family of indices that measure the magnitude of a treatment effect. Unlike significance tests, these indices are independent of sample size. ES measures are the common currency of meta-analysis studies that summarize the findings from a specific area of research. See, for example, the influential metaanalysis of psychological, educational, and behavioral treatments by Lipsey and Wilson (1993). There is a wide array of formulas used to measure ES. For the occasional reader of meta-analysis studies, like myself, this diversity can be confusing. One of my objectives in putting together this set of lecture notes was to organize and summarize the various measures of ES. In general, ES can be measured in two ways:

2 a) as the standardized difference between two means, or b) as the correlation between the independent variable classification and the individual scores on the dependent variable. This correlation is called the "effect size correlation" (Rosnow & Rosenthal, 1996). These notes begin with the presentation of the basic ES measures for studies with two independent groups. The issues involved when assessing ES for two dependent groups are then described. II. Effect Size Measures for Two Independent Groups 1. Standardized difference between two groups. d = M 1 - M 2 / σ where σ = [ (X - M)² / N] where X is the raw score, M is the mean, and N is the number of cases. d = M 1 - M 2 / σ pooled σ pooled = [(σ 1 ²+ σ 2 ²) / 2] d = 2t / (df) or d = t(n 1 + n 2 ) / [ (df) (n 1 n 2 )] Cohen's d Cohen (1988) defined d as the difference between the means, M 1 - M 2, divided by standard deviation, σ, of either group. Cohen argued that the standard deviation of either group could be used when the variances of the two groups are homogeneous. In meta-analysis the two groups are considered to be the experimental and control groups. By convention the subtraction, M 1 - M 2, is done so that the difference is positive if it is in the direction of improvement or in the predicted direction and negative if in the direction of deterioration or opposite to the predicted direction. d is a descriptive measure. In practice, the pooled standard deviation, σ pooled, is commonly used (Rosnow and Rosenthal, 1996). The pooled standard deviation is found as the root mean square of the two standard deviations (Cohen, 1988, p. 44). That is, the pooled standard deviation is the square root of the average of the squared standard deviations. When the two standard deviations are similar the root mean square will be not differ much from the simple average of the two variances. d can also be computed from the value of the t test of the differences between the two groups (Rosenthal and Rosnow, 1991).. In the equation to the left "df" is the degrees of freedom for the t test. The "n's" are the number of cases for each group. The formula without the n's should be used when the n's are equal. The formula with

3 separate n's should be used when the n's are not equal. d = 2r / (1 - r²) d can be computed from r, the ES correlation. d = g (N/df) d can be computed from Hedges's g. The interpretation of Cohen's d Cohen's Standard Effect Size Percentile Standing Percent of Nonoverlap % % % % % % % % % % % % LARGE % % % MEDIUM % % % SMALL % % % Cohen (1988) hesitantly defined effect sizes as "small, d =.2," "medium, d =.5," and "large, d =.8", stating that "there is a certain risk in inherent in offering conventional operational definitions for those terms for use in power analysis in as diverse a field of inquiry as behavioral science" (p. 25). Effect sizes can also be thought of as the average percentile standing of the average treated (or experimental) participant relative to the average untreated (or control) participant. An ES of 0.0 indicates that the mean of the treated group is at the 50th percentile of the untreated group. An ES of 0.8 indicates that the mean of the treated group is at the 79th percentile of the untreated group. An effect size of 1.7 indicates that the mean of the treated group is at the 95.5 percentile of the untreated group. Effect sizes can also be interpreted in terms of the percent of nonoverlap of the treated group's scores with those of the untreated group, see Cohen (1988, pp ) for descriptions of additional measures of nonoverlap.. An ES of 0.0 indicates that the distribution of scores for the treated group overlaps completely with the distribution of scores for the untreated group, there is 0% of nonoverlap. An ES of 0.8 indicates a nonoverlap of 47.4% in the two distributions. An ES of 1.7

4 indicates a nonoverlap of 75.4% in the two distributions. Hedges's g g = M 1 - M 2 / S pooled where S = [ (X - M)² / N-1] and S pooled = MSwithin g = t (n 1 + n 2 ) / (n 1 n 2 ) or g = 2t / N σ pooled = S pooled (df / N) were df = the degrees of freedom for the MSerror, and N = the total number of cases. g = d / (N / df) g = [r / (1 - r²)] / [df(n 1 + n 2 ) / (n 1 n 2 )] Hedges's g is an inferential measure. It is normally computed by using the square root of the Mean Square Error from the analysis of variance testing for differences between the two groups. Hedges's g is named for Gene V. Glass, one of the pioneers of metaanalysis. Hedges's g can be computed from the value of the t test of the differences between the two groups (Rosenthal and Rosnow, 1991). The formula with separate n's should be used when the n's are not equal. The formula with the overall number of cases, N, should be used when the n's are equal. The pooled standard deviation, σ pooled, can be computed from the unbiased estimator of the pooled population value of the standard deviation, S pooled, and vice versa, using the formula on the left (Rosnow and Rosenthal, 1996, p. 334). Hedges's g can be computed from Cohen's d. Hedges's g can be computed from r, the ES correlation. Glass's delta = M 1 - M 2 / σ control Glass's delta is defined as the mean difference between the experimental and control group divided by the standard deviation of the control group.

5 2. Correlation measures of effect size The ES correlation, r Yλ r Yλ = r dv,iv CORR = dv with iv r Yλ = Φ = (Χ²(1) / N) r Yλ = [t² / (t² + df)] r Yλ = [F(1,_) / (F(1,_) + df error)] r Yλ = d / (d² + 4) r Yλ = {(g²n 1 n 2 ) / [g²n 1 n 2 +( n 1 + n 2 )df]} The effect size correlation can be computed directly as the pointbiserial correlation between the dichotomous independent variable and the continuous dependent variable. The point-biserial is a special case of the Pearson product-moment correlation that is used when one of the variables is dichotomous. As Nunnally (1978) points out, the pointbiserial is a shorthand method for computing a Pearson product-moment correlation. The value of the pointbiserial is the same as that obtained from the product-moment correlation. You can use the CORR procedure in SPSS to compute the ES correlation. The ES correlation can be computed from a single degree of freedom Chi Square value by taking the square root of the Chi Square value divided by the number of cases, N. This value is also known as Phi. The ES correlation can be computed from the t-test value. The ES correlation can be computed from a single degree of freedom F test value (e.g., a oneway analysis of variance with two groups). The ES correlation can be computed from Cohen's d. The ES correlation can be computed from Hedges's g. The relationship between d, r, and r² Cohen's Standard d r r² As noted in the definition sections above, d and be converted to r and vice versa. For example, the d value of.8 corresponds to an r value of.371.

6 LARGE MEDIUM SMALL The square of the r-value is the percentage of variance in the dependent variable that is accounted for by membership in the independent variable groups. For a d value of.8, the amount of variance in the dependent variable by membership in the treatment and control groups is 13.8%. In meta-analysis studies rs are typically presented rather than r². 3. Computational Examples The following data come from Wilson, Becker, and Tinker (1995). In that study participants were randomly assigned to either EMDR treatment or delayed EMDR treatment. Treatment group assignment is called TREATGRP in the analysis below. The dependent measure is the Global Severity Index (GSI) of the Symptom Check List-90R. This index is called GLOBAL4 in the analysis below. The analysis looks at the the GSI scores immediately post treatment for those assigned to the EMDR treatment group and at the second pretreatment testing for those assigned to the delayed treatment condition. The output from the SPSS MANOVA and CORR(elation) procedures are shown below. Cell Means and Standard Deviations Variable.. GLOBAL4 GLOBAL INDEX:SLC-90R POST-TEST FACTOR CODE Mean Std. Dev. N 95 percent Conf. Interval TREATGRP TREATMEN TREATGRP DELAYED For entire sample * * * * * * * * * * * * * A n a l y s i s o f V a r i a n c e -- Design 1 * * * * * * * * * * * * Tests of Significance for GLOBAL4 using UNIQUE sums of squares

7 Source of Variation SS DF MS F Sig of F WITHIN CELLS TREATGRP (Model) (Total) GLOBAL4 TREATGRP.3134 ( 80) P= Correlation Coefficients - - Look back over the formulas for computing the various ES estimates. This SPSS output has the following relevant information: cell means, standard deviations, and ns, the overall N, and MSwithin. Let's use that information to compute ES estimates. d = M 1 - M 2 / [( σ 1 ² + σ 2 ²)/ 2] = / [(0.628² ²) / 2] = / [( ) / 2] = / ( / 2) = / = / =.65 g = M 1 - M 2 / MSwithin = / 0.41 = / =.65 = M 1 - M 2 / σ control = / = / =.66 r Yλ = [F(1,_) / (F(1,_) + df error)] = [8.49 / ( )] = [8.49 / ] = =.31 Cohen's d Cohen's d can be computed using the two standard deviations. What is the magnitude of d, according to Cohen's standards? The mean of the treatment group is at the percentile of the control group. Hedges's g Hedges's g can be computed using the MSwithin. Hedges's g and Cohen's d are similar because the sample size is so large in this study. Glass's delta Glass's delta can be computed using the standard deviation of the control group. Effect size correlation The effect size correlation was computed by SPSS as the correlation between the iv (TREATGRP) and the dv (GLOBAL4), r Yλ =.31 The effect size correlation can also be

8 computed from the F value. The next computational is from the same study. This example uses Wolpe's Subjective Units of Disturbance Scale (SUDS) as the dependent measure. It is a single item, 11-point scale ( 0 = neutral; 10 = the highest level of disturbance imaginable) that measures the level of distress produced by thinking about a trauma. SUDS scores are measured immediately post treatment for those assigned to the EMDR treatment group and at the second pretreatment testing for those assigned to the delayed treatment condition. The SPSS output from the T-TEST and CORR(elation) procedures is shown below. t-tests for Independent Samples of TREATGRP TREATMENT GROUP Number Variable of Cases Mean SD SE of Mean SUDS4 POST-TEST SUDS TREATMENT GROUP DELAYED TRMT GROUP Mean Difference = Levene's Test for Equality of Variances: F= P=.274 t-test for Equality of Means 95% Variances t-value df 2-Tail Sig SE of Diff CI for Diff Unequal (-5.814, ) Correlation Coefficients - - SUDS4 TREATGRP.7199 ( 80) P=.000 (Coefficient / (Cases) / 2-tailed Significance) Use the data in the above table to compute each of the following ES statistics: Cohen's d Compute Cohen's d using the two standard deviations. How large is the d using Cohen's interpretation of effect sizes? Cohen's d Compute Cohen's d using the value of the t-test statistic.

9 Are the two values of d similar? Hedges's g Compute Hedges's g using the t-test statistic. Glass's delta Calculate Glass's delta using the standard deviation of the control group. Effect size correlation The effect size correlation was computed by SPSS as the correlation between the iv (TREATGRP) and the dv (SUDS4), r Yλ =. Calculate the effect size correlation using the t value. Effect size correlation Use Cohen's d to calculate the effect size correlation. III. Effect Size Measures for Two Dependent Groups. There is some controversy about how to compute effect sizes when the two groups are dependent, e.g., when you have matched groups or repeated measures. These designs are also called correlated designs. Let's look at a typical repeated measures design. A Correlated (or Repeated Measures) Design O C1 O C2 O E1 X O E2 Participants are randomly assigned to one of two conditions, experimental (E.) or control (C.). A pretest is given to all

10 participants at time 1 (O.1. ). The treatment is administered at "X". Measurement at time 2 (O E2 ) is posttreatment for the experimental group. The control group is measured a second time at (O C2 ) without an intervening treatment.. The time period between O.1 and O. 2 is the same for both groups. This research design can be analyzed in a number of ways including by gain scores, a 2 x 2 ANOVA with measurement time as a repeated measure, or by an ANCOVA using the pretest scores as the covariate. All three of these analyses make use of the fact that the pretest scores are correlated with the posttest scores, thus making the significance tests more sensitive to any differences that might occur (relative to an analysis that did not make use of the correlation between the pretest and posttest scores). An effect size analysis compares the mean of the experimental group with the mean of the control group. The experimental group mean will be the posttreatment scores, O E2. But any of the other three means might be used as the control group mean. You could look at the ES by comparing O E2 with its own pretreatment score, O E1, with the pretreatment score of the control group, O C1, or with the second testing of the untreated control group, O C2. Wilson, Becker, and Tinker (1995) computed effect size estimates, Cohen's d, by comparing the experimental group's posttest scores (O E2 ) with the second testing of the untreated control group (O C2 ). We choose O C2 because measures taken at the same time would be less likely to be subject to history artifacts, and because any regression to the mean from time 1 to time 2 would tend to make that test more conservative. Suppose that you decide to compute Cohen's d by comparing the experimental group's pretest scores (O E2 ) with their own pretest scores (O E1 ), how should the pooled standard deviation be computed? There are two possibilities, you might use the original standard deviations for the two means, or you might use the paired t-test value to compute Cohen's d. Because the paired t-test value takes into account the correlation between the two scores the paired t-test will be larger than a between groups t-test. Thus, the ES computed using the paired t-test value will always be larger than the ES computed using a between groups t-test value, or the original standard deviations of the scores. Rosenthal (1991) recommended using the paired t- test value in computing the ES. A set of meta-analysis computer programs by Mullen and Rosenthal (1985) use the paired t-test value in its computations. However, Dunlop, Cortina, Vaslow, & Burke (1996) convincingly argue that the original standard deviations (or the between group t-test value) should be used to compute ES for correlated designs. They argue that if the pooled standard deviation is corrected for the amount of correlation between the measures, then the ES estimate will be an overestimate of the actual ES. As shown in Table 2 of Dunlop et al., the overestimate is dependent upon the magnitude of the correlation between between the two scores. For example, when the correlation between the scores is at least.8, then the ES estimate is more than twice the magnitude of the ES computed using the original standard deviations of the measures.

11 The same problem occurs if you use a one-degree of freedom F value that is based on a repeated measures to compute an ES value. In summary, when you have correlated designs you should use the original standard deviations to compute the ES rather than the paired t-test value or the within subject's F value. IV. Meta Analysis Overview A meta-analysis is a summary of previous research that uses quantitative methods to compare outcomes across a wide range of studies. Traditional statistics such as t tests or F tests are inappropriate for such comparisons because the values of those statistics are partially a function of the sample size. Studies with equivalent differences between treatment and control conditions can have widely varying t and F statistics if the studies have different sample sizes. Meta analyses use some estimate of effect size because effect size estimates are not influenced by sample sizes. Of the effect size estimates that were discussed earlier in this page, the most common estimate found in current meta analyses is Cohen's d. In this section we look at a meta analysis of treatment efficacy for posttraumatic stress disorder (Van Etten & Taylor, 1998). For those of you interested in the efficacy of other psychological and behavioral treatments I recommend the influential paper by Lipsey and Wilson (1993). The Research Base The meta analysis is based on 61 trials from 39 studies of chronic PTSD. Comparisons are made for Drug treatments, Psychological Treatments and Controls. Drug treatments include: selective serotonin reuptake inhibitors (SSRI; new antidepressants such as Prozac and Paxil), monoamine oxidase inhibitors (MAOI, antidepressants such as Parnate and Marplan), tricyclic antidepressants (TCA; antidepressants such as Toffranil), benzodiazepines (BDZ; minor tranquilizers such as Valium), and carbamazepine (Carbmz; anticonvulsants such as Tegretol). The psychotherapies include: behavioral treatments (primarily different forms of exposure therapies), eye movement desensitization and reprocessing (EMDR), relaxation therapy, hypnosis, and psychodynamic therapy. The control conditions include: pill placebo (used in the drug treatment studies), wait list controls, supportive psychotherapy, and no saccades (a control for eye movements in EMDR studies). Data Reduction Procedures

12 Effect sizes were computed as Cohen's d where a positive effect size represents improvement and a negative effect size represents a "worsening of symptoms." Ninety-percent confidence intervals were computed. Comparisons were made base on those confidence intervals rather than on statistical tests (e.g., t test) of the mean effect size. If the mean of one group was not included within the 90% confidence interval of the other group then the two groups differed significantly at p <.10. Comparisons across conditions (e.g., drug treatments vs. psychotherapies) were made by computing a weighted mean for each group were the individual trial means were weighted by the number of cases for the trial. This procedure gives more weight to trials with larger ns, presumably the means for those studies are more robust. Treatment Comparisons For illustrative purposes lets look at the Self Report measures of the Total Severity of PTSD Symptoms from Table 2 (Van Etten & Taylor, 1998). Overall treatments. The overall effect size for psychotherapy treatments (M = 1.17; 90% CI = ) is significantly greater than both the overall drug effect size (M = 0.69; 90% CI = ) and the overall control effect size (M = 0.43; 90% CI = ). The drug treatments are more effective than the controls conditions. Within drug treatments. Within the drug treatments SSRI is more effective than any of the other drug treatments. Within psychotherapies. Within the psychotherapies behavior modification and EMDR are equally effective. EMDR is more effective than any of the other psychotherapies. Behavior modification is more effective than relaxation therapy. Within Controls. Within the control conditions the alternatives the pill placebo and wait list controls produce larger effects than the no saccade condition. Across treatment modalities. EMDR is more effective than each of the drug conditions except the SSRI drugs. SSRI and EMDR are equally effective. Behavior modification is more effective than TCAs, MAOIs and BDZs, it is equally effective as the SSRIs and Carbmxs. Behavior Modification and EMDR are more effective than any of the control conditions. Self Report for Total Severity of PTSD Symptoms Condition M 90% CI TCA Carbmz 0.93 MAOI SSRI BDZ 0.49 Drug Tx Behav Tx EMDR Relaxat'n 0.45 Hypnosis 0.94 Dynamic 0.90 Psych Tx Pill Placebo WLC Supp Psyc

13 It is also interesting to note that the drop out rates for drug therapies (M = 31.9; 90% CI = ) are more than twice the rate for psychotherapies (M = 14.0; 90% CI = ). No sacc 0.22 Controls Fail Safe N One of the problems with meta analysis is that you can only analyze the studies that have been published. There is the file drawer problem, that is, how many studies that did not find significant effects have not been published? If those studies in the file drawer had been published then the effect sizes for those treatments would be smaller. The fail safe N is the number of nonsignificant studies that would be necessary to reduce the effect size to an nonsignificant value, defined in this study as an effect size of The fail safe Ns are shown in the table at the right. The fail safe Ns for Behavior therapies, EMDR, and SSRI are very large. It is unlikely that there are that many well constructed studies sitting in file drawers. On the other hand, the fail safe N's for BDZ, Carbmz, relaxation therapy, hypnosis, and psychodynamic therapies are so small that one should be cautious about accepting the validity of the effect sizes for those treatments. Condition Fail Safe N TCA 41 Carbmz 23 MAOI 66 SSRI 95 BDZ 10 Behav Tx 220 EMDR 139 Relaxat'n 8 Hypnosis 18 Dynamic 17 V. Effect Size Measures in Analysis of Variance Measures of effect size in ANOVA are measures of the degree of association between and effect (e.g., a main effect, an interaction, a linear contrast) and the dependent variable. They can be thought of as the correlation between an effect and the dependent variable. If the value of the measure of association is squared it can be interpreted as the proportion of variance in the dependent variable that is attributable to each effect. Four of the commonly used measures of effect size in AVOVA are: Eta squared, η 2 partial Eta squared, η p 2 omega squared, ω 2 the Intraclass correlation, ρ I Eta squared and partial Eta squared are estimates of the degree of association for the sample. Omega squared and the intraclass correlation are estimates of the degree of

14 association in the population. SPSS for Windows displays the partial Eta squared when you check the "display effect size" option in GLM. See the following SPSS lecture notes for additional information on these ANOVAbased measures of effect size: Effect size measures in Analysis of Variance VI. References Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. Dunlop, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1, Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, Mullen, H., & Rosenthal, R. (1985). BASIC meta-analysis: Procedures and programs. Hillsdale, NJ: Earlbaum. Nunnally, J. C. (1978). Psychometric Theory. New York: McGraw-Hill. Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage. Rosenthal, R. & Rosnow, R. L. (1991). Essentials of behavioral research: Methods and data analysis (2nd ed.). New York: McGraw Hill. Rosenthal, R. & Rubin, D. B. (1986). Meta-analytic procedures for combining studies with multiple effect sizes. Psychological Bulletin, 99, Rosnow, R. L., & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternulls on other people's published data: General procedures for research consumers. Pyschological Methods, 1, Van Etten, Michelle E., & Taylor, S. (1998). Comparative efficacy of treatments for posttraumatic stress disorder: A meta-analysis. Clinical Psychology & Psychotherapy, 5, Wilson, S. A., Becker, L. A., & Tinker, R. H. (1995). Eye movement desensitization and reprocessing (EMDR) treatment for psychologically traumatized individuals. Journal of Consulting and Clinical Psychology, 63, Lee A. Becker

### T-tests. Daniel Boduszek

T-tests Daniel Boduszek d.boduszek@interia.eu danielboduszek.com Presentation Outline Introduction to T-tests Types of t-tests Assumptions Independent samples t-test SPSS procedure Interpretation of SPSS

### Understanding and Quantifying EFFECT SIZES

Understanding and Quantifying EFFECT SIZES Karabi Nandy, Ph.d. Assistant Adjunct Professor Translational Sciences Section, School of Nursing Department of Biostatistics, School of Public Health, University

### Calculating, Interpreting, and Reporting Estimates of Effect Size (Magnitude of an Effect or the Strength of a Relationship)

1 Calculating, Interpreting, and Reporting Estimates of Effect Size (Magnitude of an Effect or the Strength of a Relationship) I. Authors should report effect sizes in the manuscript and tables when reporting

### 10. Analysis of Longitudinal Studies Repeat-measures analysis

Research Methods II 99 10. Analysis of Longitudinal Studies Repeat-measures analysis This chapter builds on the concepts and methods described in Chapters 7 and 8 of Mother and Child Health: Research methods.

### UNDERSTANDING ANALYSIS OF COVARIANCE (ANCOVA)

UNDERSTANDING ANALYSIS OF COVARIANCE () In general, research is conducted for the purpose of explaining the effects of the independent variable on the dependent variable, and the purpose of research design

### UNDERSTANDING THE DEPENDENT-SAMPLES t TEST

UNDERSTANDING THE DEPENDENT-SAMPLES t TEST A dependent-samples t test (a.k.a. matched or paired-samples, matched-pairs, samples, or subjects, simple repeated-measures or within-groups, or correlated groups)

### Analysis of Data. Organizing Data Files in SPSS. Descriptive Statistics

Analysis of Data Claudia J. Stanny PSY 67 Research Design Organizing Data Files in SPSS All data for one subject entered on the same line Identification data Between-subjects manipulations: variable to

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### January 26, 2009 The Faculty Center for Teaching and Learning

THE BASICS OF DATA MANAGEMENT AND ANALYSIS A USER GUIDE January 26, 2009 The Faculty Center for Teaching and Learning THE BASICS OF DATA MANAGEMENT AND ANALYSIS Table of Contents Table of Contents... i

### Two-sample hypothesis testing, II 9.07 3/16/2004

Two-sample hypothesis testing, II 9.07 3/16/004 Small sample tests for the difference between two independent means For two-sample tests of the difference in mean, things get a little confusing, here,

### Analysing Questionnaires using Minitab (for SPSS queries contact -) Graham.Currell@uwe.ac.uk

Analysing Questionnaires using Minitab (for SPSS queries contact -) Graham.Currell@uwe.ac.uk Structure As a starting point it is useful to consider a basic questionnaire as containing three main sections:

### Correlation Coefficients: Mean Bias and Confidence Interval Distortions

Journal of Methods and Measurement in the Social Sciences Vol. 1, No. 2,52-65, 2010 Correlation Coefficients: Mean Bias and Confidence Interval Distortions Richard L. Gorsuch Fuller Theological Seminary

### STATISTICS FOR PSYCHOLOGISTS

STATISTICS FOR PSYCHOLOGISTS SECTION: STATISTICAL METHODS CHAPTER: REPORTING STATISTICS Abstract: This chapter describes basic rules for presenting statistical results in APA style. All rules come from

### Standardised Effect Size in a Mixed/Multilevel Model

Standardised Effect Size in a Mixed/Multilevel Model This note uses simple examples based on two or more groups (group), and measurements at two time points (time), to consider how standardised effect

### Regression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology

Regression in SPSS Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology John P. Bentley Department of Pharmacy Administration University of

### Meta-Analytic Synthesis of Studies Conducted at Marzano Research Laboratory on Instructional Strategies

Meta-Analytic Synthesis of Studies Conducted at Marzano Research Laboratory on Instructional Strategies By Mark W. Haystead & Dr. Robert J. Marzano Marzano Research Laboratory Englewood, CO August, 2009

### INTERPRETING THE REPEATED-MEASURES ANOVA

INTERPRETING THE REPEATED-MEASURES ANOVA USING THE SPSS GENERAL LINEAR MODEL PROGRAM RM ANOVA In this scenario (based on a RM ANOVA example from Leech, Barrett, and Morgan, 2005) each of 12 participants

### ANSWERS TO EXERCISES AND REVIEW QUESTIONS

ANSWERS TO EXERCISES AND REVIEW QUESTIONS PART FIVE: STATISTICAL TECHNIQUES TO COMPARE GROUPS Before attempting these questions read through the introduction to Part Five and Chapters 16-21 of the SPSS

### About Single Factor ANOVAs

About Single Factor ANOVAs TABLE OF CONTENTS About Single Factor ANOVAs... 1 What is a SINGLE FACTOR ANOVA... 1 Single Factor ANOVA... 1 Calculating Single Factor ANOVAs... 2 STEP 1: State the hypotheses...

### Outline. Definitions Descriptive vs. Inferential Statistics The t-test - One-sample t-test

The t-test Outline Definitions Descriptive vs. Inferential Statistics The t-test - One-sample t-test - Dependent (related) groups t-test - Independent (unrelated) groups t-test Comparing means Correlation

### 7. Comparing Means Using t-tests.

7. Comparing Means Using t-tests. Objectives Calculate one sample t-tests Calculate paired samples t-tests Calculate independent samples t-tests Graphically represent mean differences In this chapter,

### Chapter 2 Probability Topics SPSS T tests

Chapter 2 Probability Topics SPSS T tests Data file used: gss.sav In the lecture about chapter 2, only the One-Sample T test has been explained. In this handout, we also give the SPSS methods to perform

### Inferences About Differences Between Means Edpsy 580

Inferences About Differences Between Means Edpsy 580 Carolyn J. Anderson Department of Educational Psychology University of Illinois at Urbana-Champaign Inferences About Differences Between Means Slide

### 1. What is the critical value for this 95% confidence interval? CV = z.025 = invnorm(0.025) = 1.96

1 Final Review 2 Review 2.1 CI 1-propZint Scenario 1 A TV manufacturer claims in its warranty brochure that in the past not more than 10 percent of its TV sets needed any repair during the first two years

Mgt 540 Research Methods Data Analysis 1 Additional sources Compilation of sources: http://lrs.ed.uiuc.edu/tseportal/datacollectionmethodologies/jin-tselink/tselink.htm http://web.utk.edu/~dap/random/order/start.htm

### One-Way Analysis of Variance

One-Way Analysis of Variance Note: Much of the math here is tedious but straightforward. We ll skim over it in class but you should be sure to ask questions if you don t understand it. I. Overview A. We

### 12/31/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Understand linear regression with a single predictor Understand how we assess the fit of a regression model Total Sum of Squares

### ANOVA MULTIPLE CHOICE QUESTIONS. In the following multiple-choice questions, select the best answer.

ANOVA MULTIPLE CHOICE QUESTIONS In the following multiple-choice questions, select the best answer. 1. Analysis of variance is a statistical method of comparing the of several populations. a. standard

### DEPARTMENT OF PSYCHOLOGY UNIVERSITY OF LANCASTER MSC IN PSYCHOLOGICAL RESEARCH METHODS ANALYSING AND INTERPRETING DATA 2 PART 1 WEEK 9

DEPARTMENT OF PSYCHOLOGY UNIVERSITY OF LANCASTER MSC IN PSYCHOLOGICAL RESEARCH METHODS ANALYSING AND INTERPRETING DATA 2 PART 1 WEEK 9 Analysis of covariance and multiple regression So far in this course,

### Chapter 5 Analysis of variance SPSS Analysis of variance

Chapter 5 Analysis of variance SPSS Analysis of variance Data file used: gss.sav How to get there: Analyze Compare Means One-way ANOVA To test the null hypothesis that several population means are equal,

### Study Guide for the Final Exam

Study Guide for the Final Exam When studying, remember that the computational portion of the exam will only involve new material (covered after the second midterm), that material from Exam 1 will make

### Using Excel for inferential statistics

FACT SHEET Using Excel for inferential statistics Introduction When you collect data, you expect a certain amount of variation, just caused by chance. A wide variety of statistical tests can be applied

### Chapter Eight: Quantitative Methods

Chapter Eight: Quantitative Methods RESEARCH DESIGN Qualitative, Quantitative, and Mixed Methods Approaches Third Edition John W. Creswell Chapter Outline Defining Surveys and Experiments Components of

### Introduction to Analysis of Variance (ANOVA) Limitations of the t-test

Introduction to Analysis of Variance (ANOVA) The Structural Model, The Summary Table, and the One- Way ANOVA Limitations of the t-test Although the t-test is commonly used, it has limitations Can only

### Database of randomized trials of psychotherapy for adult depression

Database of randomized trials of psychotherapy for adult depression In this document you find information about the database of 352 randomized controlled trials on psychotherapy for adult depression. This

### psyc3010 lecture 8 standard and hierarchical multiple regression last week: correlation and regression Next week: moderated regression

psyc3010 lecture 8 standard and hierarchical multiple regression last week: correlation and regression Next week: moderated regression 1 last week this week last week we revised correlation & regression

### Paired-samples t test Also known as the Correlated-samples and Dependent-samples t-test

Paired-samples t test Also known as the Correlated-samples and Dependent-samples t-test 1. Purpose The correlated t-test allows researchers to consider differences between two groups or sets of scores

### Chapter 13 Introduction to Linear Regression and Correlation Analysis

Chapter 3 Student Lecture Notes 3- Chapter 3 Introduction to Linear Regression and Correlation Analsis Fall 2006 Fundamentals of Business Statistics Chapter Goals To understand the methods for displaing

### Package compute.es. February 19, 2015

Type Package Title Compute Effect Sizes Version 0.2-4 Date 2014-09-16 Author AC Del Re Maintainer AC Del Re Package compute.es February 19, 2015 Description This package contains several

### PsychTests.com advancing psychology and technology

PsychTests.com advancing psychology and technology tel 514.745.8272 fax 514.745.6242 CP Normandie PO Box 26067 l Montreal, Quebec l H3M 3E8 contact@psychtests.com Psychometric Report Resilience Test Description:

### Chapter 9. Two-Sample Tests. Effect Sizes and Power Paired t Test Calculation

Chapter 9 Two-Sample Tests Paired t Test (Correlated Groups t Test) Effect Sizes and Power Paired t Test Calculation Summary Independent t Test Chapter 9 Homework Power and Two-Sample Tests: Paired Versus

### UNDERSTANDING THE INDEPENDENT-SAMPLES t TEST

UNDERSTANDING The independent-samples t test evaluates the difference between the means of two independent or unrelated groups. That is, we evaluate whether the means for two independent groups are significantly

### INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA)

INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA) As with other parametric statistics, we begin the one-way ANOVA with a test of the underlying assumptions. Our first assumption is the assumption of

### Treatment: Healing Actions, Healing Words

Treatment: Healing Actions, Healing Words 15 : Insight-Oriented Therapy Psychoanalysis (p.687) First use of a talking cure Developed by Sigmund Freud Identify unconscious motivations Free association Dream

### Elementary Statistics Sample Exam #3

Elementary Statistics Sample Exam #3 Instructions. No books or telephones. Only the supplied calculators are allowed. The exam is worth 100 points. 1. A chi square goodness of fit test is considered to

### Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Readings: Ha and Ha Textbook - Chapters 1 8 Appendix D & E (online) Plous - Chapters 10, 11, 12 and 14 Chapter 10: The Representativeness Heuristic Chapter 11: The Availability Heuristic Chapter 12: Probability

### Reporting Statistics in Psychology

This document contains general guidelines for the reporting of statistics in psychology research. The details of statistical reporting vary slightly among different areas of science and also among different

### 1 Theory: The General Linear Model

QMIN GLM Theory - 1.1 1 Theory: The General Linear Model 1.1 Introduction Before digital computers, statistics textbooks spoke of three procedures regression, the analysis of variance (ANOVA), and the

### Statistical Modelling in Stata 5: Linear Models

Statistical Modelling in Stata 5: Linear Models Mark Lunt Arthritis Research UK Centre for Excellence in Epidemiology University of Manchester 08/11/2016 Structure This Week What is a linear model? How

### EPS 625 ANALYSIS OF COVARIANCE (ANCOVA) EXAMPLE USING THE GENERAL LINEAR MODEL PROGRAM

EPS 6 ANALYSIS OF COVARIANCE (ANCOVA) EXAMPLE USING THE GENERAL LINEAR MODEL PROGRAM ANCOVA One Continuous Dependent Variable (DVD Rating) Interest Rating in DVD One Categorical/Discrete Independent Variable

### 1/27/2013. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Introduce moderated multiple regression Continuous predictor continuous predictor Continuous predictor categorical predictor Understand

### Posttraumatic Therapy

1498 Posttraumatic Therapy PTSD ASD PTSD PTSD PTSD g GABA 1499 posttraumatic stress disorder PTSD Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision DSM IV TR A B C D

### Statistics Review PSY379

Statistics Review PSY379 Basic concepts Measurement scales Populations vs. samples Continuous vs. discrete variable Independent vs. dependent variable Descriptive vs. inferential stats Common analyses

### Introduction Types of mediation Causal steps approach SPSS example Miscellaneous topics Proportion mediated Sobel test. Mediation analysis

Mediation analysis Seminar General Statistics Matthijs J Warrens GION Multiple regression analysis 1 dependent variable (INT) Several independent variables 1, 2, 3 (INT) (predictors) 1 2 3 Research question

### Chapter 6: t test for dependent samples

Chapter 6: t test for dependent samples ****This chapter corresponds to chapter 11 of your book ( t(ea) for Two (Again) ). What it is: The t test for dependent samples is used to determine whether the

### Multivariate analyses

14 Multivariate analyses Learning objectives By the end of this chapter you should be able to: Recognise when it is appropriate to use multivariate analyses (MANOVA) and which test to use (traditional

### ECON Introductory Econometrics. Lecture 17: Experiments

ECON4150 - Introductory Econometrics Lecture 17: Experiments Monique de Haan (moniqued@econ.uio.no) Stock and Watson Chapter 13 Lecture outline 2 Why study experiments? The potential outcome framework.

### SPSS: Descriptive and Inferential Statistics. For Windows

For Windows August 2012 Table of Contents Section 1: Summarizing Data...3 1.1 Descriptive Statistics...3 Section 2: Inferential Statistics... 10 2.1 Chi-Square Test... 10 2.2 T tests... 11 2.3 Correlation...

### 1. Complete the sentence with the correct word or phrase. 2. Fill in blanks in a source table with the correct formuli for df, MS, and F.

Final Exam 1. Complete the sentence with the correct word or phrase. 2. Fill in blanks in a source table with the correct formuli for df, MS, and F. 3. Identify the graphic form and nature of the source

### EFFICACY OF EYE MOVEMENT DESENSITIZATION TREATMENT THROUGH THE INTERNET ABSTRACT

EFFICACY OF EYE MOVEMENT DESENSITIZATION TREATMENT THROUGH THE INTERNET Keiko Nakano Department of Clinical Psychology/Atomi University JAPAN ABSTRACT The effectiveness of an internet based eye movement

### SIMPLE LINEAR CORRELATION. r can range from -1 to 1, and is independent of units of measurement. Correlation can be done on two dependent variables.

SIMPLE LINEAR CORRELATION Simple linear correlation is a measure of the degree to which two variables vary together, or a measure of the intensity of the association between two variables. Correlation

### Reporting and Interpreting Effect Size in Quantitative Agricultural Education Research

Journal of Agricultural Education Volume 52, Number 1, pp. 132 142 DOI: 10.5032/jae.2011.01132 Reporting and Interpreting Effect Size in Quantitative Agricultural Education Research Joe W. Kotrlik, J.

### UNDERSTANDING THE TWO-WAY ANOVA

UNDERSTANDING THE e have seen how the one-way ANOVA can be used to compare two or more sample means in studies involving a single independent variable. This can be extended to two independent variables

### DISCRIMINANT FUNCTION ANALYSIS (DA)

DISCRIMINANT FUNCTION ANALYSIS (DA) John Poulsen and Aaron French Key words: assumptions, further reading, computations, standardized coefficents, structure matrix, tests of signficance Introduction Discriminant

### Calculating Effect-Sizes

Calculating Effect-Sizes David B. Wilson, PhD George Mason University August 2011 The Heart and Soul of Meta-analysis: The Effect Size Meta-analysis shifts focus from statistical significance to the direction

### Standard Deviation Calculator

CSS.com Chapter 35 Standard Deviation Calculator Introduction The is a tool to calculate the standard deviation from the data, the standard error, the range, percentiles, the COV, confidence limits, or

### Chapter 7: Simple linear regression Learning Objectives

Chapter 7: Simple linear regression Learning Objectives Reading: Section 7.1 of OpenIntro Statistics Video: Correlation vs. causation, YouTube (2:19) Video: Intro to Linear Regression, YouTube (5:18) -

### Data analysis process

Data analysis process Data collection and preparation Collect data Prepare codebook Set up structure of data Enter data Screen data for errors Exploration of data Descriptive Statistics Graphs Analysis

### Chapter 15 Multiple Choice Questions (The answers are provided after the last question.)

Chapter 15 Multiple Choice Questions (The answers are provided after the last question.) 1. What is the median of the following set of scores? 18, 6, 12, 10, 14? a. 10 b. 14 c. 18 d. 12 2. Approximately

### Point-Biserial and Biserial Correlations

Chapter 302 Point-Biserial and Biserial Correlations Introduction This procedure calculates estimates, confidence intervals, and hypothesis tests for both the point-biserial and the biserial correlations.

### Multiple Regression in SPSS This example shows you how to perform multiple regression. The basic command is regression : linear.

Multiple Regression in SPSS This example shows you how to perform multiple regression. The basic command is regression : linear. In the main dialog box, input the dependent variable and several predictors.

### c. The factor is the type of TV program that was watched. The treatment is the embedded commercials in the TV programs.

STAT E-150 - Statistical Methods Assignment 9 Solutions Exercises 12.8, 12.13, 12.75 For each test: Include appropriate graphs to see that the conditions are met. Use Tukey's Honestly Significant Difference

### Chapter 7 Section 7.1: Inference for the Mean of a Population

Chapter 7 Section 7.1: Inference for the Mean of a Population Now let s look at a similar situation Take an SRS of size n Normal Population : N(, ). Both and are unknown parameters. Unlike what we used

### What is Effect Size? effect size in Information Technology, Learning, and Performance Journal manuscripts.

The Incorporation of Effect Size in Information Technology, Learning, and Performance Research Joe W. Kotrlik Heather A. Williams Research manuscripts published in the Information Technology, Learning,

### What Happens? Relationship of Age and Gender with Science Attitudes from Elementary to Middle School

Carmen Sorge What Happens? Relationship of Age and Gender with Science Attitudes from Elementary to Middle School This study examines the attitudes of 1008 students from rural New Mexico in elementary

### Introduction. Hypothesis Testing. Hypothesis Testing. Significance Testing

Introduction Hypothesis Testing Mark Lunt Arthritis Research UK Centre for Ecellence in Epidemiology University of Manchester 13/10/2015 We saw last week that we can never know the population parameters

### Correlation and Regression

Dublin Institute of Technology ARROW@DIT Books/Book Chapters School of Management 2012-10 Correlation and Regression Donal O'Brien Dublin Institute of Technology, donal.obrien@dit.ie Pamela Sharkey Scott

### SPSS Tests for Versions 9 to 13

SPSS Tests for Versions 9 to 13 Chapter 2 Descriptive Statistic (including median) Choose Analyze Descriptive statistics Frequencies... Click on variable(s) then press to move to into Variable(s): list

### Randomized Block Analysis of Variance

Chapter 565 Randomized Block Analysis of Variance Introduction This module analyzes a randomized block analysis of variance with up to two treatment factors and their interaction. It provides tables of

### ANOVA ANOVA. Two-Way ANOVA. One-Way ANOVA. When to use ANOVA ANOVA. Analysis of Variance. Chapter 16. A procedure for comparing more than two groups

ANOVA ANOVA Analysis of Variance Chapter 6 A procedure for comparing more than two groups independent variable: smoking status non-smoking one pack a day > two packs a day dependent variable: number of

### Data Analysis. Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) SS Analysis of Experiments - Introduction

Data Analysis Lecture Empirical Model Building and Methods (Empirische Modellbildung und Methoden) Prof. Dr. Dr. h.c. Dieter Rombach Dr. Andreas Jedlitschka SS 2014 Analysis of Experiments - Introduction

### Consider a study in which. How many subjects? The importance of sample size calculations. An insignificant effect: two possibilities.

Consider a study in which How many subjects? The importance of sample size calculations Office of Research Protections Brown Bag Series KB Boomer, Ph.D. Director, boomer@stat.psu.edu A researcher conducts

### Research Methods & Experimental Design

Research Methods & Experimental Design 16.422 Human Supervisory Control April 2004 Research Methods Qualitative vs. quantitative Understanding the relationship between objectives (research question) and

### Independent t- Test (Comparing Two Means)

Independent t- Test (Comparing Two Means) The objectives of this lesson are to learn: the definition/purpose of independent t-test when to use the independent t-test the use of SPSS to complete an independent

### HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HOD 2990 10 November 2010 Lecture Background This is a lightning speed summary of introductory statistical methods for senior undergraduate

### individualdifferences

1 Simple ANalysis Of Variance (ANOVA) Oftentimes we have more than two groups that we want to compare. The purpose of ANOVA is to allow us to compare group means from several independent samples. In general,

### Bill Burton Albert Einstein College of Medicine william.burton@einstein.yu.edu April 28, 2014 EERS: Managing the Tension Between Rigor and Resources 1

Bill Burton Albert Einstein College of Medicine william.burton@einstein.yu.edu April 28, 2014 EERS: Managing the Tension Between Rigor and Resources 1 Calculate counts, means, and standard deviations Produce

### Testing Hypotheses using SPSS

Is the mean hourly rate of male workers \$2.00? T-Test One-Sample Statistics Std. Error N Mean Std. Deviation Mean 2997 2.0522 6.6282.2 One-Sample Test Test Value = 2 95% Confidence Interval Mean of the

### DDBA 8438: The t Test for Independent Samples Video Podcast Transcript

DDBA 8438: The t Test for Independent Samples Video Podcast Transcript JENNIFER ANN MORROW: Welcome to The t Test for Independent Samples. My name is Dr. Jennifer Ann Morrow. In today's demonstration,

### Statistics and research

Statistics and research Usaneya Perngparn Chitlada Areesantichai Drug Dependence Research Center (WHOCC for Research and Training in Drug Dependence) College of Public Health Sciences Chulolongkorn University,

### Eye Movement Desensitization and Reprocessing (EMDR) Theodore Morrison, PhD, MPH Naval Center for Combat & Operational Stress Control. What is EMDR?

Eye Movement Desensitization and Reprocessing (EMDR) Theodore Morrison, PhD, MPH Naval Center for Combat & Operational Stress Control What is EMDR? Eye movement desensitization and reprocessing was developed

### Effectiveness of positive psychology training in the increase of hardiness of female headed households

Effectiveness of positive psychology training in the increase of hardiness of female headed households 1,2, Ghodsi Ahghar* 3 1.Department of counseling, Khozestan Science and Research Branch, Islamic Azad

### EXCEL Analysis TookPak [Statistical Analysis] 1. First of all, check to make sure that the Analysis ToolPak is installed. Here is how you do it:

EXCEL Analysis TookPak [Statistical Analysis] 1 First of all, check to make sure that the Analysis ToolPak is installed. Here is how you do it: a. From the Tools menu, choose Add-Ins b. Make sure Analysis

### Effect Sizes. Null Hypothesis Significance Testing (NHST) C8057 (Research Methods 2): Effect Sizes

Effect Sizes Null Hypothesis Significance Testing (NHST) When you read an empirical paper, the first question you should ask is how important is the effect obtained. When carrying out research we collect

### Multivariate Analysis of Variance (MANOVA): I. Theory

Gregory Carey, 1998 MANOVA: I - 1 Multivariate Analysis of Variance (MANOVA): I. Theory Introduction The purpose of a t test is to assess the likelihood that the means for two groups are sampled from the

### PTSD Policy and Pharmacology Update. Disclosures/Conflicts

PTSD Policy and Pharmacology Update Maryland Pharmacists Association 134th Annual Convention Ocean City, MD June 12, 2016 Marsden McGuire, M.D., M.B.A. Deputy Chief Consultant, Mental Health Services Office

### ANOVA Analysis of Variance

ANOVA Analysis of Variance What is ANOVA and why do we use it? Can test hypotheses about mean differences between more than 2 samples. Can also make inferences about the effects of several different IVs,

### Lecture 7: Binomial Test, Chisquare

Lecture 7: Binomial Test, Chisquare Test, and ANOVA May, 01 GENOME 560, Spring 01 Goals ANOVA Binomial test Chi square test Fisher s exact test Su In Lee, CSE & GS suinlee@uw.edu 1 Whirlwind Tour of One/Two