# Effect Size Calculations and Elementary Meta-Analysis

Save this PDF as:

Size: px
Start display at page:

Download "Effect Size Calculations and Elementary Meta-Analysis"

## Transcription

1 Effect Size Calculations and Elementary Meta-Analysis David B. Wilson, PhD George Mason University August 2011 The End-Game Forest-Plot of Odds-Ratios and 95% Confidence Intervals for the Effects of Cognitive-Behavioral Programs on Recidivism 1

2 Overview Logic of Meta-analysis Effect sizes What are they Common types Basic analysis Mean effect size Confidence interval Homogeneity analysis Random-effects versus Fixed-effect model Moderator analysis A note about software Logic of Meta-analysis Narrative review methods: Focuses on statistical significance Lacks transparency and replicability Weakness of statistical significance: Significant effect is a strong conclusion Non-significant effect is a weak conclusion How do you balance a collection of significant and nonsignificant effects? 2

3 Logic of Meta-analysis Meta-analysis: Focuses on direction and magnitude of effect Approaches task as a research endeavor Examines pattern of evidence across studies Average effect Consistency of effects Relationship between study features and effects Effect Size Encodes relationship of interest into a common index Must be: comparable across studies independent of sample size have a computable standard error Many different effect size indexes Multiple methods of computing each Most common: Correlation coefficient (r ) Standardized mean difference (d or g) Odds-ratio and Risk-ratio 3

4 Computing Effect Sizes Must compute effect size from information provided Conversions from other statistics t-test p-value descriptive statistics etc. Manipulation of data collapsing across subgroups adding \drop-outs" back into the treatment condition Some conversions better than others (algebraic equivalents; rough approximations) Some studies simply do not provide necessary information Standardized Mean Difference Fundamental relationship: Group contrast Continuous dependent variable Logic: scaling effects based on standard deviation Definitional equation: 4

5 Standardized Mean Difference Based on a t-test Based on a correlation Based on 2 by 2 table (dichotomous outcome; logit method) Correlation as Effect Size Fundamental relationship: Two inherently continuous constructs Correlation comes" standardized Example: Relationship between GRE scores and performance in graduate school 5

6 Odds-Ratio Fundamental relationship: Group contrast Dichotomous dependent variable Data can be represented in a 2 by 2 contingency table Odds-ratio effect size computed as: Basics of Meta-Analysis Goal: Describe the distribution, including its mean Establish a confidence interval around the mean Test that the mean differs from zero Test whether studies tell a consistent story (are homogeneous) Explore the relationship between study features and effect size 6

7 Determining the Mean Effect Size Problem: some effect sizes are more accurate than others What we need is an index of precision Standard error is a direct measure of precision Hedges and Olkin solution: Weight by the inverse variance Provides a statistical basis for: Standard error of the mean effect size Confidence intervals Homogeneity testing Some Preliminary Transformations Small sample size bias correction for the standardized mean differences: Fisher's Z r transform of correlations (ES r ): Log transform of Odds-ratios ES OR (also for Risk-ratio): 7

8 Inverse Variance Weights Standardized mean difference ES sm : Correlation ES r (actually, the Fisher's Z r ): Odds-ratio ES OR (actually, the logged odds-ratio): Inverse variance weight w: Almost ready At this point, we have for each study: An effect size An inverse variance weight Problem: statistical models assume independence Only include one effect size per study (or independent sample) Multiple analyses for different subsets of independent effects Different outcome constructs Different time periods 8

9 Inverse Variance Weighted Mean Effect Size Meta-analytic mean effect size is: where ES i is the effect size for each study (i ) and w i is the inverse variance weight Standard error of the mean effect size is: Some Basic Inferential Statistics Confidence intervals can be constructed in the usual manner: And a z-test can be performed as: 9

10 An Example: Group-Based Cognitive-Behavioral Programs for Adult Offenders An Example: Group-Based Cognitive-Behavioral Programs for Adult Offenders 10

11 An Example: Group-Based Cognitive-Behavioral Programs for Adult Offenders Homogeneity Testing Homogeneity analysis tests whether the assumption that all of the effect sizes are estimating the same population mean is a reasonable assumption. If homogeneity is rejected, the distribution of effect sizes is assumed to be heterogeneous. Single mean ES not a good descriptor of the distribution There are real between study differences, that is, studies estimate different population mean effect sizes. Three options: model between study differences fit a random effects model do both 11

12 Computation of the Homogeneity Q Statistic Q is simply a weighted sums-of-squares: There are easier computational formulas: It is distributed as a chi-square with k - 1 degrees of freedom, where k is the number of effect sizes Alternative to Q Q is statistically under-powered when the number of studies is low and when the sample size within the studies is low Larger values of I 2, the more heterogeneity 75%: large heterogeneity 50%: moderate heterogeneity 25%: low heterogeneity 12

13 Random versus Fixed Effects Models Fixed effects model assume: there is one true population effect that all studies are estimating all of the variability between effect sizes is due to sampling error Random effects model assume: there are multiple (i.e., a distribution) of population effects that the studies are estimating variability between effect sizes is due to sampling error + variability in the population of effects Fixed versus Random: Which to Use? A random-effects model becomes a fixed-effect model when distributions is homogeneous Assumptions of fixed effects model rarely plausible Consequence: standard error that is too small; confidence intervals that are too narrow Bottom-line: Use the random-effects model 13

14 Computing a Random Effects Model Fixed effects model: weights are a function of sampling error Random effects model: weights are a function of sampling error + study level variability Thus, we need a new set of weights First, compute τ 2 (random effects variance component): Second, re-compute the inverse variance weights: Third, re-compute meta-analytic results using new weight Moderator Analysis Modeling between study variability Categorical models (analogous to a one-way ANOVA) Regression models Fixed and random effects versions of each (latter often called mixed" models) 14

15 Analog to the ANOVA Useful for a single categorical independent variable Produce a separate mean effect size for each category Recall that Q is a sum-of-squares The total sum-of-squares (Q) can be partitioned Variability between groups (Q between ) Residual variability within groups (Q within ) Q between analogous to an F-test between means Q within assesses whether residual distribution homogeneous Note: in a random effects (mixed effects) version of this, the Q within is not meaningful Analog to the ANOVA Example: Experimental versus Quasi-experimental Studies in the Domestic Violence 15

16 Meta-analytic Regression Conceptually identical to multiple regression Effect size is the dependent variable Study moderator variables are the independent variables Can handle multiple variables simultaneously Don't use standard OLS regression procedures (even if weighted) Must use specialized software Meta-analytic Regression: Example 16

17 Forest Plots Visual representation of results Row for each study that shows study label sample size; may include other information effect size (dot, square, diamond) confidence interval (horizontal line) Row for the overall mean results effect size (dot, square, diam-and) confidence interval (horizontal line) Example Forest-Plot Forest-Plot of Odds-Ratios and 95% Confidence Intervals for the Effects of Cognitive-Behavioral Programs on Recidivism 17

18 Comments on Software Specialized software RevMan (developed for Cochrane) CMA - Comprehensive Meta-analysis Add-ons for Common Statistical Programs Stata - lots of macros available SPSS - macros available from my website SAS - macros available from my website R - lots of procedures available For computing effect sizes CMA RevMan ES Calculator by Wilson Final Comments Methods continue to advance Publication selection-bias should be assessed (not addressed in this course) Methods for analyzing dependent effect sizes actively advancing Common errors Incorrectly computing effect sizes Not recognizing situations were effect sizes can be computed Using fixed-effect models Not using moderator analysis to compare mean effect sizes for study subsets Focusing too much on statistical significance and not size and direction of effect 18

19 P.O. Box 7004 St. Olavs plass 0130 Oslo, Norway

### Calculating Effect-Sizes

Calculating Effect-Sizes David B. Wilson, PhD George Mason University August 2011 The Heart and Soul of Meta-analysis: The Effect Size Meta-analysis shifts focus from statistical significance to the direction

### Meta-Regression CHAPTER 20

CHAPTER 20 Meta-Regression Introduction Fixed-effect model Fixed or random effects for unexplained heterogeneity Random-effects model INTRODUCTION In primary studies we use regression, or multiple regression,

### Chapter 16 Multiple Choice Questions (The answers are provided after the last question.)

Chapter 16 Multiple Choice Questions (The answers are provided after the last question.) 1. Which of the following symbols represents a population parameter? a. SD b. σ c. r d. 0 2. If you drew all possible

### Random and Mixed-effects Modeling

Random and Mixed-effects Modeling Training Sessions: Oslo 009 Overview Effect-size estimates Random-effects model Mixed model Overview Suppose we have computed effect-size estimates from k studies, we

### Study Guide for the Final Exam

Study Guide for the Final Exam When studying, remember that the computational portion of the exam will only involve new material (covered after the second midterm), that material from Exam 1 will make

### CHAPTER 3 COMMONLY USED STATISTICAL TERMS

CHAPTER 3 COMMONLY USED STATISTICAL TERMS There are many statistics used in social science research and evaluation. The two main areas of statistics are descriptive and inferential. The third class of

### UNDERSTANDING THE TWO-WAY ANOVA

UNDERSTANDING THE e have seen how the one-way ANOVA can be used to compare two or more sample means in studies involving a single independent variable. This can be extended to two independent variables

### BIOSTATISTICS QUIZ ANSWERS

BIOSTATISTICS QUIZ ANSWERS 1. When you read scientific literature, do you know whether the statistical tests that were used were appropriate and why they were used? a. Always b. Mostly c. Rarely d. Never

### Meta-Analytic Synthesis of Studies Conducted at Marzano Research Laboratory on Instructional Strategies

Meta-Analytic Synthesis of Studies Conducted at Marzano Research Laboratory on Instructional Strategies By Mark W. Haystead & Dr. Robert J. Marzano Marzano Research Laboratory Englewood, CO August, 2009

### Causal Comparative Research: Purpose

Causal Comparative Research: Purpose Attempts to determine cause and effect not as powerful as experimental designs Alleged cause and effect have already occurred and are being examined after the fact

### 11/20/2014. Correlational research is used to describe the relationship between two or more naturally occurring variables.

Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are highly extraverted people less afraid of rejection

### Inferential Statistics

Inferential Statistics Sampling and the normal distribution Z-scores Confidence levels and intervals Hypothesis testing Commonly used statistical methods Inferential Statistics Descriptive statistics are

### HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION

HYPOTHESIS TESTING: CONFIDENCE INTERVALS, T-TESTS, ANOVAS, AND REGRESSION HOD 2990 10 November 2010 Lecture Background This is a lightning speed summary of introductory statistical methods for senior undergraduate

### T-tests. Daniel Boduszek

T-tests Daniel Boduszek d.boduszek@interia.eu danielboduszek.com Presentation Outline Introduction to T-tests Types of t-tests Assumptions Independent samples t-test SPSS procedure Interpretation of SPSS

### Introduction to Analysis of Variance (ANOVA) Limitations of the t-test

Introduction to Analysis of Variance (ANOVA) The Structural Model, The Summary Table, and the One- Way ANOVA Limitations of the t-test Although the t-test is commonly used, it has limitations Can only

### How to choose a statistical test. Francisco J. Candido dos Reis DGO-FMRP University of São Paulo

How to choose a statistical test Francisco J. Candido dos Reis DGO-FMRP University of São Paulo Choosing the right test One of the most common queries in stats support is Which analysis should I use There

### Simple Linear Regression Inference

Simple Linear Regression Inference 1 Inference requirements The Normality assumption of the stochastic term e is needed for inference even if it is not a OLS requirement. Therefore we have: Interpretation

### Module 9: Nonparametric Tests. The Applied Research Center

Module 9: Nonparametric Tests The Applied Research Center Module 9 Overview } Nonparametric Tests } Parametric vs. Nonparametric Tests } Restrictions of Nonparametric Tests } One-Sample Chi-Square Test

### Chapter 15 Multiple Choice Questions (The answers are provided after the last question.)

Chapter 15 Multiple Choice Questions (The answers are provided after the last question.) 1. What is the median of the following set of scores? 18, 6, 12, 10, 14? a. 10 b. 14 c. 18 d. 12 2. Approximately

### To open a CMA file > Download and Save file Start CMA Open file from within CMA

Example name Effect size Analysis type Level Diastolic BP Mean difference Basic Basic Synopsis This analysis includes five studies where persons who donated a kidney were compared with persons in a control

### One-Way Analysis of Variance

One-Way Analysis of Variance Note: Much of the math here is tedious but straightforward. We ll skim over it in class but you should be sure to ask questions if you don t understand it. I. Overview A. We

### Some Critical Information about SOME Statistical Tests and Measures of Correlation/Association

Some Critical Information about SOME Statistical Tests and Measures of Correlation/Association This information is adapted from and draws heavily on: Sheskin, David J. 2000. Handbook of Parametric and

### Fixed-Effect Versus Random-Effects Models

CHAPTER 13 Fixed-Effect Versus Random-Effects Models Introduction Definition of a summary effect Estimating the summary effect Extreme effect size in a large study or a small study Confidence interval

### fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson

fifty Fathoms Statistics Demonstrations for Deeper Understanding Tim Erickson Contents What Are These Demos About? How to Use These Demos If This Is Your First Time Using Fathom Tutorial: An Extended Example

### Descriptive Statistics

Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

### Analysis of Data. Organizing Data Files in SPSS. Descriptive Statistics

Analysis of Data Claudia J. Stanny PSY 67 Research Design Organizing Data Files in SPSS All data for one subject entered on the same line Identification data Between-subjects manipulations: variable to

### Regression in SPSS. Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology

Regression in SPSS Workshop offered by the Mississippi Center for Supercomputing Research and the UM Office of Information Technology John P. Bentley Department of Pharmacy Administration University of

### Applied Statistics Handbook

Applied Statistics Handbook Phil Crewson Version 1. Applied Statistics Handbook Copyright 006, AcaStat Software. All rights Reserved. http://www.acastat.com Protected under U.S. Copyright and international

### Simple Linear Regression Chapter 11

Simple Linear Regression Chapter 11 Rationale Frequently decision-making situations require modeling of relationships among business variables. For instance, the amount of sale of a product may be related

### AP Statistics: Syllabus 3

AP Statistics: Syllabus 3 Scoring Components SC1 The course provides instruction in exploring data. 4 SC2 The course provides instruction in sampling. 5 SC3 The course provides instruction in experimentation.

### When Does it Make Sense to Perform a Meta-Analysis?

CHAPTER 40 When Does it Make Sense to Perform a Meta-Analysis? Introduction Are the studies similar enough to combine? Can I combine studies with different designs? How many studies are enough to carry

### AP Statistics 2002 Scoring Guidelines

AP Statistics 2002 Scoring Guidelines The materials included in these files are intended for use by AP teachers for course and exam preparation in the classroom; permission for any other use must be sought

### Description. Textbook. Grading. Objective

EC151.02 Statistics for Business and Economics (MWF 8:00-8:50) Instructor: Chiu Yu Ko Office: 462D, 21 Campenalla Way Phone: 2-6093 Email: kocb@bc.edu Office Hours: by appointment Description This course

### Week 7 Lecture: Two-way Analysis of Variance (Chapter 12) Two-way ANOVA with Equal Replication (see Zar s section 12.1)

Week 7 Lecture: Two-way Analysis of Variance (Chapter ) We can extend the idea of a one-way ANOVA, which tests the effects of one factor on a response variable, to a two-way ANOVA which tests the effects

### Introduction to Quantitative Methods

Introduction to Quantitative Methods October 15, 2009 Contents 1 Definition of Key Terms 2 2 Descriptive Statistics 3 2.1 Frequency Tables......................... 4 2.2 Measures of Central Tendencies.................

### Testing Hypotheses using SPSS

Is the mean hourly rate of male workers \$2.00? T-Test One-Sample Statistics Std. Error N Mean Std. Deviation Mean 2997 2.0522 6.6282.2 One-Sample Test Test Value = 2 95% Confidence Interval Mean of the

### UCLA STAT 13 Statistical Methods - Final Exam Review Solutions Chapter 7 Sampling Distributions of Estimates

UCLA STAT 13 Statistical Methods - Final Exam Review Solutions Chapter 7 Sampling Distributions of Estimates 1. (a) (i) µ µ (ii) σ σ n is exactly Normally distributed. (c) (i) is approximately Normally

### Inferential Statistics. Probability. From Samples to Populations. Katie Rommel-Esham Education 504

Inferential Statistics Katie Rommel-Esham Education 504 Probability Probability is the scientific way of stating the degree of confidence we have in predicting something Tossing coins and rolling dice

### Business Statistics. Successful completion of Introductory and/or Intermediate Algebra courses is recommended before taking Business Statistics.

Business Course Text Bowerman, Bruce L., Richard T. O'Connell, J. B. Orris, and Dawn C. Porter. Essentials of Business, 2nd edition, McGraw-Hill/Irwin, 2008, ISBN: 978-0-07-331988-9. Required Computing

### " Y. Notation and Equations for Regression Lecture 11/4. Notation:

Notation: Notation and Equations for Regression Lecture 11/4 m: The number of predictor variables in a regression Xi: One of multiple predictor variables. The subscript i represents any number from 1 through

### UNDERSTANDING ANALYSIS OF COVARIANCE (ANCOVA)

UNDERSTANDING ANALYSIS OF COVARIANCE () In general, research is conducted for the purpose of explaining the effects of the independent variable on the dependent variable, and the purpose of research design

### Lecture - 32 Regression Modelling Using SPSS

Applied Multivariate Statistical Modelling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur Lecture - 32 Regression Modelling Using SPSS (Refer

### , then the form of the model is given by: which comprises a deterministic component involving the three regression coefficients (

Multiple regression Introduction Multiple regression is a logical extension of the principles of simple linear regression to situations in which there are several predictor variables. For instance if we

### 12/31/2016. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 Understand when to use multiple Understand the multiple equation and what the coefficients represent Understand different methods

### Multiple Regression Analysis in Minitab 1

Multiple Regression Analysis in Minitab 1 Suppose we are interested in how the exercise and body mass index affect the blood pressure. A random sample of 10 males 50 years of age is selected and their

### research/scientific includes the following: statistical hypotheses: you have a null and alternative you accept one and reject the other

1 Hypothesis Testing Richard S. Balkin, Ph.D., LPC-S, NCC 2 Overview When we have questions about the effect of a treatment or intervention or wish to compare groups, we use hypothesis testing Parametric

### ABSORBENCY OF PAPER TOWELS

ABSORBENCY OF PAPER TOWELS 15. Brief Version of the Case Study 15.1 Problem Formulation 15.2 Selection of Factors 15.3 Obtaining Random Samples of Paper Towels 15.4 How will the Absorbency be measured?

### DATA COLLECTION AND ANALYSIS

DATA COLLECTION AND ANALYSIS Quality Education for Minorities (QEM) Network HBCU-UP Fundamentals of Education Research Workshop Gerunda B. Hughes, Ph.D. August 23, 2013 Objectives of the Discussion 2 Discuss

### e = random error, assumed to be normally distributed with mean 0 and standard deviation σ

1 Linear Regression 1.1 Simple Linear Regression Model The linear regression model is applied if we want to model a numeric response variable and its dependency on at least one numeric factor variable.

### c. The factor is the type of TV program that was watched. The treatment is the embedded commercials in the TV programs.

STAT E-150 - Statistical Methods Assignment 9 Solutions Exercises 12.8, 12.13, 12.75 For each test: Include appropriate graphs to see that the conditions are met. Use Tukey's Honestly Significant Difference

### Chi-Square & F Distributions

Chi-Square & F Distributions Carolyn J. Anderson EdPsych 580 Fall 2005 Chi-Square & F Distributions p. 1/55 Chi-Square & F Distributions... and Inferences about Variances The Chi-square Distribution Definition,

### Examining Differences (Comparing Groups) using SPSS Inferential statistics (Part I) Dwayne Devonish

Examining Differences (Comparing Groups) using SPSS Inferential statistics (Part I) Dwayne Devonish Statistics Statistics are quantitative methods of describing, analysing, and drawing inferences (conclusions)

### Course Text. Required Computing Software. Course Description. Course Objectives. StraighterLine. Business Statistics

Course Text Business Statistics Lind, Douglas A., Marchal, William A. and Samuel A. Wathen. Basic Statistics for Business and Economics, 7th edition, McGraw-Hill/Irwin, 2010, ISBN: 9780077384470 [This

### Elementary Statistics. Scatter Plot, Regression Line, Linear Correlation Coefficient, and Coefficient of Determination

Scatter Plot, Regression Line, Linear Correlation Coefficient, and Coefficient of Determination What is a Scatter Plot? A Scatter Plot is a plot of ordered pairs (x, y) where the horizontal axis is used

### SPSS: Descriptive and Inferential Statistics. For Windows

For Windows August 2012 Table of Contents Section 1: Summarizing Data...3 1.1 Descriptive Statistics...3 Section 2: Inferential Statistics... 10 2.1 Chi-Square Test... 10 2.2 T tests... 11 2.3 Correlation...

### Topic 6: Randomized Complete Block Designs (RCBD's) [ST&D sections (except 9.6) and section 15.8]

Topic 6: Randomized Complete Block Designs (RCBD's) [ST&D sections 9. 9.7 (except 9.6) and section 5.8] 6.. Variability in the completely randomized design (CRD) In the CRD, it is assumed that all experimental

### STATISTICAL ANALYSIS WITH EXCEL COURSE OUTLINE

STATISTICAL ANALYSIS WITH EXCEL COURSE OUTLINE Perhaps Microsoft has taken pains to hide some of the most powerful tools in Excel. These add-ins tools work on top of Excel, extending its power and abilities

### Chapter Eight: Quantitative Methods

Chapter Eight: Quantitative Methods RESEARCH DESIGN Qualitative, Quantitative, and Mixed Methods Approaches Third Edition John W. Creswell Chapter Outline Defining Surveys and Experiments Components of

### 2. Simple Linear Regression

Research methods - II 3 2. Simple Linear Regression Simple linear regression is a technique in parametric statistics that is commonly used for analyzing mean response of a variable Y which changes according

### Point-Biserial and Biserial Correlations

Chapter 302 Point-Biserial and Biserial Correlations Introduction This procedure calculates estimates, confidence intervals, and hypothesis tests for both the point-biserial and the biserial correlations.

### Overview Classes. 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7)

Overview Classes 12-3 Logistic regression (5) 19-3 Building and applying logistic regression (6) 26-3 Generalizations of logistic regression (7) 2-4 Loglinear models (8) 5-4 15-17 hrs; 5B02 Building and

### Practice 3 SPSS. Partially based on Notes from the University of Reading:

Practice 3 SPSS Partially based on Notes from the University of Reading: http://www.reading.ac.uk Simple Linear Regression A simple linear regression model is fitted when you want to investigate whether

### The scatterplot indicates a positive linear relationship between waist size and body fat percentage:

STAT E-150 Statistical Methods Multiple Regression Three percent of a man's body is essential fat, which is necessary for a healthy body. However, too much body fat can be dangerous. For men between the

### Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Readings: Ha and Ha Textbook - Chapters 1 8 Appendix D & E (online) Plous - Chapters 10, 11, 12 and 14 Chapter 10: The Representativeness Heuristic Chapter 11: The Availability Heuristic Chapter 12: Probability

### Quantitative Data Analysis: Choosing a statistical test Prepared by the Office of Planning, Assessment, Research and Quality

Quantitative Data Analysis: Choosing a statistical test Prepared by the Office of Planning, Assessment, Research and Quality 1 To help choose which type of quantitative data analysis to use either before

### Methods for Meta-analysis in Medical Research

Methods for Meta-analysis in Medical Research Alex J. Sutton University of Leicester, UK Keith R. Abrams University of Leicester, UK David R. Jones University of Leicester, UK Trevor A. Sheldon University

### Statistics for Management II-STAT 362-Final Review

Statistics for Management II-STAT 362-Final Review Multiple Choice Identify the letter of the choice that best completes the statement or answers the question. 1. The ability of an interval estimate to

### Research Methods & Experimental Design

Research Methods & Experimental Design 16.422 Human Supervisory Control April 2004 Research Methods Qualitative vs. quantitative Understanding the relationship between objectives (research question) and

### 1. Why the hell do we need statistics?

1. Why the hell do we need statistics? There are three kind of lies: lies, damned lies, and statistics, British Prime Minister Benjamin Disraeli (as credited by Mark Twain): It is easy to lie with statistics,

### SPSS Tests for Versions 9 to 13

SPSS Tests for Versions 9 to 13 Chapter 2 Descriptive Statistic (including median) Choose Analyze Descriptive statistics Frequencies... Click on variable(s) then press to move to into Variable(s): list

### Technology Step-by-Step Using StatCrunch

Technology Step-by-Step Using StatCrunch Section 1.3 Simple Random Sampling 1. Select Data, highlight Simulate Data, then highlight Discrete Uniform. 2. Fill in the following window with the appropriate

### Chapter 11: Linear Regression - Inference in Regression Analysis - Part 2

Chapter 11: Linear Regression - Inference in Regression Analysis - Part 2 Note: Whether we calculate confidence intervals or perform hypothesis tests we need the distribution of the statistic we will use.

### Consider a study in which. How many subjects? The importance of sample size calculations. An insignificant effect: two possibilities.

Consider a study in which How many subjects? The importance of sample size calculations Office of Research Protections Brown Bag Series KB Boomer, Ph.D. Director, boomer@stat.psu.edu A researcher conducts

### UNDERSTANDING MULTIPLE REGRESSION

UNDERSTANDING Multiple regression analysis (MRA) is any of several related statistical methods for evaluating the effects of more than one independent (or predictor) variable on a dependent (or outcome)

### Complex Data Structures

PART 5 Complex Data Structures Introduction to Meta-Analysis. Michael Borenstein, L. V. Hedges, J. P. T. Higgins and H. R. Rothstein 2009 John Wiley & Sons, Ltd. ISBN: 978-0-470-05724-7 CHAPTER 22 Overview

### Interpretation of forest plots Part I

Interpretation of forest plots Part I 1 At the end of this lecture, you should be able to understand the principles and uses of forest plot. You should also be able to modify the forest plots. 2 What is

### psyc3010 lecture 8 standard and hierarchical multiple regression last week: correlation and regression Next week: moderated regression

psyc3010 lecture 8 standard and hierarchical multiple regression last week: correlation and regression Next week: moderated regression 1 last week this week last week we revised correlation & regression

### Software for Publication Bias

CHAPTER 11 Software for Publication Bias Michael Borenstein Biostat, Inc., USA KEY POINTS Various procedures for addressing publication bias are discussed elsewhere in this volume. The goal of this chapter

### Inferences About Differences Between Means Edpsy 580

Inferences About Differences Between Means Edpsy 580 Carolyn J. Anderson Department of Educational Psychology University of Illinois at Urbana-Champaign Inferences About Differences Between Means Slide

### ANOVA Analysis of Variance

ANOVA Analysis of Variance What is ANOVA and why do we use it? Can test hypotheses about mean differences between more than 2 samples. Can also make inferences about the effects of several different IVs,

### SPSS Explore procedure

SPSS Explore procedure One useful function in SPSS is the Explore procedure, which will produce histograms, boxplots, stem-and-leaf plots and extensive descriptive statistics. To run the Explore procedure,

### Statistics courses often teach the two-sample t-test, linear regression, and analysis of variance

2 Making Connections: The Two-Sample t-test, Regression, and ANOVA In theory, there s no difference between theory and practice. In practice, there is. Yogi Berra 1 Statistics courses often teach the two-sample

### DEPARTMENT OF HEALTH AND HUMAN SCIENCES HS900 RESEARCH METHODS

DEPARTMENT OF HEALTH AND HUMAN SCIENCES HS900 RESEARCH METHODS Using SPSS Session 2 Topics addressed today: 1. Recoding data missing values, collapsing categories 2. Making a simple scale 3. Standardisation

### II. DISTRIBUTIONS distribution normal distribution. standard scores

Appendix D Basic Measurement And Statistics The following information was developed by Steven Rothke, PhD, Department of Psychology, Rehabilitation Institute of Chicago (RIC) and expanded by Mary F. Schmidt,

### ANOVA MULTIPLE CHOICE QUESTIONS. In the following multiple-choice questions, select the best answer.

ANOVA MULTIPLE CHOICE QUESTIONS In the following multiple-choice questions, select the best answer. 1. Analysis of variance is a statistical method of comparing the of several populations. a. standard

### Introduction to Meta-Analysis Charles DiMaggio, PhD

Introduction to Meta-Analysis Charles DiMaggio, PhD If only we knew what we know! Caria O Dell and Jack Grayson Anything you can do, I can do meta! Objectives Define Meta-Analysis Strengths and Limitations

### Chapter 11: Two Variable Regression Analysis

Department of Mathematics Izmir University of Economics Week 14-15 2014-2015 In this chapter, we will focus on linear models and extend our analysis to relationships between variables, the definitions

### Systematic Reviews and Meta-analyses

Systematic Reviews and Meta-analyses Introduction A systematic review (also called an overview) attempts to summarize the scientific evidence related to treatment, causation, diagnosis, or prognosis of

### INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA)

INTERPRETING THE ONE-WAY ANALYSIS OF VARIANCE (ANOVA) As with other parametric statistics, we begin the one-way ANOVA with a test of the underlying assumptions. Our first assumption is the assumption of

### 0.1 Multiple Regression Models

0.1 Multiple Regression Models We will introduce the multiple Regression model as a mean of relating one numerical response variable y to two or more independent (or predictor variables. We will see different

### Bivariate Analysis. Comparisons of proportions: Chi Square Test (X 2 test) Variable 1. Variable 2 2 LEVELS >2 LEVELS CONTINUOUS

Bivariate Analysis Variable 1 2 LEVELS >2 LEVELS CONTINUOUS Variable 2 2 LEVELS X 2 chi square test >2 LEVELS X 2 chi square test CONTINUOUS t-test X 2 chi square test X 2 chi square test ANOVA (F-test)

### Simple Linear Regression in SPSS STAT 314

Simple Linear Regression in SPSS STAT 314 1. Ten Corvettes between 1 and 6 years old were randomly selected from last year s sales records in Virginia Beach, Virginia. The following data were obtained,

### Chi-Square. The goodness-of-fit test involves a single (1) independent variable. The test for independence involves 2 or more independent variables.

Chi-Square Parametric statistics, such as r and t, rest on estimates of population parameters (x for μ and s for σ ) and require assumptions about population distributions (in most cases normality) for

### Review Statistics review 9: One-way analysis of variance Viv Bewick 1, Liz Cheek 1 and Jonathan Ball 2

Review Statistics review 9: One-way analysis of variance Viv Bewick 1, Liz Cheek 1 and Jonathan Ball 1 Senior Lecturer, School of Computing, Mathematical and Information Sciences, University of Brighton,

### Categorical Variables in Regression: Implementation and Interpretation By Dr. Jon Starkweather, Research and Statistical Support consultant

Interpretation and Implementation 1 Categorical Variables in Regression: Implementation and Interpretation By Dr. Jon Starkweather, Research and Statistical Support consultant Use of categorical variables

### Data and Regression Analysis. Lecturer: Prof. Duane S. Boning. Rev 10

Data and Regression Analysis Lecturer: Prof. Duane S. Boning Rev 10 1 Agenda 1. Comparison of Treatments (One Variable) Analysis of Variance (ANOVA) 2. Multivariate Analysis of Variance Model forms 3.

### Assessing Forecasting Error: The Prediction Interval. David Gerbing School of Business Administration Portland State University

Assessing Forecasting Error: The Prediction Interval David Gerbing School of Business Administration Portland State University November 27, 2015 Contents 1 Prediction Intervals 1 1.1 Modeling Error..................................

### Introduction to Meta-Analysis

Introduction to Meta-Analysis Michael Borenstein Larry Hedges Hannah Rothstein www.meta-analysis.com Page 1 Dedication Dedicated in honor of Sir Iain Chalmers and to the memory of Dr. Thomas Chalmers.