1 Introduction to the Design of Experimental and Observational Studies Chapter 15 Lecture 5 Psych 791
2 Is Experimental Design Important? A well designed study will make analysis easy. A poorly designed study can make even the biggest differences impossible to detect. The proper design of a study is often more important than the actual analysis. How are statistics and design related?
3 Relationship Stats & Design Thinking from a statistical consulting prospective, most people think they can just collect the data any way they want, and just tell someone to analyze it. Some of the biggest problems from a consulting prospective is having data that you just can t analyze. Your role as a statistician really begins when you plan your study.
4 Today s Lecture Today we are going to talk about some types of studies that exist, some buzz words and terminology related to experimental design, and just get you familiar with the different types of studies that exist. You will use this knowledge the rest of this semester (and your research career) in thinking about the proper way to analyze data. The way the data is collected plays a MAJOR role in the way you analyze it. We are going to learn some experimental design issues and the rest of the semester we will learn how to analyze data collected from these particular designs.
5 Experimental Studies Experimental studies designed to test a cause and effect relationship. Causality can be inferred from results. You alter levels of some IV to see how it effects your DV. Example: Give people a different number of beers (controlled) to see how their performance on a test changes.
6 Experimental Studies Buzz words Experimental group subjects with treatment (have beer). Control group subject absent of treatment (no beer). Treatments levels of the DV (number of beers). Experimental units pc for subjects, or whatever you are testing on.
7 Randomization This buzz word gets it s own page because it is just that important. Randomization is important for experimental studies so that you control for any other variable that may effect your DV. When you randomize, you KNOW the beer is what caused the changes in the test. Randomization is a necessity for any statistical analysis. If you don t randomize, you need to control for other factors in your model that may have an effect.
8 Beer Study Example Let s think of a good way to design this beer study. Now, let s think of some problems that could be created in our study if you fail to randomize.
9 Observational Study Main difference between obs and exp ---- Randomization. Basically, we study the way the IV and DV variables change together, but do not control the levels of the IV. How would we change our beer/test example into an observational study?
10 Mixing the two We can also do both at the same time. This would be just as it sounds, we would control and randomize some IV and observe some other variables. How could we alter our beer/test example to be a mixed design study.
11 Experimental Design The design of an experiment refers to the structure of the experiment and the following: Set of explanatory factors. Set of treatments. Set of experimental units. Rules and procedures for random assignment. Outcome measures.
12 Factors Factors big word for explanatory variable (IV) (Beer in our example). Experimental Factor controlled. Observational Factor not controlled. Qualitative Factor levels. Quantitative Factor measured. If you talk about a factor level, that is the particular level of the factor. If you measure gender, a factor level is female.
13 Crossed & Nested Factors Crossed Factor There are subjects in every level of every factor. Nested Factor There are not subjects in every level of every factor. Example Have people drink beer (either 2 or 4 beers) and Eat pretzels (either 1 bag or 2 bags). Crossed (2X2 design four conditions with subjects in them: drink 2 eat 1, drink 2 eat 2, drink 4 eat 1, drink 4 eat 2). Nested (have 2 conditions with subjects: drink 2 eat 1, drink 4 eat 2).
14 Treatments When designing a study, you need to determine how many treatments you want/need. Need to decide: Number of factors Number of levels of each factor Range of levels for quantitative factor Control treatment (if needed)
15 More buzz words Experimental units smallest unit which the experimental treatment can be assigned. Most of the time it a person (or intro psych undergrad). Sample Size number of subjects (or EU) that you have. This is often important for statistical considerations. Replication You often can give subjects more than one treatment. The replication of the order of the treatments is often a factor in your model.
16 Randomization Take 2 Randomization is so important it gets another slide. Again, randomization is important to run almost any statistical analysis. Need to make sure subjects are assigned to condition randomly, they are equally likely to be in any condition. If you put all the females into condition a and all the males into condition b, how do you know if it was the condition that caused the change or their gender? Blocking First, you divide subjects into blocks (some variable you want to control for), then run the same experiment on each block So, you make 2 blocks of subjects (1 male block, 1 female block), then you run the entire experiment on both of the blocks.
17 Measurement How you measure the variables is also important. You want to stay away from measurement bias a bias in the way a variable is measured that will alter your results. An experimenter may often be guilty of experimenter bias (I know it is the treatment so I think it will do better). Double-blind studies are often performed (experimenter and subject both do not know which condition they are in).
18 Standard Experimental Designs To follow are some more buzz words which you should become familiar with. To follow are some experimental designs that are often used and analyzed in psychology. The experimental design dictates the type of analysis that is/can be performed.
19 Completely Randomized Design Every subject is randomly assigned to one treatment condition (may have multiple factors and multiple levels of each factor). We then can look at a linear statistical model that accounts for this randomization: Y = [constant] + [treatment] + [error]
20 Factorial Experiments Completely randomized designs can have a single factor or multiple factors. Those with multiple factors are referred to as completely randomized factorial designs (ex. 2X2, 4X6X2). The structure of the linear model is as follows: Y = [constant] + [first order] + [interaction] + [error]
21 Randomized Complete Block Design Puts together the idea of a randomized design and a blocking design. So you create the blocks, and then randomize blocks into the treatment conditions (so subjects are in blocks, then the blocks are in the treatment conditions). Model for this would be: Y = [constant] + [treatment] + [block] + [error]
22 Nested Designs These are designs in which the subjects are nested within some of the factor levels. First, you have some factor, say classroom, and you want to look at the students in each classroom. The students are nested within the classroom (not randomized and you cannot have students cross over to the other classroom).
23 Repeated Measures Design This is a design where all subjects receive every experimental condition. You wear out the subjects. Another more complicated design includes randomization and repeated measures. Split-Plot design (popular at the University of Illinois at the morrow plots). Create treatments, randomize and replicate.
24 Morrow Plots
25 University of Illinois
26 Incomplete Block Design Take a Randomized Complete Block Design, then just take some of the blocks out of each condition. So, you block the subjects, then take each of the blocks and randomly assign them to a subset of the conditions (for example, you have 10 blocks and 5 conditions, but due to financial limitations you can only give each block 3 conditions, give each block 3 conditions so that you have an equal number of blocks in each condition)
27 Fractional Factorial Design You take a factorial design and then delete some of the levels to make data collection easier, but still test to factor levels. If you have a 2X2X5 design, you don t want to collect 20 conditions, so you carefully select an appropriate subset of conditions.
28 Response Surface Experiments For use when all factors are quantitative and you want to determine precisely the factor level that leads to the optimum response. You are truly looking at a response surface instead of a function because all variables are quantitative.
29 Observational Studies The next set of experiments are observational in nature and do not address a cause and effect relationship. Only establish a relationship between variables.
30 Cross-Sectional Study Observational measurements taken from one or more populations at a single time point. So, you can think of stopping time at some moment, then collecting all the observations you can (or are of interest to you) from your subjects at that moment. Not controlling the variables, simply observing them
31 Prospective Study One or more groups are formed in a nonrandom manner according to the levels of a factor, then these groups are observed over time. You think gender might cause some differences in behaviors over time, separate by gender, then just observe.
32 Retrospective study Defined by some outcome, then look back and collect data from earlier time point. Observe people who either ate cake or didn t, then look back at what led them to eat cake.
33 Matching Matching is the observational equivalent to blocking. You take out additional error in the model by matching subjects based on certain factors. For instance, you may match subjects in two groups based on gender, age, etc.
34 Example 1 An economist compiled data on productivity improvements last year for a sample of firms producing electronic computing equipment. The firms were classified according to the level of their average expenditures for research and development in the past three years (low, moderate, high). Is this study experimental, obervational, or mixed? What are the factors and factor levels?
35 Example 2 In a study to investigate the effect of color of paper (blue, green, orange) on response rates for questionnaires distributed by the windshield method in supermarket parking lots, four supermarket parking lots were chosen in a metropolitan area and 10 questionnaires of each color were assigned at random to cars in the parking lots. Is this study experimental, observational, or mixed? Identify all factors and factor levels.
36 Next Time Chapter 16 Single Factor Studies. Regression (but now with dummy variables).
Statistics Notes Revision in Maths Week 1 Section - Producing Data 1.1 Introduction Statistics is the science that studies the collection and interpretation of numerical data. Statistics divides the study
Research Methods Family in the News Can you identify some main debates (controversies) for your topic? Do you think the authors positions in these debates (i.e., their values) affect their presentation
1.1 What is statistics? Data Collection Chapter 1 A science (bet you thought it was a math) of Collecting of data (Chap. 1) Organizing data (Chap. 2) Summarizing data (Chap. 2, 3) Analyzing data (Chap.
1 Chapter 2 Quantitative, Qualitative, and Mixed Research This chapter is our introduction to the three research methodology paradigms. A paradigm is a perspective based on a set of assumptions, concepts,
Welcome back to EDFR 6700. I m Jeff Oescher, and I ll be discussing quantitative research design with you for the next several lessons. I ll follow the text somewhat loosely, discussing some chapters out
Announcements: Midterm Friday. Bring calculator and one sheet of notes. Can t use the calculator on your cell phone. Assigned seats, random ID check. Review Wed. Review sheet posted on website. Fri discussion
MAT 1000 Mathematics in Today's World We talked about Cryptography Last Time We will talk about probability. Today There are four rules that govern probabilities. One good way to analyze simple probabilities
DESCRIPTIVE RESEARCH DESIGNS Sole Purpose: to describe a behavior or type of subject not to look for any specific relationships, nor to correlate 2 or more variables Disadvantages since setting is completely
Research Design Concepts Independent and dependent variables Data types Sampling Validity and reliability Research Design Action plan for carrying out research How the research will be conducted to investigate
The Effect of Dropping a Ball from Different Heights on the Number of Times the Ball Bounces Or: How I Learned to Stop Worrying and Love the Ball Comment [DP1]: Titles, headings, and figure/table captions
Experimental Design Three Essential Properties of a Well-Designed Experiment Experimenter must: 1. Systematically vary (manipulate) at least one independent variable 2. Assign participants to experimental
How Data are Obtained The distinction between observational study and experiment is important in statistics. Observational Study Experiment Observes individuals and measures variables of interest but does
Chapter 10 - Practice Problems 1 1. A researcher is interested in determining if one could predict the score on a statistics exam from the amount of time spent studying for the exam. In this study, the
Elementary Statistics Chapter 1 Dr. Ghamsary Page 1 Elementary Statistics M. Ghamsary, Ph.D. Chap 01 1 Elementary Statistics Chapter 1 Dr. Ghamsary Page 2 Statistics: Statistics is the science of collecting,
Carolyn Anderson & Youngshil Paek (Slides created by Shuai Sam Wang) Department of Educational Psychology University of Illinois at Urbana-Champaign Key Points 1. Data 2. Variable 3. Types of data 4. Define
Hints for Success on the AP Statistics Exam. (Compiled by Zack Bigner) The Exam The AP Stat exam has 2 sections that take 90 minutes each. The first section is 40 multiple choice questions, and the second
Survey Research Types of Surveys Surveys are classified according to their focus and scope (census and sample surveys) or according to the time frame for data collection (longitudinal and cross-sectional
AP Statistics 2014 Scoring Guidelines College Board, Advanced Placement Program, AP, AP Central, and the acorn logo are registered trademarks of the College Board. AP Central is the official online home
Parametric and Nonparametric: Demystifying the Terms By Tanya Hoskin, a statistician in the Mayo Clinic Department of Health Sciences Research who provides consultations through the Mayo Clinic CTSA BERD
206 RESEARCH MET HODOLOGY Non-random/non-probability sampling designs in quantitative research N on-probability sampling designs do not follow the theory of probability in the choice of elements from the
CURRICULUM FOR STATISTICS & PROBABILITY GRADES 11 & 12 This curriculum is part of the Educational Program of Studies of the Rahway Public Schools. ACKNOWLEDGMENTS Christine H. Salcito, Director of Curriculum
ROCHESTER INSTITUTE OF TECHNOLOGY COURSE OUTLINE FORM COLLEGE OF SCIENCE John D. Hromi Center for Quality and Applied Statistics NEW (or REVISED) COURSE: COS-STAT-701 Foundations of Experimental Design
Data Types By Tanya Hoskin, a statistician in the Mayo Clinic Department of Health Sciences Research who provides consultations through the Mayo Clinic CTSA BERD Resource. Don t let the title scare you.
A Survey of Social Media Use in Software Systems Development Sue Black University of Westminster Department of Information and Software Systems London HA TP +()9 email@example.com Rachel Harrison Oxford
AP Statistics 2012 Scoring Guidelines The College Board The College Board is a mission-driven not-for-profit organization that connects students to college success and opportunity. Founded in 1900, the
When to Use Which Statistical Test Rachel Lovell, Ph.D., Senior Research Associate Begun Center for Violence Prevention Research and Education Jack, Joseph, and Morton Mandel School of Applied Social Sciences
Testing Scientific Explanations (In words slides page 7) Most people are curious about the causes for certain things. For example, people wonder whether exercise improves memory and, if so, why? Or, we
Midterm Review Problems October 19, 2013 1. Consider the following research title: Cooperation among nursery school children under two types of instruction. In this study, what is the independent variable?
1 Experimental Design Part I Richard S. Balkin, Ph. D, LPC-S, NCC 2 Overview Experimental design is the blueprint for quantitative research and serves as the foundation of what makes quantitative research
Prospective, retrospective, and cross-sectional studies Patrick Breheny April 3 Patrick Breheny Introduction to Biostatistics (171:161) 1/17 Study designs that can be analyzed with χ 2 -tests One reason
Introduction to nonparametric regression: Least squares vs. Nearest neighbors Patrick Breheny October 30 Patrick Breheny STA 621: Nonparametric Statistics 1/16 Introduction For the remainder of the course,
ANOVA Analysis of Variance What is ANOVA and why do we use it? Can test hypotheses about mean differences between more than 2 samples. Can also make inferences about the effects of several different IVs,
Qualitative vs Quantitative research & Multilevel methods How to include context in your research April 2005 Marjolein Deunk Content What is qualitative analysis and how does it differ from quantitative
University of California, Davis Department of Statistics Summer Session II Statistics 13 August 20, 2012 Date of latest update: August 20 Lecture 5: Estimation with Confidence intervals 5.1 Identifying
Experimental Design and Hypothesis Testing Rick Balkin, Ph.D. 1 Let s s review hypothesis testing and experimental design 3 types of hypothesis testing in experimental research: z-test t-test F-test Balkin,
Biost/Stat 578 B: Data Analysis Emerson, September 29, 2003 Handout #1 Organizing Your Approach to a Data Analysis The general theme should be to maximize thinking about the data analysis and to minimize
Research Methods II 99 10. Analysis of Longitudinal Studies Repeat-measures analysis This chapter builds on the concepts and methods described in Chapters 7 and 8 of Mother and Child Health: Research methods.
Introduction: Statistics, Data and Statistical Thinking The First Measured Century FREC 408 Dr. Tom Ilvento 213 Townsend Hall firstname.lastname@example.org http://www.udel.edu/frec/ilvento http://www.pbs.org/fmc/index.htm
Nonexperimental Research Designs and Survey Research Katie Rommel-Esham Education 504 Research Design Research design deals with the ways in which data are gathered from subjects Relationships in Nonexperimental
Research Methods & Experimental Design 16.422 Human Supervisory Control April 2004 Research Methods Qualitative vs. quantitative Understanding the relationship between objectives (research question) and
Comparing Disabled and Non- Disabled Students Reasons for Studying Psychology Lucy Zinkiewicz (School of Psychology, Deakin University, Australia; email email@example.com) & James Short (School of Psychology,
A CRIME HAS BEEN COMMITTED QUICK PEEK In this lesson, students will demonstrate all of the techniques they have learned during the lessons Ink Chromatography, What Could It Be? Glass Chip Density, Forensics,
Overview of Non-Parametric Statistics PRESENTER: ELAINE EISENBEISZ OWNER AND PRINCIPAL, OMEGA STATISTICS About Omega Statistics Private practice consultancy based in Southern California, Medical and Clinical
Psychology 205: Research Methods in Psychology Using R to analyze the data for study 2 Department of Psychology Northwestern University Evanston, Illinois USA November, 2012 1 / 38 Outline 1 Getting ready
Basic Concepts in Research and Data Analysis Introduction: A Common Language for Researchers...2 Steps to Follow When Conducting Research...3 The Research Question... 3 The Hypothesis... 4 Defining the
Operations and Supply Chain Management Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology Madras Lecture - 41 Value of Information In this lecture, we look at the Value
UCLA STAT 13 Statistical Methods - Final Exam Review Solutions Chapter 7 Sampling Distributions of Estimates 1. (a) (i) µ µ (ii) σ σ n is exactly Normally distributed. (c) (i) is approximately Normally
Research Methods Courses ACCTG 501 ADTED 550 ADTED 551 A ED 502 AEE 520 AEE 521 AEREC 510 AEREC 511 APLNG 578 APLNG 581 BB H 505 Research Methods in Accounting Qualitative Research in Adult Ed (Introduction
Fun with Fractions: A Unit on Developing the Set Model: Unit Overview www.illuminations.nctm.org Number of Lessons: 7 Grades: 3-5 Number & Operations In this unit plan, students explore relationships among
RACHEL DUNIFON How to Read a Research Article The goal of this Research Brief is to provide information that will make reading a research article more illuminating. For example, you might want to learn
UNDERSTANDING ANALYSIS OF COVARIANCE () In general, research is conducted for the purpose of explaining the effects of the independent variable on the dependent variable, and the purpose of research design
Research design and methods Part II Dr Brian van Wyk POST-GRADUATE ENROLMENT AND THROUGHPUT From last week Research methodology Quantitative vs. Qualitative vs. Participatory/action research Research methods
USING DIRECTED ONLINE TUTORIALS FOR TEACHING ENGINEERING STATISTICS Richard J. School of Mathematics and Physics, The University of Queensland, Australia firstname.lastname@example.org Since 2006, an internet
Quantitative Prepared by: Amanda J. Rockinson-Szapkiw Liberty University A research design is a plan that guides the decision as to: when and how often to collect data what data to gather and from whom
Title: Using the Area on a Pie Chart to Calculate Probabilities Mathematics Content: Pie Charts; Area as Probability; Probabilities as Percents, Decimals & Fractions Objectives: To calculate probability
Togo / Student 1 Basic questions Which university did you attend? University of Lome (TOGO) Did you have a choice of universities to attend, and if so why did you choose the school you attended? No, there
Name Two-Way ANOVA Lab: Interactions Perhaps the most complicated situation that you face in interpreting a two-way ANOVA is the presence of an interaction. This brief lab is intended to give you additional
2.2 Master of Science Degrees The Department of Statistics at FSU offers three different options for an MS degree. 1. The applied statistics degree is for a student preparing for a career as an applied
Chapter 1 Assignment Part 1 Careers in Psychology 1. Which of the following psychological professionals must always have a medical degree? a. psychologist b. psychiatric social worker c. psychiatrist d.
Statistical Foundations: Measurement Scales Psychology 790 Lecture #1 8/24/2006 Today s Lecture Measurement What is always assumed. What we can say when we assign numbers to phenomena. Implications for
Chapter 415 Multivariate Analysis of Variance (MANOVA) Introduction Multivariate analysis of variance (MANOVA) is an extension of common analysis of variance (ANOVA). In ANOVA, differences among various
June 23, 2012 1 review session Basic Data Analysis Stephen Turnbull Business Administration and Public Policy Lecture 12: June 22, 2012 Review session. Abstract Quantitative methods in business Accounting
COMP6053 lecture: Relationship between two variables: correlation, covariance and r-squared email@example.com Relationships between variables So far we have looked at ways of characterizing the distribution
Causality and Treatment Effects Prof. Jacob M. Montgomery Quantitative Political Methodology (L32 363) October 21, 2013 Lecture 13 (QPM 2013) Causality and Treatment Effects October 21, 2013 1 / 19 Overview
DATA COLLECTION AND ANALYSIS Quality Education for Minorities (QEM) Network HBCU-UP Fundamentals of Education Research Workshop Gerunda B. Hughes, Ph.D. August 23, 2013 Objectives of the Discussion 2 Discuss
Graduate Certificate in Systems Engineering Systems Engineering is a multi-disciplinary field that aims at integrating the engineering and management functions in the development and creation of a product,
A Primer on Mathematical Statistics and Univariate Distributions; The Normal Distribution; The GLM with the Normal Distribution PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 4: September
FINANCE AND ACCOUNTING OUTSOURCING AN EXPLORATORY STUDY OF SERVICE PROVIDERS AND THEIR CLIENTS IN AUSTRALIA AND NEW ZEALAND. Graham Ray, Accounting Lecturer, School of Commerce and Management, Southern
Statistical Foundations: and Central Tendency and and Lecture 4 September 5, 2006 Psychology 790 Lecture #4-9/05/2006 Slide 1 of 26 Today s Lecture Today s Lecture Where this Fits central tendency/location
NON-PROBABILITY SAMPLING TECHNIQUES PRESENTED BY Name: WINNIE MUGERA Reg No: L50/62004/2013 RESEARCH METHODS LDP 603 UNIVERSITY OF NAIROBI Date: APRIL 2013 SAMPLING Sampling is the use of a subset of the
guidance for research and evaluation in Fife What this is about? Sampling allows you to draw conclusions about a particular population by examining a part of it. When carrying out a survey, it is not usually
Instrumental Variables Regression Instrumental Variables (IV) estimation is used when the model has endogenous s. IV can thus be used to address the following important threats to internal validity: Omitted
RP7-1 Using Proportions to Solve Percent Problems I Pages 46 48 Standards: 7.RP.A. Goals: Students will write equivalent statements for proportions by keeping track of the part and the whole, and by solving
1 An Introduction to Secondary Data Analysis What Are Secondary Data? In the fields of epidemiology and public health, the distinction between primary and secondary data depends on the relationship between
1 Spanish Speaking test Practice Form XX General Directions This is a SOPI practice test designed by CeLTA to help MSU students improve their knowledge of how the test format works and hopefully increase
: Calculating Probabilities of Events Using Two- Way Tables Student Outcomes Students calculate probabilities given a two-way table of data. Students construct a hypothetical 1000 two-way table given probability
How to Use the Research Design Algorithm Below is a diamond by diamond guide for using the Research Design Algorithm developed by the Academy of Nutrition and Dietetics, 2010. Included are Tips on what
HOW TO WRITE A LABORATORY REPORT Pete Bibby Dept of Psychology 1 About Laboratory Reports The writing of laboratory reports is an essential part of the practical course One function of this course is to
The Analysis of Research Data The design of any project will determine what sort of statistical tests you should perform on your data and how successful the data analysis will be. For example if you decide