# Testing Scientific Explanations (In words slides page 7)

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Testing Scientific Explanations (In words slides page 7) Most people are curious about the causes for certain things. For example, people wonder whether exercise improves memory and, if so, why? Or, we wonder whether using cell phones causes cancer and, if so, how? Science allows us to systematically test competing hypotheses of such relations and the explanations for them. For example, exercise may affect neurotransmitters or increase blood flow to the brain, either of which could increase memory. Scientists conduct experiments to test their hypotheses. Any conclusions are only as good as the experiments that were used to support the conclusion. In this module, we will teach you to detect problems with experiments that lead to conclusions that seem suspect. A hypothesis specifies a testable relationship between variables. Thus, a hypothesis has three important elements: the independent variable, the dependent variable, and the type of relationship between them. The independent variable (IV from here on) is the condition that we vary to see if it has an effect on the other variable. In the examples above, the independent variables are (A) exercise and (B) cell phone usage, because we think they will causally affect another variable. The dependent variable (DV from here on) is the variable that we think will be affected. It is also called an outcome variable. The dependent variables in the examples above are (A) memory and (B) cancer. Finally, both example hypotheses assume a causal relationship, specifically, causes an increase. You should know that hypotheses are not always about causal relationships. One could also have correlational hypotheses, which merely predict that two variables change in relation to each other, but they do not assume that a change in one causes the change in the other. For example, we could change the verb in our example hypotheses to get these correlational hypotheses: (A) more frequent exercise is associated with better memory or (B) cell phone usage is related to the incidence of cancer. Figure 1. General statement. Conclusion: IV causes DV Conclusion: exercise (IV) causes increase memory (DV) Conclusion: cell phone (IV) causes increase cancer (DV) When evaluating research, the first things you should do are to identify the independent and dependent variables and determine if a causal or correlational prediction is being made. There are more specific issues related to these topics that you should consider when evaluating research. In these pages, you will read about some important flaws that may occur when research is conducted. A. Possible problems with IVs and DVs (1) Use valid measure of IVs and DVs Causal hypotheses predict that a change in the IV will cause a change in the DV. To test such a hypothesis, one therefore has to manipulate or vary an appropriate IV of interest. For the cell phone hypothesis, for example, you could compare cancer rates (DV) for participants who are grouped based on minutes of cell phone usage per day (e.g., more than 2 hours a day,.5-2 hours a day, less than 30 mins a day). This is a valid definition of cell phone usage (IV) because you are varying the amount of usage. An invalid definition of the IV would be the amount of money spent on a cell phone per month (e.g., more than \$100 dollars, \$40-99 dollars, less than \$40 dollars a month). This is invalid because it doesn t really indicate how much time one spends on the phone. You could have a very cheap plan, pay for much more time than you actually use, or call a lot but only on off hours. It is also important to have a valid measure of your DV. For example, if you want to test whether exercise (IV) improves memory (DV), you need to measure memory in a way that accurately reflects the aspect you are interested in. A standard memory test would be a valid measure; weighing someone s head would not. Construct validity. We are often interested in variables that cannot be directly measured or observed, such as memory, intelligence, or depression. We refer to such variables as constructs. When our variables are constructs, we have to rely on scientifically developed measures as indicators of those constructs. For example, we may use a score on an intelligence test to indicate that a person is intelligent. Because these are not direct measures, they always depend on the developer s ideas of what the construct is. Because these can vary, an experimenter has to be very careful in the choice of measure. When a measure really reflects the construct well, we say it has construct validity. Construct 1

3 the improved memories. Random assignment means that everyone in a study has an equal chance of being in any condition, and this allows us to distribute such differences across conditions so that hopefully, the only thing that varies is our manipulation (IV). Thus, even the best comparison group will not allow the researcher to make a causal statement about the relationship between the IV and DV if people are not randomly assigned to condition. (3) Use appropriately sensitive DVs To test a hypothesis, we vary one or more IVs to see whether they affect performance on the DV. We have to make sure that the test or measure we are using for the DV is sensitive enough to detect any possible effect. There are two main flaws that can affect DV sensitivity. Ceiling or floor effects. Ceiling effects happen when a test is so easy that everyone achieves a near perfect score (at the ceiling or top of the scale). So even if you have a perfect treatment, you would never know it because you cannot make people do better than the control condition, which is already performing at the top of the test. Floor effects occur when the test is so hard that participants are not getting any items right. In this case, your manipulation, even if effective, may not be powerful enough to improve the scores even a little. Imprecise measures. The second concern with DV sensitivity is the use of an imprecise measure. This requires the researcher to determine how fine a scale should be used. For example, suppose you want to measure how much someone exercises. You could use a three-point scale (none, some, a lot). This is easy to answer, but if people start to exercise a little more, it is unlikely that this would be reflected by this broad 3-option scale. If, however, you measured exercise on a 100% scale, you would be able to see numbers change representing even slight increases in exercising. B. Problems with eliminating unwanted variables When you test the effect of an IV on some DV, it is important that you control the rest of the environment. There are always other variables that also vary between groups that could affect the outcome, but researchers try to isolate the effect of the IV by minimizing the influence of these other variables. You want to hold as many things constant between groups as possible especially if you think they may affect the DV. Otherwise, you cannot be sure that your varying of the IV(s) was really the reason for any change in performance (DV). For our exercise example, there are several possible variables that we might worry about (such as, motivation, starting health level, previous level of exercise, attitude and experience with exercise, people working out beyond what is required for the study). Two standard practices to help the groups be comparable before a treatment are to randomly assign participants to your groups, and to manipulate the IV rather than using naturally existing groups (which may differ along other variables). While there are many serious flaws that concern controlling the environment, we will only mention two: attrition and experimenter bias. (4) Limit attrition Groups can become different even after random assignment because sometimes participants drop out or fail to complete the experiment. We call this attrition. Attrition can be a serious problem, especially if the number of participants failing to complete the experiment is different for the various conditions. This can lead to differences between groups that affect the DV, which makes it impossible to conclude that the IV was responsible for any DV differences between groups. Attrition is most problematic if it is just one group that looses participants, or if participants with specific characteristics that may affect the DV drop out. Attrition is less of a concern when it is random and not due to the participants, such as equipment failure. However, researchers should always report the level of attrition in all conditions. Drop out. Attrition due to drop out can be a serious flaw. This happens when, after randomly assigning participants to condition or groups, a participant leaves without finishing, or stops showing up for sessions. When this happens, the groups may then be different for a reason other than the manipulation. In the case of the exercise experiment, we might find that unmotivated people may be more likely to drop out of the exercise group than out of the comparison group. This would leave the experimental group with more highly-motivated individuals than the control group. Because highlymotivated individuals will probably try harder on the memory test (DV), we would likely find memory differences between groups even if the exercise has no effect, simply because of the characteristics of those who dropped out. 3

4 Missing information. Attrition due to missing information is usually less of a threat but is still a concern. This happens when a participant completes the experiment but fails to answer every question. In our exercise-memory experiment, we could find missing information if participants do not indicate their gender or age on the pre-interview survey, or more important, they may skip one or more trials in the memory test. Missing information could lead to biased results, for example if people in one group systematically skip specific problems. (5) Limit experimenter bias One of the easiest factors of the environment to control is the effect of the researcher or experimenter. We know from several experiments, that the behavior or expectations of a researcher can have a significant impact on the results of an experiment. When this happens, it is not possible to determine whether the results were due to the IV or to actions of the experimenter. There are two important flaws to consider here. The first type of flaw focuses on our motivation (conscious or unconscious) to bias the results; the second type of flaw focuses on our ability to bias the results. Conflict of interest. Conflict of interest occurs when a researcher has a strong investment (fame or fortune) in a particular outcome of the experiment. This is a problem because the researcher may intentionally or unintentionally bias the experiment. For our exercise example, if we were paid by an exercise equipment company to show that using their equipment leads to better memory, we could have a conflict of interest. You should know that a conflict of interest doesn t guarantee that a researcher will bias the results, but it creates a situation in which a researcher is more motivated to influence the outcome. Opportunity for bias. Opportunity for bias is a flaw in which the experimenter does not take important precautions to reduce his or her chances of affecting the behavior of the participants. The best precaution is using a double-blind technique in which the researcher is unaware of which condition the participant is in. A second method is to reduce contact with the participants. Most experiments are automated so that a computer presents the instructions rather than the experimenter doing so. A third method is to use automated or objective scoring of data. If that is not possible, then two raters should score all data independently (inter-rater reliability) without knowing the participant s condition (doubleblind). For our memory experiment, if we use a recall rather then multiple choice test of memory, then the experimenter will have to determine which recall responses are correct and which are incorrect. This is an opportunity for bias. Therefore, two raters who are unaware of conditions should be used. We need to reduce the opportunity and motivation for experimenters to bias the results of an experiment, or we will not be able to rely on the results. C. Problems with the sample (6) Use appropriate sample size Of course, we can only test a hypothesis on a sample from the larger population. There are several possible flaws that can occur in an experiment that could be due to the characteristics of such a sample. We will only focus on those that relate to the size of the sample. In experiments, we use statistics to compare performance on the DV(s) to see if there is a difference between conditions. The size of the sample that we need to get reliable results is determined by how variable the population is. Variability refers to the extent to which the population s scores on a DV tend to naturally vary, regardless of any sort of manipulation. Let s say we have two conditions (exercise and no exercise) in the memory experiment above. Imagine using a highly variable population such as 5 year-olds worldwide to study the relationship between exercise and memory. It is likely that there is automatically a great deal of difference in the memory scores of all 5 year-olds even prior to any sort of exercise manipulation. When scores vary a lot in a population, and we draw small samples, our groups are likely to be quite different at the outset of the study, because people with extreme scores (high or low) have a big impact on the group average. As we increase sample size, however, individual extreme scores get increasingly balanced out by other scores, leading to more comparable groups. Typically, we have little information about how much a variable varies in a population; so, it is always a good idea to have large samples. There are primarily two flaws associated with an inadequate sample size: lack of power and low generalizability. Small sample size and lack of power. Power, or the likelihood of finding a difference that really does exist between groups, depends on the size of the sample. Power increases with larger samples 4

6 QA9. Using the score card below, determine the flaws, if any, with each experiment description. If there are none, write good experiment. No credit will be given for questions such as how many subjects were there?. It is either clearly a flaw (So do not put in comments like it doesn t say how many subjects there were ). A. Possible problems with IVs and DVS (1) Use valid measure of IVs and DVs manipulate (IV) and measure (DV) the variables you claim you are, especially the most important aspects of the variable. This is important because valid measures are required to be able to appropriately interpret the results from an experiment and draw appropriate conclusions. Flaw: (1) Validity of DV (2) Must have comparison group(s) randomly assign participants to one or more groups that are not exposed to the treatment or are exposed to less of the treatment so you can rule out alternative explanations for any effects. Flaws: (1) Poor or missing comparison group, (2) No random assignment (3) Use appropriately sensitive DVs use tests or measures that people do not score so well or so poorly on that you fail to detect a true treatment effect just because you have insensitive measures. Flaws: (1) Sensitivity of DV, (2) DV is not scored objectively B. Problems with unwanted, extraneous or confounding variables (4) Limit attrition try to limit the number of participants who failed to complete the experiment or who skipped information because this can lead to differences between groups that affect the DV, which makes it impossible to conclude that the IV was responsible for any DV differences between groups. Flaw: (1) Mortality or attrition (5) Limit experimenter bias try to limit the impact of the behaviors or expectations of the researcher on the results of an experiment. When this happens, it is not possible to determine whether the results were due to the IV or to actions of the experimenter. Flaws: (1) Experimenter bias, (2) Conflict of interest C. Problems with the sample (6) Use appropriate sample size make sure that you have a large enough sample to detect a true effect of the treatment and a large enough sample to generalize to the entire population. Flaws: (1) Small sample size; (2) Poor sample selection D. Problems with Conclusions (7) Make tentative conclusions experiments always have limits, and conclusions must reflect the possibility that the conclusions can be shown to be incorrect. Replicating the experiment and getting the same result helps increase our confidence in the results but will never prove the hypothesis true. Flaws: (1) Not a tentative conclusion/ Premature generalization of results (8) Requirements for causal conclusions to make a causal conclusion, one has to randomly assign participants to a valid IV which has at least one comparison condition, measure a DV that is sensitive and valid, and then control unwanted variables (e.g., limit attrition and experimenter bias). Flaw: (1) Inappropriate causal statement 6

7 Scientists are curious about the causes for things. Testing Scientific Explanations E.g., Whether exercise improves memory and, if so, why? Whether using cell phones causes cancer and, if so, how? Science allows us to systematically test competing hypotheses of such relations and the explanations for them. E.g., Exercise neurotransmitters increase memory Exercise more blood to brain increase memory Use scientific method to conduct experiments to test such hypotheses. Our conclusions are only as good as the experiments used to test our hypotheses. Hypothesis a testable relationship between 2 or more variables 3 important elements: 1. Independent variable (IV) 2. Dependent variable (DV) 3. Type of relationship between them 1. IV what we want to see is effective; the thing we vary in an experiment 2. DV the variable that we think will be affected; the outcome variable. 3. Type of relationship causal or correlational Causal: IV causes DV exercise (IV) causes increase memory (DV) cell phone (IV) causes increase cancer (DV) Correlational: E.g., People who exercise more improves memory More frequent cell phone usage is related to the incidence of cancer. 7

8 NONCAUSAL -- Verbs of Association is associated with correlates with goes along with co-occurs with is related to CAUSAL Positive Negative Nondirectional Improve Worsen Affect Better Hinder Have effect on Develop Hurts Have influence on Increase Destroys Have impact on Stop Lead to Impede Cause Weaken Bring about Reduce Lower Decrease Slow Evaluating research ID IVs, DVs Determine if a causal or correlational prediction or conclusion is being made. Then evaluate based on: A. Possible problems with IVs and DVs (1) Use valid measure of IVs and DVs (2) Must have comparison group(s) (3) Use appropriately sensitive DVs B. Problems with eliminating unwanted variables (4) Limit attrition (5) Limit experimenter bias C. Problems with the sample (6) Use appropriate sample size D. Problems with Conclusions (7) Make tentative conclusions (8) Requirements for causal conclusions (1) Use valid measure of IVs and DVs Possible problems with IVs and DVs Constructs scientifically developed measures of variables that cannot be directly measured or observed (e.g., memory, intelligence, or depression). We need to manipulate or vary an appropriate IV. Valid definition of cell phone usage: Cell phone usage per day (e.g., 2+ hours a day,.5-2 hours a day, less than 30 mins a day) 8

9 Invalid definition of cell phone usage: Amount of money spent on a cell phone per month (e.g., more than \$100 dollars, \$40-99 dollars, less than \$40 dollars a month). It doesn t really indicate how much time one spends on the phone. Valid definition of memory improvement: Number of items correctly recalled Invalid definition of memory improvement: Weighing someone s head or self-reported perception of memory improvement * 2 problems with VALIDITY * 1. Construct validity the degree to which the test measures the construct it is supposed to measure. Exercise: hours per week on treadmill (0, 3, 6 h/w) If your experiment does not have good construct validity, then you are not actually manipulating or measuring what you are claiming to and thus, you will not test your actual hypothesis. 2. Content validity the degree to which your measure reflects the actual material, substance, or content of the variables measured Most constructs have many aspects. The more of them you include in your measure or manipulation, the better your content validity becomes. Memory: test several types of memory (e.g., verbal, spatial) Summary: To draw good conclusions, you have to manipulate and measure the variables you claim you are (construct) as completely as possible (content) (2) Must have comparison group(s) Possible problems with IVs and DVs To test a correlational hypothesis, you just compare already existing groups that differ on their level of exercise. exercise (IV) is associated with increase memory (DV) To test a causal hypothesis, you need to manipulate the IV (randomly assign to different levels of the IV) exercise (IV) causes increase memory (DV) For causal conclusions, you need one or more comparison groups that are not exposed to the treatment. 2 Exercise (6 times a week, 0 times a week) 3 Exercise (6 / week, 3 / week, and 1 / week). 9

10 1. No comparison group * 2 problems with COMPARISON GROUPS * Pretest Treatment Post test? Memory test Exercise Memory test It may be that everyone would improve on the second test even without the treatment (e.g., learn to take test, more focused, more motivated) To be able to attribute any change to the IV, we need a baseline (no or less treatment) for comparing change. 2. Poor comparison group. The groups vary on things other than the IV. E.g., if people in the exercise condition work out in a fun group class, then any improvement may be due to the social interaction. Comparison groups allow researchers to rule out alternative explanations for any effects, but only if the comparison group is equivalent in all regards except for the IV. To support causal statements, need random assignment. To make sure the groups are as similar as possible except for the manipulation is to random assign participants to condition (using chance). Contrast: put people into groups based on their natural, pre-existing characteristics, such as how much they usually exercise per week (0 hours, 3 hours, or 6 hours). But because many other things vary with how much someone chooses to exercise per week (e.g., diet, fitness level, motivation level, weight), it could be that one of these other reasons could explain the improved memories. Summary: Need to randomly assign participants to a good comparison group to make causal conclusions. (3) Use appropriately sensitive DVs Possible problems with IVs and DVs To test a hypothesis, we vary one or more IVs to see whether they affect performance on the DV. We have to make sure that the test or measure we are using for the DV is sensitive enough to detect any possible effect. * 2 problems with DV SENSITIVITY * 1. Ceiling or floor effects. People either all do extremely well (i.e., ceiling effects) or do extremely poorly (i.e., floor effects) making it hard to find a real effect Your manipulation, even if actually effective, may not be powerful enough to improve the scores even a little 2. Imprecise measures. The scale is not detailed enough to find a difference. E.g., Categorizing people s recall as: a lot, some, none. It would take a very large jump in memory to go from some to a lot. 10

11 B. Problems with eliminating unwanted variables Isolate variables of interest -- when you test the effect of an IV on some DV, it is important that you control the rest of the environment (i.e., eliminate or minimize the influence of other variables). Goal is the groups are not different before manipulation. o o Randomly assign participants to your groups Manipulate the IV rather than using naturally existing groups (which may differ along other variables). Otherwise, you cannot be sure that your varying of the IV(s) was really the reason for any change in performance (DV). E.g., for exercise and memory what other factors? (motivation, starting health level, previous level of exercise, attitude and experience with exercise). 2 of many serious flaws that concern controlling the environment, we will only mention two: attrition and experimenter bias. (4) Limit attrition B. Problems with eliminating unwanted variables Attrition participants drop out or fail to complete the experiment. Attrition can lead groups to become different. Attrition is most problematic if it is just one group that looses participants, or if participants with specific characteristics that may affect the DV drop out. * 2 problems with ATTRITION * 1. Drop out after randomly assigning participants to condition or groups, a participant leaves without finishing, or stops showing up for sessions. When this happens, the groups may then be different for a reason other than the manipulation. E.g. unmotivated people may be more likely to drop out of the exercise group than out of the comparison group. Therefore, in addition to assigned exercise, groups also differ on motivation level (highly-motivated individuals may try harder on the memory test (DV)) 2. Missing information participant fails to answer every question. Missing information could lead to biased results, if people in one group systematically skip specific problems. B. Problems with eliminating unwanted variables (5) Limit experimenter bias We know from several experiments, that the behavior or expectations of a researcher can have a significant impact on the results of an experiment. When this happens, it is not possible to determine whether the results were due to the IV or to actions of the experimenter. 11

12 * 2 problems with EXPERIMENTER BIAS * 1. Conflict of interest when a researcher has a strong investment (fame or fortune) in a particular outcome of the experiment. E.g., exercise equipment company to show that using their equipment leads to better memory The researcher may intentionally or unintentionally bias the experiment. 2. Opportunity for bias important precautions are not made to reduce chances of the experimenter affecting the behavior. Best precaution is using a double-blind technique in which the researcher is unaware of which condition the participant is in. Also to reduce contact with the participants (e.g., automate experiment) Scoring should be automated or as objective as possible (inter-rater reliability) C. Problems with the sample (6) Use appropriate sample size We use statistics to compare performance on the DV to see if there is a difference between groups. E.g., M (exercise) = 9.4 items correctly recalled vs M (control) = 7.2 items correctly recalled The size of the sample that we need to get reliable results is determined by how variable the population is. Variability refers to the extent to which the population s scores on a DV tend to naturally vary, regardless of any sort of manipulation. Variability is inevitable but the greater the variability, the harder it is to detect a difference. A group of year olds college students will have less variability in memory than a group of year olds from general population. E.g., 2.2 difference may be detectable in the 20 something population due to small variability Other possible flaws such as sample selection but not covered here. * 2 problems with SAMPLE SIZE * 1. Small sample size and lack of power. Power the likelihood of finding a difference that really does exist between groups. Power increases with larger samples because there is less extreme variation in larger groups. Therefore, if you do not find a difference between groups with small samples, it could simply be that you do not have enough power. With many experiments using humans, is often large enough. With potatoes or plants, it 4 or 5 may be enough. (depends on variability) 2. Small sample size and limited generalizability. Generalizability the extent to which the results will likely be true for all members of a population or all materials (recall only test some people and some materials) We may be less confident that our results will generalize to the larger population of interest if we have smaller samples 12

13 Really a problem only when you find a significant difference between groups Scores will vary in a population, and the chances of having unusual results are higher in small samples where extreme scores have a large impact on our statistical measures. (7) Make tentative conclusions D. Problems with Conclusions Instead of talking of proving a hypothesis true, we can only say that our results support the hypothesis Based on probability (e.g.,.05) not deduction. Even in the absence of obvious problems with an experiment, there is always the possibility that some effect was due to chance (e.g., 5/100 conclusion could be wrong) and not to the manipulation. Conclusions must that acknowledge the possibility of future revisions of the hypothesis. * 2 problems with CONCLUSIONS * 1. Conclusions too strong. A researcher fails to acknowledge the possibility of alternative explanations for the results. A researcher s conclusion must always reflect the limitations of an experiment, and the potential for error. 2. Need for replication. Replication involves repeating a study with the same (or very similar) methods but with different participants and sometimes with different researchers. When obtain similar results, we can be more confident that the results generalize to a larger population. (8) Requirements for causal conclusions D. Problems with Conclusions To find support for a hypothesized causal relationship, you need to consider several things. 1st: your IVs and DVs must be valid and sensitive. 2nd: Your IV must be manipulated by the experimenter. Randomly assigned to conditions with at least one condition being a good comparison group. 3rd: You need to eliminate unwanted variables, such as reducing attrition (especially unequal drop out among conditions) and limiting the effects of the experimenter. 4th: You need to make sure to have a large enough sample which depends on the variability in the population. 13

### Internal and External Validity

Internal and External Validity ScWk 240 Week 5 Slides (2 nd Set) 1 Defining Characteristics When research is designed to investigate cause and effect relationships (explanatory research) through the direct

### Step 8: Considering Validity and Discussing Limitations Written and Compiled by Amanda J. Rockinson-Szapkiw and Anita Knight

Step 8: Considering Validity and Discussing Limitations Written and Compiled by Amanda J. Rockinson-Szapkiw and Anita Knight Introduction It is important to think about threats to validity prior to planning

### The Mozart effect Methods of Scientific Research

The Mozart effect Methods of Scientific Research Chapter 2 Experimental Research: p42 49 http://www.mozarteffect.com/ http://www.amazon.com/mozart-sonata-pianos-schubert-fantasia/dp/b0000cf330 http://www.youtube.com/watch?v=hhqn2qjhlcm

### Reliability and Validity. Measurement Error. Random vs. Systematic Error 10/7/10. Random error. Systematic error. Chance fluctuations in measurement

Reliability and Validity Measurement Error Chance fluctuations in measurement More precise measurement Fluctuations noted more easily Random vs. Systematic Error Random error Chance fluctuations in measurement

### In an experimental study there are two types of variables: Independent variable (I will abbreviate this as the IV)

1 Experimental Design Part I Richard S. Balkin, Ph. D, LPC-S, NCC 2 Overview Experimental design is the blueprint for quantitative research and serves as the foundation of what makes quantitative research

### Experimental Design. Overview. Foundations of Quantitative Research. Part I

Experimental Design Part I Richard S. Balkin, Ph. D, LPC-S, NCC Overview Experimental design is the blueprint for quantitative research and serves as the foundation of what makes quantitative research

### Experimental Designs. Y520 Strategies for Educational Inquiry

Experimental Designs Y520 Strategies for Educational Inquiry Experimental designs-1 Research Methodology Is concerned with how the design is implemented and how the research is carried out. The methodology

### Chapter 1. Thinking Critically with Psychological Science

Chapter 1 Thinking Critically with Psychological Science Psychology s Roots Prescientific Psychology Is the mind connected to the body or distinct? Are ideas inborn or is the mind a blank slate filled

### Chpt 1. The Nature of Probability and Statistics. 1-5 Observational and Experimental Studies

Chpt 1 The Nature of Probability and Statistics 1-5 Observational and Experimental Studies 1 /15 Chpt 1 Homework 1-5 Read pages 13-16 p16 Applying the Concepts p26 16-19 2 /15 Chpt 1 Objectives Identify

### Correlational Research

Correlational Research Chapter Fifteen Correlational Research Chapter Fifteen Bring folder of readings The Nature of Correlational Research Correlational Research is also known as Associational Research.

### Welcome back to EDFR 6700. I m Jeff Oescher, and I ll be discussing quantitative research design with you for the next several lessons.

Welcome back to EDFR 6700. I m Jeff Oescher, and I ll be discussing quantitative research design with you for the next several lessons. I ll follow the text somewhat loosely, discussing some chapters out

### Chapter 2 Quantitative, Qualitative, and Mixed Research

1 Chapter 2 Quantitative, Qualitative, and Mixed Research This chapter is our introduction to the three research methodology paradigms. A paradigm is a perspective based on a set of assumptions, concepts,

### psychology the science of psychology CHAPTER third edition Psychology, Third Edition Saundra K. Ciccarelli J. Noland White

psychology third edition CHAPTER 1 the science of psychology Learning Objectives LO 1.1 Definition and Goals of Psychology LO 1.2 Structuralism and Functionalism LO 1.3 Early Gestalt, Psychoanalysis, and

### Test of proportion = 0.5 N Sample prop 95% CI z- value p- value (0.400, 0.466)

STATISTICS FOR THE SOCIAL AND BEHAVIORAL SCIENCES Recitation #10 Answer Key PROBABILITY, HYPOTHESIS TESTING, CONFIDENCE INTERVALS Hypothesis tests 2 When a recent GSS asked, would you be willing to pay

### Experiment 1: The Coin Experiment Introduction to the Scientific Method

5 Learning Objectives and Outcomes: Experiment 1: The Coin Experiment Introduction to the Scientific Method 1. Relating how science works to what scientists do. 2. Defining key terms used by scientists.

### Causal Comparative Research: Purpose

Causal Comparative Research: Purpose Attempts to determine cause and effect not as powerful as experimental designs Alleged cause and effect have already occurred and are being examined after the fact

### Characteristics of Experimental Research

Characteristics of Experimental Research There are several key ideas or characteristics involving experimental research that individuals must be aware of to truly grasp how experimental research differs

### 1.1 Research in Geography [Meaning & Importance]

Department of Geography GEO 271 Everything is related to everything else, but near things are more related than distant things. - Waldo Tobler s First Law of Geography 1.1 Research in Geography [Meaning

### Inclusion and Exclusion Criteria

Inclusion and Exclusion Criteria Inclusion criteria = attributes of subjects that are essential for their selection to participate. Inclusion criteria function remove the influence of specific confounding

### Research Design Concepts. Independent and dependent variables Data types Sampling Validity and reliability

Research Design Concepts Independent and dependent variables Data types Sampling Validity and reliability Research Design Action plan for carrying out research How the research will be conducted to investigate

### Chapter 21. More About Tests and Intervals. Copyright 2012, 2008, 2005 Pearson Education, Inc.

Chapter 21 More About Tests and Intervals Copyright 2012, 2008, 2005 Pearson Education, Inc. Zero In on the Null Null hypotheses have special requirements. To perform a hypothesis test, the null must be

### Three Essential Properties of a Well-Designed Experiment

Experimental Design Three Essential Properties of a Well-Designed Experiment Experimenter must: 1. Systematically vary (manipulate) at least one independent variable 2. Assign participants to experimental

### AP STATISTICS 2007 SCORING GUIDELINES

AP STATISTICS 2007 SCING GUIDELINES Question 2 Intent of Question The three primary goals of this question are to assess a student s ability to: (1) clearly explain the importance of a control group in

### Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur

Analysis of Variance and Design of Experiments-I MODULE IV LECTURE - 19 EXPERIMENTAL DESIGNS AND THEIR ANALYSIS Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur

### Experiment 1: The Penny Experiment. Introduction to The Scientific Method

Learning Objectives and Outcomes: Experiment 1: The Penny Experiment Introduction to The Scientific Method 1. Relating how science works to what scientists do. 2. Defining key terms used by scientists.

### Chapter 8 Lecture 8. Hypothesis Testing, Validity & Threats to Validity

Chapter 8 Lecture 8 Hypothesis Testing, Validity & Threats to Validity Hypothesis Question Initial Idea (0ften Vague) Initial Observations Search Existing Lit. Statement of the problem Operational definition

### RESEARCH METHODS IN I/O PSYCHOLOGY

RESEARCH METHODS IN I/O PSYCHOLOGY Objectives Understand Empirical Research Cycle Knowledge of Research Methods Conceptual Understanding of Basic Statistics PSYC 353 11A rsch methods 01/17/11 [Arthur]

### Observing and describing the behavior of a subject without influencing it in any way.

HOW TO CHOOSE FROM THE DIFFERENT RESEARCH METHODS* The design is the structure of any scientific work. It gives direction and systematizes the research. The method you choose will affect your results and

### Experimental (quantitative) studies: An overview and general issues by Stephanie Tierney

Experimental (quantitative) studies: An overview and general issues by Stephanie Tierney (stephanie.tierney@manchester.ac.uk) Experimental designs are said to be the approach for obtaining information

### Chapter 8: Introduction to Hypothesis Testing

Chapter 8: Introduction to Hypothesis Testing We re now at the point where we can discuss the logic of hypothesis testing. This procedure will underlie the statistical analyses that we ll use for the remainder

### How do we know what we know?

Research Methods Family in the News Can you identify some main debates (controversies) for your topic? Do you think the authors positions in these debates (i.e., their values) affect their presentation

### Journal Article Reporting Standards (JARS)

APPENDIX Journal Article Reporting Standards (JARS), Meta-Analysis Reporting Standards (MARS), and Flow of Participants Through Each Stage of an Experiment or Quasi-Experiment 245 Journal Article Reporting

### 2. Hypotheses reflect past experience with similar questions (educated propositions about cause). 3. Multiple hypotheses should be proposed whenever p

Large Sample tests of hypothesis Main points in this chapter 1. Standard method to test research questions. 2. Discussion - risks involved when decision based on the test is incorrect. 3. Detailed discussion

### Chapter 3: Producing Data (Part 1) Dr. Nahid Sultana

Chapter 3: Producing Data (Part 1) Dr. Nahid Sultana Chapter 3: Producing Data Introduction 3.1 Design of Experiments 3.2 Sampling Design 3.3 Toward Statistical Inference 3.4 Ethics Introduction Anecdotal

### Scientific Methods II: Correlational Research

Scientific Methods II: Correlational Research EXAMPLES "MARRIAGE SLOWS CANCER DEATHS Evidence that married people have a better chance of surviving cancer than do singles means that the unmarried might

### AP PSYCHOLOGY 2006 SCORING GUIDELINES

AP PSYCHOLOGY 2006 SCORING GUIDELINES Question 1 Psychologists use a variety of research methods to study behavior. Three of the main research methods used are Case study Correlational study, and Experiment.

### Research Design Issues in Experimental Studies

Research Design Issues in Experimental Studies Jonathan J. Koehler Northwestern University School of Law jay.koehler@northwestern.edu Nick Schweitzer Arizona State University New College of Interdisciplinary

### Experimental methods. Elisabeth Ahlsén Linguistic Methods Course

Experimental methods Elisabeth Ahlsén Linguistic Methods Course Experiment Method for empirical investigation of question or hypothesis 2 types a) Lab experiment b) Naturalistic experiment Question ->

### Some Basic Threats to Experimental Validity

Department of Psychology and Human Development Vanderbilt University Threats to Validity 1 Introduction 2 3 Demand Characteristics Interaction between and the Experimental Variable The File Drawer Problem

### Social Psychology Chapter 1 David Myers

Social Psychology Chapter 1 David Myers Chapter Outline What is Social Psychology? Major Themes in Social Psychology Values and Social Psychology Is Social Psychology Just Common Sense Research Methods

### Experimental Design Part I

Experimental Design Part I Yi-Ju Li, Ph.D. Department of Biostatistics & Bioinformatics Duke University Medical Center July 13, 2015 Outline Definition of DOE 7/13 (Monday) Part I Definition of Design

### EBM Cheat Sheet- Measurements Card

EBM Cheat Sheet- Measurements Card Basic terms: Prevalence = Number of existing cases of disease at a point in time / Total population. Notes: Numerator includes old and new cases Prevalence is cross-sectional

### Which Design Is Best?

Which Design Is Best? Which Design Is Best? In Investigation 2-8: Which Design Is Best? students will become more familiar with the four basic epidemiologic study designs, learn to identify several strengths

### RESEARCH METHODS IN I/O PSYCHOLOGY

RESEARCH METHODS IN I/O PSYCHOLOGY Objectives Understand Empirical Research Cycle Knowledge of Research Methods Conceptual Understanding of Basic Statistics PSYC 353 11A rsch methods 09/01/11 [Arthur]

About Sampling TABLE OF CONTENTS About Sampling... 1 Why is SAMPLING important?... 1 Common Language... 1 Probability & Nonprobability Sampling... 1 Sampling Methods... 2 Simple Random Sample (SRS)...

### Basic Concepts in Research and Data Analysis

Basic Concepts in Research and Data Analysis Introduction: A Common Language for Researchers...2 Steps to Follow When Conducting Research...3 The Research Question... 3 The Hypothesis... 4 Defining the

### Biodiversity Data Analysis: Testing Statistical Hypotheses By Joanna Weremijewicz, Simeon Yurek, Steven Green, Ph. D. and Dana Krempels, Ph. D.

Biodiversity Data Analysis: Testing Statistical Hypotheses By Joanna Weremijewicz, Simeon Yurek, Steven Green, Ph. D. and Dana Krempels, Ph. D. In biological science, investigators often collect biological

### User Testing & Experiments

User Testing & Experiments Objectives Explain the process of running a user testing or experiment session. Describe evaluation scripts and pilot tests and explain why they are important Explain the difference

### Research Variables. Measurement. Scales of Measurement. Chapter 4: Data & the Nature of Measurement

Chapter 4: Data & the Nature of Graziano, Raulin. Research Methods, a Process of Inquiry Presented by Dustin Adams Research Variables Variable Any characteristic that can take more than one form or value.

### Confidence intervals and hypothesis tests

Patrick Breheny January 19 Patrick Breheny STA 580: Biostatistics I 1/46 Recap In our last lecture, we discussed at some length the Public Health Service study of the polio vaccine We discussed the careful

### Chi Square Analysis. When do we use chi square?

Chi Square Analysis When do we use chi square? More often than not in psychological research, we find ourselves collecting scores from participants. These data are usually continuous measures, and might

### Bock-Ch13 Review.docx

Multiple Choice 1. Which of the following situations qualifies as an observational study? (A) The girls at your high school are surveyed to determine if they believe there is any sexual stereotyping in

### Chapter 4. Examples (details given in class) Who is Measured: Units, Subjects, Participants. Research Studies to Detect Relationships

Announcements: Midterm Monday. Bring calculator and one sheet of notes. No calculator = cell phone! Assigned seats, random ID check. Review Friday. Review sheet posted on website. Mon discussion is for

### CHAPTER 8 TESTING HYPOTHESES. Most of the information presented in the first seven chapters of this text has focused on

CHAPTER 8 TESTING HYPOTHESES Most of the information presented in the first seven chapters of this text has focused on the theoretical and mathematical foundations for a range of statistics beginning with

### Defining Characteristics. Manipulation and Control. Experimental Validity

Defining Characteristics Research designed to investigate cause and effect relationships through the direct manipulation of an independent variable and control of extraneous variables Independent variable

### Table of Random Digits

Table of Random Digits A table can be given that represents a list of completely random digits it is essentially like picking from a hat Each entry is equally likely to be any digit The entries are independent

### IPDET Module 6: Descriptive, Normative, and Impact Evaluation Designs

IPDET Module 6: Descriptive, Normative, and Impact Evaluation Designs Intervention or Policy Evaluation Questions Design Questions Elements Types Key Points Introduction What Is Evaluation Design? Connecting

### Homework #3 is due Friday by 5pm. Homework #4 will be posted to the class website later this week. It will be due Friday, March 7 th, at 5pm.

Homework #3 is due Friday by 5pm. Homework #4 will be posted to the class website later this week. It will be due Friday, March 7 th, at 5pm. Political Science 15 Lecture 12: Hypothesis Testing Sampling

### Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Readings: Ha and Ha Textbook - Chapters 1 8 Appendix D & E (online) Plous - Chapters 10, 11, 12 and 14 Chapter 10: The Representativeness Heuristic Chapter 11: The Availability Heuristic Chapter 12: Probability

### LAB : THE CHI-SQUARE TEST. Probability, Random Chance, and Genetics

Period Date LAB : THE CHI-SQUARE TEST Probability, Random Chance, and Genetics Why do we study random chance and probability at the beginning of a unit on genetics? Genetics is the study of inheritance,

### Scientific Methods in Psychology

Scientific Methods in Psychology Why do research? To demonstrate that psychology is a science! Psychology would like to have the same academic standing as other sciences like biology, chemistry, astronomy,

### Multivariate Analysis of Variance. The general purpose of multivariate analysis of variance (MANOVA) is to determine

2 - Manova 4.3.05 25 Multivariate Analysis of Variance What Multivariate Analysis of Variance is The general purpose of multivariate analysis of variance (MANOVA) is to determine whether multiple levels

### Written Example for Research Question: How is caffeine consumption associated with memory?

Guide to Writing Your Primary Research Paper Your Research Report should be divided into sections with these headings: Abstract, Introduction, Methods, Results, Discussion, and References. Introduction:

### 1. The Scientific Method

1 1. The Scientific Method The distinctive characteristic of science is its methodology. Science is not just a body of knowledge, but knowledge assembled by the application of the scientific methodology.

### Chapter 6 Experiment Process

Chapter 6 Process ation is not simple; we have to prepare, conduct and analyze experiments properly. One of the main advantages of an experiment is the control of, for example, subjects, objects and instrumentation.

### Topic (2) Sampling and Experimental Design. Experimental (or Sampling) Unit is the object or event from which the measurements are taken

Topic (2) Sampling and Experimental Design 2-1 Topic (2) Sampling and Experimental Design Some Definitions Experimental (or Sampling) Unit is the object or event from which the measurements are taken e.g.

### M. Com (Third Semester) Examination, Paper Title: Research Methodology. Paper Code: AS-2375

Model Answer/suggested solution Research Methodology M. Com (Third Semester) Examination, 2013 Paper Title: Research Methodology Paper Code: AS-2375 * (Prepared by Dr. Anuj Agrawal, Assistant Professor,

### TRANSCRIPT: In this lecture, we will talk about both theoretical and applied concepts related to hypothesis testing.

This is Dr. Chumney. The focus of this lecture is hypothesis testing both what it is, how hypothesis tests are used, and how to conduct hypothesis tests. 1 In this lecture, we will talk about both theoretical

### Variables. Defining Variables. Independent and Dependent Variables. Defining Variables. Operational and Conceptual Definitions

A variable is a characteristic of a person, place or a thing that can change (vary) over time or from one situation to another. E.g.: temperature, height, weight, etc Almost anything can be considered

### G544: Examination feedback

G544: Examination feedback Section A: Questions: Q1: State the NULL hypothesis for your practical project (3) For 3 marks both the IV and DV must be operationalised Frequent cause of lost marks: Few students

### Section: 101 (10am-11am) 102 (11am-12pm) 103 (1pm-2pm) 104 (1pm-2pm)

Stat 0 Midterm Exam Instructor: Tessa Childers-Day 1 May 014 Please write your name and student ID below, and circle your section. With your signature, you certify that you have not observed poor or dishonest

### Draft. Another Doctor s Opinion. Another Doctor s Opinion DRAFT for Fellowship Review Page 1 of 5

Another Doctor s Opinion Marvin D. Seppala, MD Chief Medical Officer Hazelden Betty Ford Foundation I hope that this brief writing may provide those who are seeking help a general understanding of the

### What is Psychology? chapter 1

What is Psychology? chapter 1 Overview! The science of psychology! What psychologists do! Critical and scientific thinking! Correlational studies! The experiment! Evaluating findings What is psychology?

### Experimental Designs leading to multiple regression analysis

Experimental Designs leading to multiple regression analysis 1. (Randomized) designed experiments. 2. Randomized block experiments. 3. Observational studies: probability based sample surveys 4. Observational

### Today we begin our discussion of the existence of God.

Aquinas Five Ways Today we begin our discussion of the existence of God. The main philosophical problem about the existence of God can be put like this: is it possible to provide good arguments either

### Lecture 10-12: Non-Experimental, Observational, Quasi- Experimental, and Developmental Designs

Lecture 10-12: Non-Experimental, Observational, Quasi- Experimental, and Developmental Designs I. INTRODUCTION A. Experimental vs. Correlational research For the most part we have been discussing Experimental

### 11.2 POINT ESTIMATES AND CONFIDENCE INTERVALS

11.2 POINT ESTIMATES AND CONFIDENCE INTERVALS Point Estimates Suppose we want to estimate the proportion of Americans who approve of the president. In the previous section we took a random sample of size

### 11. Logic of Hypothesis Testing

11. Logic of Hypothesis Testing A. Introduction B. Significance Testing C. Type I and Type II Errors D. One- and Two-Tailed Tests E. Interpreting Significant Results F. Interpreting Non-Significant Results

### Inferential Statistics. Probability. From Samples to Populations. Katie Rommel-Esham Education 504

Inferential Statistics Katie Rommel-Esham Education 504 Probability Probability is the scientific way of stating the degree of confidence we have in predicting something Tossing coins and rolling dice

### Statistics 2014 Scoring Guidelines

AP Statistics 2014 Scoring Guidelines College Board, Advanced Placement Program, AP, AP Central, and the acorn logo are registered trademarks of the College Board. AP Central is the official online home

### Independent samples t-test. Dr. Tom Pierce Radford University

Independent samples t-test Dr. Tom Pierce Radford University The logic behind drawing causal conclusions from experiments The sampling distribution of the difference between means The standard error of

### The Idea of Probability

AP Statistics 5.1 Reading Guide Name Directions: Read the following pages and then answer the questions at the end. We ll have a short mini-quiz over this material (for Mastery) when we return from Thanksgiving

### PISA Style Scientific Literacy Question

PISA Style Scientific Literacy Question Read the text about Statins Statins are drugs which stop the liver producing too much cholesterol. Doctors say that many heart attacks and strokes are prevented

### Study Design for Chemoprevention. Cancer Epidemiology, Prevention and Control Workshop Shanghai, March 12, 2008

Study Design for Chemoprevention Cancer Epidemiology, Prevention and Control Workshop Shanghai, March 12, 2008 I. INTRODUCTION Experimental studies are conducted to assess the effect of a treatment using

### Understanding and Using The Scientific Method

The Scientific Method by Science Made Simple Made Simple Understanding and Using The Scientific Method Now that you have a pretty good idea of the question you want to ask, it's time to use the Scientific

### Chapter 7 Part 2. Hypothesis testing Power

Chapter 7 Part 2 Hypothesis testing Power November 6, 2008 All of the normal curves in this handout are sampling distributions Goal: To understand the process of hypothesis testing and the relationship

### Question about the History of Psychology Who is considered to have been the Father of the study of Psychology?

EXPLORING PSYCHOLOGY David Myers Thinking Critically With Psychological Science Chapter 1 Psychology s Roots Aristotle (384-322 B.C.) Psychological Science is Born Wundt and psychology s first graduate

### Correlational Research. Correlational Research. Stephen E. Brock, Ph.D., NCSP EDS 250. Descriptive Research 1. Correlational Research: Scatter Plots

Correlational Research Stephen E. Brock, Ph.D., NCSP California State University, Sacramento 1 Correlational Research A quantitative methodology used to determine whether, and to what degree, a relationship

### Sample Size and Power in Clinical Trials

Sample Size and Power in Clinical Trials Version 1.0 May 011 1. Power of a Test. Factors affecting Power 3. Required Sample Size RELATED ISSUES 1. Effect Size. Test Statistics 3. Variation 4. Significance

### "Statistical methods are objective methods by which group trends are abstracted from observations on many separate individuals." 1

BASIC STATISTICAL THEORY / 3 CHAPTER ONE BASIC STATISTICAL THEORY "Statistical methods are objective methods by which group trends are abstracted from observations on many separate individuals." 1 Medicine

### Subject Sampling and Design

Subject Sampling and Design General Control Procedures Control through the selection and assignment of participants Control through specific experimental design Control over subject and experimenter effects

### Single sample hypothesis testing, II 9.07 3/02/2004

Single sample hypothesis testing, II 9.07 3/02/2004 Outline Very brief review One-tailed vs. two-tailed tests Small sample testing Significance & multiple tests II: Data snooping What do our results mean?

### AP Stats- Mrs. Daniel Chapter 4 MC Practice

AP Stats- Mrs. Daniel Chapter 4 MC Practice Name: 1. Archaeologists plan to examine a sample of 2-meter-square plots near an ancient Greek city for artifacts visible in the ground. They choose separate

### Introduction to Statistics

1 Introduction to Statistics LEARNING OBJECTIVES After reading this chapter, you should be able to: 1. Distinguish between descriptive and inferential statistics. 2. Explain how samples and populations,

### Randomized Field Trials and Internal Validity: Not So Fast My Friend

A peer-reviewed electronic journal. Copyright is retained by the first or sole author, who grants right of first publication to the Practical Assessment, Research & Evaluation. Permission is granted to

### UNDERSTANDING ANALYSIS OF COVARIANCE (ANCOVA)

UNDERSTANDING ANALYSIS OF COVARIANCE () In general, research is conducted for the purpose of explaining the effects of the independent variable on the dependent variable, and the purpose of research design

### AP Psychology STUDY GUIDE: Introduction to Psychology

AP Psychology STUDY GUIDE: Introduction to Psychology Directions: Read selections carefully, select the answer that is best in each case. 1. To say that psychology is a science means that: A) psychologists