How Data are Obtained The distinction between observational study and experiment is important in statistics. Observational Study Experiment Observes individuals and measures variables of interest but does not attempt to influence the responses; The purpose is to describes some group or situation; Where? - Sample surveys are one type of observational study. Deliberately imposes some treatment on individuals in order to observe their responses; The purpose is to study whether the treatment causes a change in the response; Where? - Studies whether the treatment causes change in the response. Common: Both typically have the goal of detecting a relationship between the explanatory and response variables. Difference: In contrast to observational studies, experiments don t just observe individuals or ask them questions. They actively impose some treatment in order to measure the response. When our goal is to understand cause and effect, experiments are the only source of fully convincing data. Why Not Always Use an Experiment? Sometimes it is unethical or impossible to assign people to receive a specific treatment. Sometimes too expensive (special equipments, paying individuals,...) Certain explanatory variables are inherent traits and cannot be randomly assigned (e.g. handedness or gender). Vocabulary for Experiments: Subjects - individuals studied in an experiment, particularly when they are people (= experimental units). Factors - the explanatory variables in an experiment (1, 2, or sometimes more factors). Treatment - any specific experimental condition applied to the subjects; if there are several factors, a treatment is a combination of specific values of each factor. Response variable The basic principles of well-design experiment are: Comparative design. Control the effects of lurking variables on the response, most simply by comparing two or more treatments. Randomization. Use chance to assign subjects to treatments. Replication. Use enough subjects in each group to reduce chance variation in the results. 1
Problem 1. A report in Circulation, the journal of the American Heart Association, examined whether taking aspirin helps with the recovery from heart attacks. The subjects were the 539 patients who were admitted with heart attack or stroke symptoms to a University Hospital in Barcelona, Spain, between October 1991 and March 1993. The patients were asked if they had taken aspirin during the prior week: 214 said yes and 325 said no. It was found that the group who had taken aspirin had a much lower amount of permanent damage from their heart attacks than the group who had not taken aspirin. Is this an observational study or an experiment? Explain your answer. Problem 2. A researcher conducts a study to investigate the effect of exercise and diet on mood. She selects three types of exercise: yoga, weight lifting, and aerobics, and two diet types: low sugar, and low fat. A 12-week experiment will compare all combinations of exercise and diet. Each treatment will have 30 different participants. The mood of each participate will be scored before and after each treatment. a) Is this an observational study or an experiment? Explain your answer. b) How many factors are in the study? Specify. c) How many treatments are in the study? Specify. d) How many subjects does the experiment require? e) What is the response variable? Factor 1 (Exercise) Low sugar Factor 2 (Diet) Low fat yoga Tr. 1 Tr. 2 weight lifting Tr. 3 Tr. 4 aerobics Tr. 5 Tr. 6 Confounding A lurking variable is a variable that is not among the explanatory or response variables in a study but that may influence the response variable. Confounding occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other. Well-designed experiments take steps to avoid confounding. 2
Problem 3. Many studies have found that people who drink alcohol in moderation have lower risk of heart attacks than either nondrinkers or heavy drinkers. Does alcohol consumption also improve survival after a heart attack? One study followed 1913 people who were hospitalized after severe heart attacks. In the year before their heart attacks, 47% of these people did not drink, 36% drank moderately, and 17% drank heavily. After four years, fewer of the moderate drinkers had died. a) Is this an observational study or an experiment? Why? b) What are the explanatory and response variables? c) Suggest some lurking variables that may be confounded with the drinking habits of the subjects. The possible confounding makes it difficult to conclude that drinking habits explain death rates. Answers: a) [ in class ] b) The explanatory variable is alcohol consumption, and the response variable is whether or not a subject dies. c) Many answers are possible. For example, some nondrinkers might avoid drinking because of other health concerns. We do not know what kind of alcohol (beer? wine? whiskey?) the subjects were drinking. We don't know diet and lifestyles of people within groups. Design of Experiments The design of an experiment describes the choice of treatments and the manner in which the subjects are assigned to the treatments. Type 1: Randomized Comparative Experiments / Completely Randomized Experimental Design An experiment that uses both comparison of two or more treatments and random assignment of subjects to treatments. So, some units receive one treatment and similar units receive another and then experimental units are assigned to treatments at random, that is, using some sort of chance process. Randomized comparative experiments are designed to give good evidence that differences in the treatments actually cause the differences we see in the response. Example. The college decides to compare the progress of 25 on-campus students taught in the classroom with that of 25 students taught the same material online. Select the students who will be taught online by taking a simple random sample of size 25 from the 50 available subjects. The remaining 25 students form the control group. They will receive classroom instruction. The result is a randomized comparative experiment with two groups. The design in graphical form: 3
The selection procedure is exactly the same as it is for sampling. Label: Label the 50 students 01 to 50. Table: Go to the table of random digits and read successive two-digit groups.the first 25 labels encountered select the online group. As usual, ignore repeated labels and groups of digits not used as labels. For example, if you begin at line 125 in Table B, the first five students chosen are those labeled 21, 49, 37, 18, and 44, and so on. Also experiments in medicine and psychology often give a placebo to a control group because just being in an experiment can affect responses. A placebo is a dummy treatment. Type 2: Double-Blind Experiments In a double-blind experiment, neither the subjects nor the people who interact with them (e.g. doctor) know which treatment each subject is receiving. Problem 4. As men age, their testosterone levels gradually decrease. This may cause a reduction in lean body mass, an increase in fat, and other undesirable changes. Do testosterone supplements reverse some of these effects? A study in the Netherlands assigned 237 men aged 60 to 80 with low or low-normal testosterone levels to either a testosterone supplement or a placebo. The report in the Journal of the American Medical Association described the study as a "double-blind, randomized, placebo-controlled trial". Explain each of these terms to someone who knows no statistics. Answers: "Double-blind" means that the treatment (testosterone or placebo) assigned to a subject was unknown to both the subject and those responsible for assessing the effectiveness of that treatment. "Randomized" means that the patients were randomly assigned to receive either the testosterone supplement or a placebo. "Placebo-controlled" means that some of the subjects were given placebos. Even though these possess no medical properties, some subjects may show improvement or benefits just as a result of participating in the experiment; the placebos allow those doing the study to observe this effect. 4
Type 3: Block Designs A block is a group of individuals that are known before the experiment to be similar in some way that is expected to affect the response to the treatments. In a block design, the random assignment of individuals to treatments is carried out separately within each block. Example. Women and men respond differently to advertising. An experiment to compare the effectiveness of three advertisements for the same product will want to look separately at the reactions of men and women, as well as to assess the overall response to the ads. A completely randomized design considers all subjects, both men and women, as a single pool. The randomization assigns subjects to three treatment groups without regard to their sex. This ignores the differences between men and women. A block design considers women and men separately. Randomly assign the women to three groups, with one to view each advertisement. Then separately assign the men at random to three groups. A block design allows us to draw separate conclusions about each block, for example, about men and women. A wise experimenter will form blocks based on the most important unavoidable sources of variability among the subjects. Type 4: Matched Pairs Design Matched pairs are a common form of blocking design for comparing only two treatments. Choose pairs of subjects that are as closely matched as possible. Use chance to decide which subject in a pair gets the first treatment. The other subject in that pair gets the other treatment. Sometimes each "pair" in a matched pairs design consists of just one subject, who gets both treatments one after the other, also twins could be subjects in such design. Use chance to decide the order in which subjects receive the treatments. Problem 5. Researchers from the United Kingdom studied the effect of two breathing frequencies on performance times and on several physiological parameters in front crawl swimming. The breathing frequencies were one breath every second stroke (B2) and one breath every fourth stroke (B4). Subjects were 10 male collegiate swimmers. Each subject swam 200 meters, once with breathing frequency B2 and once on a different day with breathing frequency B4. Describe the design of this matched pairs experiment, including the randomization required by this design. 5
Answers: Each swimmer swims one time using each breathing technique (B2 and B4). A coin is tossed to determine the order in which these techniques are used. Inference for Experiments In an experiment, researchers usually hope to see a difference in the responses so large that it is unlikely to happen just because of chance variation. We can use the laws of probability, which describe chance behavior, to learn whether the treatment effects are larger than we would expect to see if only chance were operating. If they are, we call them statistically significant. Problem 6. An experiment was conducted by some students to explore the nature of the relationship between a heart rate (measured in beats per minute) and the frequency at which that person stepped up and down on steps of various heights. Two rates of stepping, three different step heights. Forty randomly assigned people in their thirties are subjected to each treatment. Heart rate was then measured before and after treatment. a) How many factors are in the study? Specify. b) How many treatments are in the study? Specify. c) How many subjects does the experiment require? d) What is the response variable? e) What is the population? f) What are the three elements of a well-designed experiment? Explain by this study. Problem 7. An experiment was conducted by some students to explore the nature of the relationship between a person's heart rate (measured in beats per minute) and the frequency at which that person stepped up and down on steps of various heights. Four rates of stepping, three different step heights. Forty randomly assigned people in their thirties are subjected to each treatment. Heart rate was then measured before and after treatment. How many treatments are in the study? Specify. 6