Topic (2) Sampling and Experimental Design. Experimental (or Sampling) Unit is the object or event from which the measurements are taken

Transcription

1 Topic (2) Sampling and Experimental Design 2-1 Topic (2) Sampling and Experimental Design Some Definitions Experimental (or Sampling) Unit is the object or event from which the measurements are taken e.g. tree where the number of leaves is measured leaf where the weight is measured acre of land where the number of trees is measured hurricane where maximum inches of rain/hour is recorded year where the number of hurricanes of class 4 is recorded A Variable is the information recorded on an experimental unit e.g. Y = number leaves/tree Y = weight of leaf Y = number of trees/acre Y = max inches of rain/hour for a hurricane Y = number of hurricanes of class 4/ year NOTE: There can be several variables measured at once on the same experimental unit

2 Topic (2) Sampling and Experimental Design 2-2 e.g. number of leaves on the tree, the diameter at breast height of the tree, the species, etc. Categorical Variables are variables that can only take on values that are non-numeric categories or groups. The set of possible values should be mutually exclusive and exhaustive. Mathematical operations cannot be performed on categorical variables even if coded with numbers. e.g. color of a rose make of a car ethnic group species of tree name Quantitative Variables are numeric in value and hence have meaning as a measurement or quantity. Further, arithmetic operations can be performed on quantitative variables. e.g. number of car accidents on campus in July length of an antenna weight of a book kilowatts used in an hour number of segments in the abdomen of a arthropod

3 Topic (2) Sampling and Experimental Design 2-3 1) Discrete Variable quantitative variable which can take on only certain values number of car accidents on campus in July number of segments in the abdomen of a arthropod size class of a fish when recorded to the nearest 5 cm 2) Continuous Variable quantitative variable which can take on any value in one or more intervals length of an antenna weight of a book kilowatts used in an hour Population: the complete set of values of one or more variables that could be measured on the entire set of experimental units under study. The definition of population depends on the particular research problem under study and on the method of sampling from that population. Sample: the set of values of one or more variables that were measured on a subset of the population of interest.

4 Topic (2) Sampling and Experimental Design 2-4 EXAMPLE Research problem: effect of a drug on blood plasma cholesterol level in people who present at the doctor s office with high levels Population: the cholesterol levels before and after drug treatment of individuals who 1) have high cholesterol levels and 2) could be picked for the study Sample: the before and after blood plasma cholesterol levels of the 100 people selected to participate in he study EXAMPLE Research problem: effect of a herbicide on corn yields in 100 m 2 plots Population: set of corn yields for all 100 m 2 plots that could have been selected for herbicide treatment Sample: set of corn yields for the 100 m 2 plots that were selected for herbicide treatment

5 Topic (2) Sampling and Experimental Design 2-5 RESEARCH STUDIES An important part of statistics is the data collection effort. It is done in order to draw some conclusions or make some inferences about the population being sampled. To do this one develops and carries out a RESEARCH STUDY. The study describes the methods for collecting and analyzing the data in order to test the hypotheses of interest. Two kinds of research studies: OBSERVATIONAL STUDIES are those in which data are observed on a sample of the population. Here the usual interest is either describing the population or comparing two or more populations. A problem with observational studies is the inability to assign causality. Effects of environmental conditions on lung cancer rates.

6 Topic (2) Sampling and Experimental Design 2-6 EXPERIMENTAL STUDIES are planned and are studies in which units are manipulated or treated in order to observe the responses to the treatment. Comparing damage by the corn earworm on corn treated with one of three levels of pesticide. EXAMPLE: Suppose you are interested in studying the current diet of wild striped bass in the Chesapeake Bay and whether diet varies by region. So, you identify five sites at which you will collect fish and determine their stomach contents. Data will be collected in summer only. You will record the length, weight, body condition index, stomach weight, stomach contents by weight within categories, and you intend to age the fish using otoliths. Type of Research Project: observational Variables: Length, weight, body condition index, stomach weight, stomach content by species, weight of each species found in the stomach, age, location of the capture of the fish, date of capture, any variables describing the location where the fish was caught such as water temperature, etc.

7 Topic (2) Sampling and Experimental Design 2-7 Sampling Unit: an individual fish Population: the values of the variables under study for all fish that could have been caught at these 5 locations at the time of the study NOTE: please note the explicitness of this description. It does not extend to other fish at other locations and times since we did not randomly select these 5 sites or the times of collection (summer). The sites and times were chosen specifically for their physico-chemical characteristics and so all we can really determine is estimation of the characteristics of fish at those locations at that time. Sample: The fish caught during the study. The Population Size is the number of individuals, objects or events in your population. When the population is countable, the size is denoted with N. E.g. for the striped bass study, we might have that N = 132,879.

8 Topic (2) Sampling and Experimental Design 2-8 The Sample Size is the number of individuals, objects or events in the sample and is denoted with n. E.g. for the striped bass study, n = 150. Sampling in Observational vs Experimental Studies a) Observational Study: Example - A study of the relationship between height above mean low water and length of the shell of the intertidal limpet, Acmaea testudinalis. Here the researcher would likely select several different rocky sites and sample limpets at several different heights in the intertidal zone at each site. Population: the values of (X,Y,Z)=(height above MLW, shell length, site) for every limpet now or will be living in a rocky intertidal zone in the area of interest to the researcher. Sample (two-stage sampling): a sample of locations was first selected and then at each location a sample of (X,Y,Z) from limpets was observed.

9 Topic (2) Sampling and Experimental Design 2-9 In general, sampling strategies for observational studies must consider how to select a sample o good representation of the population o no systematic bias o small sampling variability o cost constraints (time, money, feasibility) how large a sample to select. b) Experimental Study: Example - A study of the effect of food type on growth of the limpet. Here the researcher might collect young limpets of the same size and assign each one to receive a specific food (algae type). At the end of the experiment the scientist compares mean body weight for limpets on the 3 food treatments. Population: Three populations, one for each algae type body weights (X) for all limpets that could be fed algae #1, 2, or 3. (Note that these are conceptual populations and may or may not really exist in nature) Samples: Three one from each population. One of the biggest differences between the two types of studies is that the scientist controls to which

10 Topic (2) Sampling and Experimental Design 2-10 group (population) the experimental units are assigned. Hence, in addition to the method of selecting limpets to be used in the experiment the researcher must also assign treatments to the limpets in such a way as to have good representation of the true effects of the treatments unbiased results minimize variability sufficient numbers of animals in each treatment

11 Topic (2) Sampling and Experimental Design 2-11 Random Sampling SAMPLING Defn: A SIMPLE RANDOM SAMPLE (SRS) of n items is one in which a) every member of the population is equally likely to be included in the sample and b) the units in the sample are selected independently of each other. An alternative definition is that every possible sample of n units is equally likely to be observed. In some ways, SRS is the gold standard for sampling. 1) good representation every sample is equally likely to be the one picked. As a result, no single sample deliberately misrepresents the population 2) unbiased there is no systematic sampling effort to either underestimate or overestimate the population quantity of interest. Sampling is completely independent of the variable being studied and of any variables correlated with the one of interest.

12 Topic (2) Sampling and Experimental Design 2-12 How does one actually take a random sample of size n from a population? 1) Typically, we have a list (frame) containing ID numbers for every element in the population. e.g. Every cow in a dairy herd on a specific farm has a unique ear tag. Every person in the US has a social security number. 2) Use a random number generator (table of random digits, computer, calculator) to get a list of n randomly generated ID numbers. List of n numbers Random Sample EXAMPLE Suppose we wish to compare finishing weight for pigs given one of two possible diet formulations. Our population consists of 5000 swine with ear tag numbers between 1 and We intend to put 10 on diet 1 and 10 on diet 2 for the experiment. The 20 pigs are to be randomly selected and treatments will then be randomly assigned to each pig.

13 Topic (2) Sampling and Experimental Design 2-13 We would use a random number table to pick 20 pigs. First, randomly select a page and then a spot on the page and a direction from that spot. Then start recording the first 4 numbers of each random number listed. If the number is outside of the range, then throw it out and continue until you get the number of random numbers required. Some random numbers between 1 and 5000 are:

14 Topic (2) Sampling and Experimental Design , 3169, 0911, 0218, 0465, 4331, 1255, This is a random sample taken without replacement from a population of N=5000 units How might we assign these pigs to treatments? Flip a coin and heads are assigned treatment one and tails, treatment 2. Comments: 1) The reality is that we almost never have ideal conditions for taking random samples. BUT an objective researcher makes every effort to ensure that any samples taken or treatment assignments are as random as possible. 2) By taking random samples we ensure that our samples are unbiased ( x is a good estimator of µ) we can estimate the population variability (s is a good estimator of σ).

15 Topic (2) Sampling and Experimental Design ) Random sampling is not the only sampling (or even the most common) technique used. Others include stratified sampling (taking random samples within each subgroup of a population), e.g. dividing the intertidal zone into three levels (High, medium and low) and taking random samples within each level cluster sampling (random selections of whole groups of units), e.g. rather than randomly selecting individual limpets at random, pick whole rocks from the intertidal zone and measure every limpet found on the sampled rocks. Cluster = rock = primary sampling unit The secondary sampling units are the limpets systematic sampling (sampling every j th unit in the ordered list of units),

16 Topic (2) Sampling and Experimental Design 2-16 e.g. rather than selecting limpets at random over the site, take limpets in small squares placed every 5 meters apart along the shoreline. If you use a different sampling method you need to use different sample statistics as well in order to estimate the unknown population quantities (especially the population variability σ). Important Point: All of the statistical concepts and methods we cover in this class assume that units were randomly selected or randomly assigned to treatments in experiments. Should you decide to use these methods you must do random sampling in order for the analyses to be valid. If you do otherwise, then the methods could either underestimate or overestimate the population quantities of interest (biased estimates!) and as a result give invalid conclusions to any tests.

17 Topic (2) Sampling and Experimental Design 2-17 EXPERIMENTAL DESIGN Defn: An experiment is a planned intervention undertaken to observe the effects of one or more explanatory variables, called FACTORS, on a response variable. Any particular combination of values of the factors is called an EXPERIMENTAL CONDITION or TREATMENT. The DESIGN of an experiment is the plan for conducting the experiment. EXAMPLE: Suppose you wish to study the effects of hours of sleep and of several different levels of carbohydrates in breakfast on math test scores for freshmen in college. Assume the test will be taken at 10:30am. We shall suppose we have a large random pool of students who would participate. Response Variable: Math test score Factors: A) Hours of Sleep B) Carbohydrate Level Hours of sleep will be categorized into 4 levels: <4 hrs, 4-6 hrs, 6-8 hrs, and > 8 hours.

18 Topic (2) Sampling and Experimental Design 2-18 Carbohydrate levels will be controlled at 10, 20, 30 and 40% of breakfast calories. Ten students will be assigned to each combination of hours of sleep and breakfast carbohydrates. The design to be used is called a COMPLETELY RANDOMIZED BALANCED 3 4 FACTORIAL It s called a complete factorial experiment because every combination of levels, i.e. every treatment, is assigned to one or more students. Called balanced because equal numbers of students are assigned to each of the 12 treatments. Called randomized because one of the 12 treatments is randomly assigned to each student participant.

19 Topic (2) Sampling and Experimental Design 2-19 It is designed to test the hypotheses: 1) Does any kind of sleep deprivation affect test scores? Compare no deprivation to each of other types 2) Does breakfast carbohydate level affect performance? Look at the relationship of scores to sugar level. 3) Do the two factors interact in the sense that the effect of carbohydrate level depends on the hours of sleep gotten the night before? Look at whether the relationship of scores to sugar level depends on the hours of sleep gotten. A Different Example Suppose a farmer is interested in the best level of herbicide to place on her fields. Too little and there is competition from weeds which reduces yield. Too much and it suppresses the food plant being grown which reduces yield.

20 Topic (2) Sampling and Experimental Design 2-20 The farmer has 8 fields (the columns in the figure below), each with 20 acres, spread out over a large region which includes a gradual elevation change from west to east. She plans to apply one of 4 levels of pesticide (0.5X, 0.75X, 1X, 1.25X) to the fields. How might the farmer assign treatments to 5 acres plots (the 4 boxes in each column) in such a way as to minimize effects of topography and at the same time assign herbicide levels as randomly as possible? 0.5X 0.75X 0.75X 1X 0.75X etc 1X 1.25X 1X 1.25X 1.25X 0.75X 0.5X 0.5X 0.75X 0.5X 1.25X 1X 1.25X 0.5X 1X Note that each column has all 4 treatments and that they were randomly assigned to each square in each column. Such a technique is called BLOCKING since you are creating blocks (the columns) which are similar for some variable that is likely very correlated with the response variable of interest. Here the variable that makes the units

21 Topic (2) Sampling and Experimental Design 2-21 within blocks similar is elevation since we were told that the elevation changed from west to east. This is an example of an experimental design known as RESTRICTED RANDOMIZATION since you are randomizing the treatments to the experimental units but only within small groupings of similar units. Many other experimental designs could be used, some very complicated. The basic idea is to determine the effect of some treatment and to control extraneous sources of variability that could obscure the effect of interest.