Sampling Frames: Peter M. Lance, PhD, UNC CPC Baltimore, Thursday September 17

Transcription

1 Sampling Frames: Peter M. Lance, PhD, UNC CPC Baltimore, Thursday September 17

2 Plan of this discussion Introduce the basics of sampling Describe sampling frames and how to sample from them (as well as the consequences of the different sampling approaches) Discuss stratification

3 Sampling: Sampling The selection of individual observations from a population of interest. The sample is selected so that, through it, we can learn something about that population. Example: We may want to learn about the contraceptive prevalence rate (CPR) among women aged We can t study the entire population of women aged so we select a sample of year old women from it. But how do we go about gathering a sample of them so that we can learn something about their CPR?

4 Two Major Considerations When Deciding How to Sample From a Population Bias: Will the sample of individual observations that we gather provide us with an unbiased estimate of the indicator for the population of interest? Key Idea: Estimate is Correct on average across samples Efficiency: Will the sample of individual observations that we gather provide estimates of the indicator that are as precise as possible? Key Idea: Estimate varies as little as possible from sample to sample

5 Probability versus Non-Probability Sampling Probability Sampling: 1.) every unit in the population has some chance of being selected; 2.) this probability is accurately known. Can produce unbiased estimates of indicators; Can produce weights.

6 Probability versus Non-Probability Sampling Non-Probability Sampling: -or- 1.) Some elements of the population have no chance of selection; 2.) the probabilities of selection for elements of the population is not known. Examples: Quota sampling, convenience sampling, snowball sampling.

7 Probability versus Non-Probability Sampling Non-Probability Sampling (cont d): 1. Can lead to a distorted representation of the population of interest; 2. You don t have any information by which you could correct for this (by assigning more weight to some observations than others). Biased Estimates

8 So, How Does One Do Probability Sampling? Randomization: This is the key element in the sample selection process in probability sampling. Randomization means that units of observation are randomly selected from the population.

9 Sampling Frames Probability sampling tends to rely on: 1. Well defined lists of (ideally) all of the members of a population of interest. 2. Well defined sample selection procedures to insure random selection of individual observations from the list for inclusion in the sample

10 Sampling Frame: Simplest Definition A sampling frame is: 1. A list of the units of observation of a population of interest called sampling units (e.g. all of the women aged 15-49) 2. The list must include the information required to randomly choose a sample from that list according to your sample selection rules

11 Example: Getting the Contraceptive Prevalence Rate in One Neighborhood Suppose we wanted to know about the CPR among women aged in one neighborhood. We had a list of all women (suppose there are 15,000) between the ages of 15 and 49 in the neighborhood We need a sample of 5000 women

12 The Sampling Frame Woman # Name Address 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street Dana Merson 95 Herndon Street

13 The Sampling Frame Woman # Name Address 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street Dana Merson 95 Herndon Street So how would we select a sample of, say, 5,000 sampling units/women from a list like this?

14 The Sampling Frame Woman # Name Address 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street Dana Merson 95 Herndon Street So how would we select a sample of, say, 5,000 sampling units/women from a list like this? Systematic Random Sampling

15 Simple Systematic Random Sampling: Key Steps Planned number of observations to be selected: This is simply the required number of sampling units you wish to draw from your sampling frame (5000). Sampling Interval (SI): Interval separating selections, (SI=3=15,000/5,000). We will select sampling unit (i + SI) after selecting the i th sampling unit in the frame Random Start (RS): First woman to be selected. Between 1 st woman on list and the Sl: 2 Then select RS, RS+SI, RS+2*SI, RS+3*SI, etc. until you have 5000 observations

16 The Sampling Frame Woman # Name Address Selected 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue RS (2) 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street Dana Merson 95 Herndon Street

17 The Sampling Frame Woman # Name Address Selected 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue RS (2) x 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street Dana Merson 95 Herndon Street

18 The Sampling Frame Woman # Name Address Selected 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue RS (2) x 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue RS +SI (2+3=5) 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street Dana Merson 95 Herndon Street

19 The Sampling Frame Woman # Name Address Selected 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue RS (2) x 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue RS +SI (2+3=5) x 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street Dana Merson 95 Herndon Street

20 The Sampling Frame Woman # Name Address Selected 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue RS (2) x 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue RS +SI (2+3=5) x 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street RS+2SI (2+6=8) Dana Merson 95 Herndon Street

21 The Sampling Frame Woman # Name Address Selected 1 Emily Jackson 110 Smith Avenue 2 Kara Choi 133c Smith Avenue RS (2) x 3 Amelia Darby 47 Lacebark Street 4 Jennifer Towers 115 Smith Avenue 5 Patricia Clark 127 Smith Avenue RS +SI (2+3=5) x 6 Beverly Wright 41 Lacebark Avenue 7 Tanya Edelman 118 Smith Avenue 8 Mary Calhoun 77 Herndon Street RS+2SI (2+6=8) x Dana Merson 95 Herndon Street

22 Limits to this Approach to Building a Sampling Frame It is too cumbersome for really large populations Example: there are probably about 35 million people in urban Uttar Pradesh; too many women aged to list Key information might not be accessible (Eg Census privacy issues) or very timely (women move)

23 Multi-Stage Sampling Design Divide the population into mutually exclusive and exhaustive groups (primary sampling units (PSUs)), then select a sample of PSUs. In selected PSUs, build sampling frames of secondary sampling units (SSUs) and select SSUs.

24 Classic Example: Traditional Cluster Sampling Divide the geographic area in which a population of interest lives into mutually exclusive and exhaustive areas/clusters which will serve as the primary sampling units (PSUs). Build a list of these PSUs and select a sample of them. Conduct a household listing in each selected cluster. Randomly select a fixed number of households (the SSUs) from each selected PSU and interview all women aged in them.

25 Example Sampling Frame: a PSU Sampling Frame for Metropolitan Lagos, Nigeria Cluster Number Local Government Area 1 Agege 2 2 Ikeja 3 3 Lagos Island 1 4 Mushin 7 5 Lagos Island 2 6 Alimosho 8 7 Ojo 1 8 Alimosho 8 9 Somolu Apapa 5 Ward

26 Key Features 1. Divides Metropolitan Lagos into mutually exclusive and exhaustive primary sampling units (the clusters). 2. Identifies which cluster is which (would also need maps).

27 Problems with Systematic Simple Random Sampling in this Example 1. It will not automatically lead to unbiased estimates of the contraceptive prevalence rate. Key: In earlier case of women in neighborhood, all women had a an equal probability of selection. The result is a self-weighted sample: one in which all units of observation have same overall probability of selection into the final sample. With this multi-stage design, women from larger clusters less likely to be chosen than women from smaller ones. Maybe contraceptive behavior different in more densely population clusters? Remedy: Compute weights using the household listing in selected PSUs.

28 Problems with Systematic Simple Random Sampling in this Example 2. It will not result in the most efficient estimates of the CPR. What does this mean? That, were we to repeatedly draw samples of the same size, the variation in the estimate of the CPR would be comparatively large But why? Because the selection probability does not take into account size, from sample to sample their will be excessive variation in the proportions of PSUs of different size selected. Remedy: Compute weights using the household listing in selected PSUs. This will help some.

29 Selection with Probability Proportional to Size (PPS) Under PPS the probability of selection for any given PSU depends on its population size: larger PSUs have a greater probability of being selected. Can lead to a self-weighting sample, eliminating the need for weights. (Why?) Generally provides more efficient (ie less variation from sample to sample) estimates. Requires more information than simple random sampling: need to know the population size of each primary sampling unit

30 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster Number Local Government Area Ward 1 Agege Ikeja Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Apapa Number of Households

31 Prepare the Sampling Frame: PPS: Key Steps Calculate cumulative household size column

32 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative 1 Agege Ikeja Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Apapa

33 Prepare the Sampling Frame: Key Concepts Calculate cumulative household size column Sampling Interval (SI): Suppose we want 1000 PSUs. Then SI is 1,450,000/2,000=725 Random Start (RS): Between first 0 cumulative population and SI: 502 Then select the PSU for which RS is in the cumulative size, the PSU for which RS+SI is in the cumulative size, the PSU for which RS+2*SI is in the cumulative size, etc. until you have 2000 PSUs

34 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative Chosen 1 Agege Ikeja Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Apapa

35 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative Chosen 1 Agege Ikeja RS (502) is here 3 Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Apapa

36 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative Chosen 1 Agege Ikeja RS (502) is here x 3 Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Apapa

37 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative Chosen 1 Agege Ikeja RS (502) is here x 3 Lagos Island Mushin Lagos Island RS+SI ( = 1227) is here 6 Alimosho Ojo Alimosho Somolu Apapa

38 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative Chosen 1 Agege Ikeja RS (502) is here x 3 Lagos Island Mushin Lagos Island RS+SI ( = 1227) is here 6 Alimosho Ojo Alimosho Somolu Apapa x

39 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative Chosen 1 Agege Ikeja RS (502) is here x 3 Lagos Island Mushin Lagos Island RS+SI ( = 1227) is here 6 Alimosho Ojo Alimosho RS+2*SI (1952) is here 9 Somolu Apapa x

40 Example Sampling Frame: a Sampling Frame for Metropolitan Lagos, Nigeria Cluster LGA Ward Hhlds Cumulative Chosen 1 Agege Ikeja RS (502) is here x 3 Lagos Island Mushin Lagos Island RS+SI ( = 1227) is here 6 Alimosho Ojo x 8 Alimosho RS+2*SI (1952) is here 9 Somolu x 5000 Apapa

41 Advantages Leads to self-weighting sample because the first stage assigns higher likelihood of selection to larger clusters, thus assuring that women in those clusters have the same ultimate probability of selection as those in small clusters: Comparatively efficient Disadvantages Requires a lot of information Information can get easily out of date, requiring weighting anyway

42 Stratification Grouping the population into distinct categories and generating estimates for each. Example: Might want estimates of CPR by LGA for metropolitan Lagos. Handled easily enough in our sampling frame framework: 1. Pick out the rows of the original sampling frame from each strata so that that strata forms its own independent sampling frame. 2. Then just sample from the independent sampling frame for each strata in the manner that I have described.

43 Metropolitan Lagos

44 Original Sample Frame Cluster Number Local Government Area Ward 1 Agege Ikeja Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Apapa Number of Households

45 Building the Alimosho Sample Frame Cluster Number Local Cluster Government Number Area Ward Local Government Number of Ward Area Households 1 Agege Ikeja Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Alimosho Apapa Number of Households

46 Building the Alimosho Sample Frame Cluster Number Local Cluster Government Number Ward Local Government Number of Ward Number of Area Area Households Households 1 Agege 6 2 Alimosho Ikeja Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Alimosho Apapa 5 257

47 Building the Alimosho Sample Frame Cluster Number Local Cluster Government Number Ward Local Government Number of Ward Number of Area Area Households Households 1 Agege 6 2 Alimosho Ikeja 8 3 Alimosho Lagos Island Mushin Lagos Island Alimosho Ojo Alimosho Somolu Alimosho Apapa 5 257

48 Building the Alimosho Sample Frame Cluster Number Local Cluster Government Number Ward Local Government Number of Ward Number of Area Area Households Households 1 Agege 6 2 Alimosho Ikeja 8 3 Alimosho Lagos Island Mushin Alimosho Lagos Island Alimosho Ojo Alimosho Somolu Alimosho Apapa 5 257

49 Slum and Non-Slum Areas An important line of stratification for our work will be on slum and non-slum lines. Stratifying by slum and non-slum implies that our clusters can be sorted into slum and non-slum clusters Thus, we must divide each city into mutually exclusive, exhaustive PSUs that can be categorized as slum or non-slum. Can be a little tougher.

50 Slum and Non-Slum Areas Typically need to map the locations and borders of the slums and, if you want to pursue sampling of slums with probability proportional to population size, their populations. Also need to put together a non-slum domain. Surprisingly, this can often be the conceptually tougher half of the exercise. Sorting the two out often involves sophisticated GIS analysis.

51

52

53