Jeff Jackson Sarah Likshis Devon Sheppard

Transcription

1 James Hedges James Hedges Jeff Jackson Sarah Likshis Devon Sheppard

2 Introduction Telephone surveying is defined as a systematic collection of data from a sample population using a standardized questionnaire. Today we ll discuss the History Use of RDD to attain a sampling frame Current issues with a focus on cell phones Questionnaire design in regards to script structure

3 The History By 1970 about 90% of all households could be reached by telephone. In 1986 the percentage was over 92. For most applications of sample surveys this an adequate percentage (Groves & Kahn, 1979).

4 Sampling Frame The ideal Sampling Frame was telephone directories. In 1970 the percentage of unlisted phone homes averaged about 20% country wide As high as 40% in certain locations (Groves and Kahn,1979). To overcome the problem of unlisted numbers, a method called Random Digit Dialing was developed (Glaser & Metzger,1972).

5 Random Digit Dialing (RDD) A: Area Code B: Exchange C: Particular phone line in the exchange

6 RDD Not all of these digit combinations are in use. For sampling purposes, only those areas and exchanges that are known to correspond to working area codeexchanges are used. Within each exchange there are 10,000 numbers. Those that correspond to a telephone household are not known.

7 RDD A random digit dialing sample is selected by randomly selecting area code-exchange combinations and then appending four-digit random numbers. This method leads to an equal probability sample of all telephones. However, in the average exchange, fewer than 30% of all possible numbers are working residential numbers so this method is somewhat inefficient. Various methods were later developed that increase the efficiency while retaining the coverage of both listed and unlisted numbers (Frankel and Frankel, 1977).

8 Current RDD Problems For the past several decades, random-digit-dial (RDD) landline telephone sampling has provided a cost-efficient strategy for conducting surveys of the U.S. household population. Since the number of landlines are reducing, the validity of the basic RDD landline sampling model has come into question. Non-coverage rates in U.S. landline-based telephone samples due to cell phone only households and the corresponding potential for bias in estimates from surveys that sample only from landline frames are growing issues.

9 What has changed? Many factors have changed telephone surveys since their extreme popularity in the 1970 s. Recall our previous discussions during the semester about cell phone usage and phone surveys. In what ways do cell phones affect phone surveys today?

10 Cell Phone Issues There is a younger population using phones (including children). These numbers would need to be removed from the study. Cell phones are not included on landline frames People often share cell phones. You may be reaching the spouse family or friend of the intended recipient. 10 to 15% of households take place in sharing cell phones among adults (Link, Michael P. Battaglia, 2007). Also a growing issues with landlines, caller ID on cell phones can be used to screen calls.

11 Cell Phones Issues Researchers must understand that the level of "cognitive engagement" during cell phone surveys is much different than if the person were at home. Not only do researchers need to be concerned about questions of safety (e.g., potentially interviewing someone who is driving), but they also need to be concerned about how respondents answer questions while engaged in other activities, such as shopping, being in a restroom or outdoors, or dining in a restaurant. Ideally, researchers would like respondents to be "fully engaged" mentally in the interview process and not distracted by other activities or stimuli. Researchers must be careful not to include business-only numbers in a sampling frame where businesses are not the population of interest (Link, Michael P. Battaglia,2007).

12 Cell Phones Cell phones are dominating American Culture and in turn play a large role on the way that telephone surveys are being conducted. For the second half of 2006, the percentage of cell phone only households was 12.8 percent (Blumberg and Luke 2007) percent of all adults living with unrelated roommates and one in four adults aged 18 to 24 years live in cell phone only households (Blumberg and Luke 2007).

13 Cell Phones These adults are not covered by current RDD landline sampling procedures, which exclude telephone exchanges and 1,000 banks used exclusively for cell phones, and the percentages are trending upward. These are some of the same demographics groups that are increasingly underrepresented in current RDD landline telephone surveys due to differential nonresponse. Cell phone only respondents were more likely to be younger, unmarried, and Hispanic according to Tucker, Brick and Meekins (2007)

14 Controlling Demographic Bias In terms of the potential for bias in survey estimates, Keeter (2006) provided reason for optimism by demonstrating that bias in 2004 presidential voting preferences due to exclusion of cell phone only households was essentially eliminated when age was incorporated into the weighting methodology.

15 Weighting Methodology Weighting is used to take account for under coverage and demographic problems. i.e. geographic region, age, ethnic background, education, income characteristics, type of dwelling, home ownership, marital status, employment status, and presence of a child in the household. Regardless of how weighting may be approached, it seems reasonable to conclude that many differences between the landline and cell-only populations arise from differences in demographic factors, such as age.

16 Appropriate Weights Establishing appropriate values for weights is difficult since the size and the demographic characteristics of the cell-only population are moving targets. Those 18 and over with cell phones and no landlines increased from 2.8 to 11.8 percent in the United States from the first half of 2003 to the last half of 2006 (Blumberg and Luke 2007), showing an annual rate of increase of about 51 percent. Such a rapid increase in the size of the cell-only group raises the specter of equally dramatic shifts in demographic characteristics over time, making it difficult to maintain appropriate demographic weights.

17 Weighting in the Future It is certainly possible that, as more people from different demographics replace their landlines with cell phones, continued wireless substitution for landlines will lead to greater demographic homogeneity, thereby reducing the need for demographic weighting (Link, Michael P. Battaglia, 2007).

18 Discussion Question Imagine that you are about to conduct a phone survey. What kind of questions would you ask? How would you prepare?

19 Questionnaire Design Many of the elements of questionnaire design are similar to mail and electronic surveys Don t ask leading/double-barreled questions Use closed- or open-ended questions where appropriate Some aspects of questionnaire design are not relevant to phone surveys Visual elements/white space Survey navigation prompted by interviewer

20 Questionnaire Design Some elements of questionnaire design are specific to phone surveys Survey-taker is not looking at the survey questions Not every survey-taker necessarily has the same interviewer A researcher must consider these aspects of phone surveys while constructing the questionnaire

21 Phone Survey Scripts Using a script during a phone survey helps maintain consistent conditions during administration. These scripts are structured to allow for efficient data collection. On the phone, time is of the essence (more so than with mail or electronic surveys).

22 Introductory Scripts According to the Survey Research Center (SRC) at the University of Waterloo, an introductory script is needed in every phone survey. The intro needs to be clear and concise, and needs to: Identify the owner (or organization) State the purpose of the survey Give an estimated length Assure the survey taker of confidentiality Inform the survey taker that responding is voluntary

23 Identifying the Owner In general, people are reluctant to participate in phone surveys because of calls by telemarketers and technology features like caller ID. This means it is important to show that your study is done by a reputable organization, and that you are not interested in selling anything.

24 Stating the Purpose Respondents should be told early in the conversation what the purpose of the study is. Inform them that the results of the study could be beneficial to the individual responding. Also, tell how the data will be used.

25 Survey Length According to the SRC, someone is more likely to agree to participate in a survey if it is relatively short. Be clear and honest about how long the survey should take to complete in order to avoid a hang-up mid survey.

26 Ensure Confidentiality State that answers will be kept anonymous so that responses will be honest. Volunteer Participation Do not coerce individuals into taking surveys. Inform them that participation is voluntary so that they feel comfortable about answering.

27 Introductory Scripts Two Types: Recruited Ask for the respondent by name Random Be very clear about the background of the study Get the respondent s attention

28 Script Example

29 Script Example

30 Structure of Phone Survey Scripts Closed-ended questions are usually most appropriate. Close-ended questions: Have quick answers Allow for faster recording of answers Help structure how remaining items flow Often, future questions depend on answers to earlier questions.

31

32

33

34

35

36 In Conclusion Telephone surveys are an ideal way to maximize response rates, as well as to maintain control over the quality of the data. The population is currently continuing its trend of switching to cell-only households, which will play a significant role in future telephone surveying. As with all other public opinion polling, telephone surveys require a great deal of insight as to the basic script to ensure meaningful results.

37 Works Cited Groves, R. M., and R. L. Kahn (1979) Surveys by Telephone. New York: Academic Press. Glasser, G. J., and G. D. Metzger (1972) "Random-digit dialing as a method of telephone sampling." Journal of Marketing Research Frankel, M. R., and L. R. Frankel (1977) "Some recent developments in sample survey design." Journal of Marketing Research 14: Link, Michael P. Battaglia (2007) Reaching the U.S. Cell Phone Population Oxford Journals Survey Research Center (2005) The Importance of the Introductory Script in Telephone Interviews University of Waterloo. Blumberg and Luke (2007) Coverage Bias in Traditional Telephone Surveys of Low- Income and Young Adults Oxford University Press Tucker, J.Michael Brick, and Brian Meekins Household Telephone Service and Usage Patterns in the Unites States in 2004: Implications for Telephone Samples, Public Opinion Quarterly, Keeter, Kennedy, Clark, Thompson and Mokrzycki(2006) What s Missing from National RDD Surveys? The Impact of the Growing Cell-Only Population American Association for Public Opinion Research