Survey Research. Classifying surveys on the basis of their scope and their focus gives four categories:

Transcription

1 Survey Research Types of Surveys Surveys are classified according to their focus and scope (census and sample surveys) or according to the time frame for data collection (longitudinal and cross-sectional surveys). A survey that covers the entire population of interest is referred to as a census. In research, however the population is used to refer to the entire group of individuals to whom the findings of a study apply. The researcher defines the specific population of interest. Classifying surveys on the basis of their scope and their focus gives four categories: 1) A census of tangibles: one seeks information about a small population, such as a single school, and when the variables are concrete, there is little challenge in finding the required answers. 2) A census of intangibles: one seeks information about constructs that aren t directly observable but must be inferred from indirect measures. Such constructs may include pupil achievement or aspirations, teacher morale, parents attitudes toward school, or the achievement testing program carried out by most schools. 3) A sample survey of tangibles: one seeks information about large groups. Sampling techniques are used and the information collected from the sample is used to make inferences about the population as a whole. 4) A sample survey of intangibles: one seeks information about constructs that aren t directly observable but must be inferred from responses made by the subjects to questionnaires or interviews. For example, how someone is going to vote is an intangible, but what is marked on a ballot is tangible. Surveys Classified According to the Time Dimension 1) Longitudinal surveys: gather information at different points in time in order to study changes over extended periods of time. 2) Panel study: the same subjects are surveyed at different times over an extended period. Because the same subjects are studied over time, researchers can see the changes in the individuals behavior and investigate the reasons for the changes. 3) Trend study: different people from the same population are surveyed at different times. 4) Cohort study: a specific population whose members changed over the duration of the study is followed over a length of time. 5) Cross-sectional surveys: study a cross section (sample) of a population at a single point in time. The Survey Technique The survey permits one to gather information from a large sample of people relatively quickly and inexpensively. There are five basic steps involved in survey research: Ma660 Survey Research Notes Page 1

2 1) Planning: Survey research begins with a question that the researcher believes can be answered most appropriately by means of the survey method. The researcher needs to decide on the data-gathering technique that will be used. 2) Sampling: The researcher must make decisions about the sampling procedure that will be used and the size of the sample to survey. If one is to generalize the sample findings to the population, it is essential that the sample selected be representative of that population. 3) Constructing the instrument: A major task in survey research is the construction of the instrument that will be used to gather the data from the sample. 4) Conducting the survey: Once the data-gathering instrument is prepared, it must be field-tested to determine if it will be provide the desired data. Also included in this step would be training of the users of the instrument, interviewing subjects or distributing questionnaires to them, and verifying the accuracy of the data gathered. 5) Processing the data: The last step includes coding the data, statistical analysis, interpreting the results, and reporting the findings. Data Gathering Techniques There are two basic ways in which data are gathered in a survey research: interviews and questionnaires. Each of these has two options, thus providing four different approaches to collecting data: 1) Personal interview: the interviewer reads the questions to the respondent in a face-to-face setting and records the answers. Advantages of personal interviews: a. Flexibility: The interviewer has the opportunity to observe the subject and the total situation in which he or she is responding. Questions can be repeated or their meanings explained in case they are not understood by the respondent(s). The interview can also press for additional information when a response seems incomplete or not entirely relevant. b. Greater Response Rate: Response rate refers to the proportion of the selected sample that agrees to be interviewed or returns a completed questionnaire. Personal contact increases the likelihood that the individual will participate and will provide the desired information. Furthermore, the interviewer is able to obtain an answer to all or most of the questions. c. Interviewer Control of Question Order: The interviewer has control over the order with which questions are considered. In some cases, it is very important that respondents not know the nature of later questions because their responses to these questions might influence earlier responses. This problem is eliminated in an interview, where the subject does not know what questions are coming up and cannot go back and change answers previously given. For individuals who cannot read and understand a written questionnaire, interviews provide the only possible information-gathering technique. Disadvantages of personal interviews: a. Cost: Personal interviews are more costly than other survey methods. The selection and training of the interviewers and their travel to the interview site make the procedure costly. It Ma660 Survey Research Notes Page 2

3 takes a great deal of time to contact potential respondents, set up appointments, and actually conduct the interview. b. Possibility of Interviewer Bias: Interviewer bias occurs when the interviewer s own feelings and attitudes or the interview s gender, race, and other characteristics influence the way in which the questions are asked or interpreted. The interviewer may verbally or nonverbally encourage or reward correct responses that fit his or her expectations. c. Social Desirability Bias: Social desirability bias occurs when the respondents want to please the interviewer by giving socially acceptable responses that they would not necessarily give on an anonymous questionnaire. 2) Telephone interview: The telephone interview has become more popular and compares favorably with face-to-face interviewing. Advantages of telephone interviews: a. Lower Cost and Faster Completion with Relatively High Response Rates: Telephone interviews can be conducted over a relatively short time span with persons scattered over a large geographic area. The phone permits the survey to reach people who would not open their doors to an interviewer, but who would not open their doors to an interviewer, but who might be willing to talk on the telephone. b. Respondent Anonymity: Respondents have a greater feeling of anonymity hence there may be less interviewer bias and less social desirability bias than is found with personal interviews. c. Computer Usage: Computers can be used in administering the interview and in coding the responses. Wearing earphones, the interviewer can sit at a computer while it randomly selects a telephone number and dials. When the respondent answers, the interviewer reads the questions that appear on screen and types the answers directly into the computer. This saves the researcher time usually spent in coding, getting the data organized, and entering it into the computer for analysis. Disadvantages of telephone interviews: a. Less Opportunity for Establishing Rapport with the Respondent: It takes a great deal of skill to carry out a telephone interview so that valid results are obtained. It is often difficult to overcome the suspicions of the surprised respondents, especially when personal or sensitive questions are asked. An advance letter that informs the potential respondents of the approaching call is sometimes used to deal with this problem, but the letter can induce another problem. The recipient has time to think about responses or to prepare a refusal to participate when the call comes. b. Survey Exclusion: Households without telephones and those with unlisted numbers are automatically excluded from the survey, which may bias results. There is, however, a technique known as random digit dialing that solves the problem of unlisted numbers (although it does not help to reach the households without a telephone). Ma660 Survey Research Notes Page 3

4 3) Mailed questionnaire: Often much of the same information can be obtained by means of a questionnaire that is mailed to each individual in the sample, with a request that it be completed and returned at a given date. Because the questionnaire is mailed, it is possible to include a larger number of subjects as well as subjects in more diverse locations than is practical with the interview. Advantages of mailed questionnaires: a. Confidentiality or Anonymity: A mailed questionnaire guarantees confidentiality or anonymity, thus perhaps eliciting more truthful responses than would be obtained with a personal interview. b. No Interview Bias: The mailed questionnaire eliminates the problem of interviewer bias. Disadvantages of mailed questionnaires: a. Possibility of Question Misinterpretation: It is difficult to formulate a series of questions whose meanings are crystal-clear to every reader. The investigator may know exactly what is meant by a question, but the respondent may interpret question differently. b. Low Return Rate: It is easy for the individual who receives a questionnaire to lay it aside and simply forget to complete and return it. A low response rate limits the generalizability of the results of the questionnaire study. Factors that have been found to influence the rate of returns for a mailed questionnaire are: 1) the length of the questionnaire 2) the cover letter 3) the sponsorship of the questionnaire 4) the attractiveness of the questionnaire 5) the ease of completing it and mailing it back 6) the interest aroused by the content 7) the use of a monetary incentive 8) the follow-up procedures 4) Directly-administered questionnaire: This questionnaire is administered to a group of people at a certain place for a specific purpose. Examples include surveying the freshmen or their parents attending summer orientation at a university. Advantages of directly-administered questionnaire: a. High Response Rate: The response typically reaches 100 percent. b. Low Cost c. Researcher Availability: Researcher is present to provide assistance or answer questions. Disadvantages of directly-administered questionnaire: a. Researcher is restricted in terms of where and when the questionnaire can be administered. b. Because the sample is usually quite specific, the findings are generalizable only to the population that the sample represents. Ma660 Survey Research Notes Page 4

5 Constructing the Instrument Types of Questions: Because survey data consist of peoples responses to questions, it is very important to start with good questions. Two basic types of questions are used in survey instruments: 1) Closed-ended Questions: One uses closed-ended questions when all of the possible, relevant responses to a question can be specified and the number of possible responses is limited. Advantages: a. Responses are easier to tabulate. b. Responses can be coded directly on scannable sheets that can be read and the data put into a computer for analysis. c. Respondents can be answered more easily and quickly. d. Ensures that all subjects will have the same frame of reference in responding and may also make it easier for subjects to respond to questions dealing with topics of a sensitive or private nature. Disadvantages: a. Take more time to construct. b. Do not provide more insight into whether respondents really have information or any clearly formulated opinions about an issue. c. Easier for the uninformed respondent to choose one of the suggested answers than to admit to lack of knowledge on an issue. d. Respondents who have the knowledge or who have well-informed opinions on the issue may dislike being restricted to simple response categories that do not permit them to qualify their answers. 2) Open-ended questions are used when there are a great number of possible answers or when the researcher is not able to predict all the possible answers. Advantages: a. Permit a free response rather than restricting the respondent to a choice from among stated alternatives. b. Individuals are free to respond from their own frame of reference, thus providing a wide range of responses. c. Easier to construct Disadvantages: a. Tedious analysis and time-consuming as the researcher must read and interpret each response, then develop a coding system that will make possible a quantitative analysis of the responses. b. Some responses may be unclear, and the researcher is unsure how to classify or code the response. Ma660 Survey Research Notes Page 5

6 Writing Survey Questions Before beginning to write a structured set of survey questions, it can be helpful to have focus groups discuss the questions in a nonstructured form. A moderator keeps the discussions focused on a preset agenda and asks questions to clarify comments. Focus group discussions help the researcher understand how people talk about the survey issues, which is helpful in choosing vocabulary and in phrasing questions. Focus groups often can suggest issues, concerns, or points of view about the topic that the researcher has not considered. Characteristics of a Good Questionnaire: 1) It deals with a significant topic, one the respondent will recognize as important enough to warrant spending one s time on. The significance should be clearly and carefully stated on the questionnaire or in the letter that accompanies it. 2) It only seeks information that cannot be obtained from other sources such as school reports or census data. 3) It is as short as possible and only long enough to get the essential data. Long questionnaires frequently find their way into the wastebasket. 4) It is attractive in appearance, neatly arranged, and clearly duplicated or printed. 5) Directions are clear and complete. Important terms are defined. Each question deals with a single idea and is worded as simply and clearly as possible. 6) The questions are objective with no leading suggestions as to the responses desired. 7) Questions are presented in good psychological order, proceeding from general to more specific responses. The order helps respondents to organize their own thinking so that their answers are logical and objective. 8) It is easy to tabulate and interpret. Preparing the Cover Letter Researchers may find it useful to mail an introductory letter to potential respondents in advance of the questionnaire itself. This procedure alerts the subject to the study so that he/she is not overwhelmed by the questionnaire package. In any case, a cover letter addressed to the respondent by name and title must accompany the questionnaire. The cover letter should be as brief as possible. One page is the maximum recommended length. Enclose the letter in an envelope along with the questionnaire. Always include a selfaddressed, stamped return envelope for the respondent s use. This is indispensable for a good return rate. Token monetary incentive increases response rate; however, offering a payment isn t always possible because a token amount can greatly increase the cost of the survey if the sample is large. The cover letter introduces the potential respondents to the questionnaire and sells them on responding. The cover letter should include the following elements: 1) The Purpose of the Study: The first paragraph should explain the purpose of the study and its potential usefulness. It will be helpful to relate the importance of the study to a reference group with which the individuals may identify. 2) A Request for Cooperation: The letter should explain why the potential respondent was included in the sample and should make an appeal for the respondent s cooperation. Respondents should be made to feel that they can make an important contribution to the study. Ma660 Survey Research Notes Page 6

7 3) The Protection Provided the Respondent: The letter must not only assure the respondents that their responses will be confidential but must also explain how that confidentiality will be maintained. Researchers may use an identification coding system. 4) Sponsorship of the Study: The signature on the letter is important in influencing the return of the questionnaire. If the study is part of a doctoral dissertation, it would be helpful if a person well known to the respondents, such as the head of a university department or the dean of the school, signs or countersigns the letter. Such a signature is likely to be more effective than that of an unknown graduate student. 5) Promise of Results: An offer may be made to share the findings of the study with the respondents if they are interested. They should be told how to make the request for the results known to the researcher. 6) Appreciation: An expression of appreciation for their assistance and cooperation with the study should be included. 7) Recent Data on the Letter: The cover letter should be dated near the day of mailing. A potential respondent will not be impressed by a letter dated several weeks before receipt. 8) Request for Immediate Return: Urge immediate return of the questionnaire. A questionnaire that fails to receive attention within a week is not likely ever to be returned. In order to reach the maximum percentage of returns in a mailed questionnaire survey, planned follow-up mailings are essential. First follow-up having both a letter and a replacement questionnaire should occur in about 7-10 days after the initial mailing; second follow-up in about 3 weeks, and third follow-up in about 6 to 7 weeks. It is suggested that the researcher include in the third follow-up a postcard on which subjects could indicate that they do not wish to participate in the survey and will not be returning the questionnaire. Such a procedure permits definite identification of nonrespondents. Ma660 Survey Research Notes Page 7