W I S C O N S I N T O B A C C O C O N T R O L B O A R D. Guide to conducting restaurant and worksite tobacco policy surveys.

Transcription

1 W I S C O N S I N T O B A C C O C O N T R O L B O A R D Monitoring and Evaluation Program Technical Assistance and Training for Tobacco-free Coalitions Guide to conducting restaurant and worksite tobacco policy surveys June 2001 Designed for use by Wisconsin s local tobacco-free coalitions by: University of Wisconsin Madison Comprehensive Cancer Center University of Wisconsin Cooperative Extension University of Wisconsin Madison Center for Health Policy and Program Evaluation

2 User s guide to conducting restaurant and worksite tobacco policy surveys Table of contents Page 1 Introduction and purpose of this guide Page 2 Ten steps to conducting a telephone survey Page 3 The questionnaires: What they are and how to use them Page 4 Conducting the survey: Step-by-step instructions Appendices 1 Key terms and phrases 2 Troubleshooting telephone interviews 3 The basics of proper interviewing 4 Restaurant and worksite telephone interview call logs 5 Wisconsin county codes 6 Industry codes 7 Where to get help 8 Restaurant smoking policy questionnaire 9 Worksite smoking policy questionnaire

3 Introduction Restaurant and worksite smoking policies affect everyone in Wisconsin. As local tobacco-free coalitions implement tobacco control programs, restaurants and worksite smoking policies are a principal target for change. As coalitions implement strategies to encourage smoke-free restaurants and worksites, gathering information about current policies will allow coalitions to track changes over time, target businesses with the potential for change, and contribute data to the statewide monitoring and evaluation effort to better understand effective strategies. As you set out to conduct your surveys, the following questions will help guide choices about how you proceed: Why you are conducting the survey? Who will use the data? How they they use the data? What resources, including time, people, skills and funds, do you have available to conduct the survey, enter data, and analyze results? The Wisconsin Tobacco Control Board Monitoring and Evaluation Program The University of Wisconsin Monitoring and Evaluation Project monitors trends in tobacco use, evaluates statewide efforts in tobacco control, and provides technical assistance and training in program evaluation for local health coalitions. The partnership includes the UW Comprehensive Cancer Center, the Center for Health Policy and Program Evaluation, and the University of Wisconsin Cooperative Extension. Contact information for Regional Evaluation Specialists appears on page 19. Purpose of this guide This guide provides standard questionnaires for conducting telephone surveys with worksites and restaurants, as well as step-by-step instructions and tools to guide the survey process. The surveys will allow coalitions to gather information about current practices and monitor smoking policy changes over time. These questionnaires were designed to serve as a resource for tobacco control efforts across the state. Coalitions are not required to use these questionnaires. However, coalitions can contribute community-level data to a statewide database on restaurant and worksite smoking policies that will allow us to compare changes over time across geographic regions. This effort will help us understand more about local approaches that work, those that don t, and why. A questionnaire is a standardized group of questions, and a survey is the process of using a standardized instrument to collect data. This guide offers information about how to use the questionnaires as a means to conducting surveys of worksites and restaurants. Tobacco Control Monitoring and Evaluation Program 1

4 Ten steps to conducting a telephone survey These questionnaires will allow you to collect information about worksite and restaurant smoking policies. However, administering the questionnaire is just one part of the survey process. To ensure your data are useful and credible, be prepared to implement the steps listed below. More detail about each of these steps appears in this guide. 1. Clarify the reasons you are conducting the survey 2. Clearly identify the resources (time, people, funds, and skills) you have available. 3. Identify the region of your survey. 4. Identify your population. Obtain a list of restaurants or worksites in this region. 5. Select and train interviewers. 6. Prepare to conduct telephone interviews. 7. Conduct telephone interviews. Make sure that this telephone survey is the most appropriate method to achieve your goals. These questionnaires were designed to collect standardized information to determine tobacco use policies at restaurants and worksites at the point in time when you conduct the survey. Given your resources, surveying all restaurants or worksites in your region may not be realistic. If not, determine an adequate sample size based on the purpose of your survey. Depending on your goals, you may wish to target businesses in a city, a group of towns, or a county. Be explicit about your choices. Specify the population of potential worksites and restaurants. Remove from the list those businesses that do not meet the criteria of a worksite or a restaurant, defined on page 4 of this guide. Your main goals in this step are to ensure interviewers: 1) become familiar with the questionnaire 2) ask questions as stated on the questionnaire 3) know how to handle the interview process 4) understand how to record responses 5) know how to enter data consistently Several tools provided in this guide will help you track and record the interviews you conduct. Make an adequate number of copies of the questionnaire. Assign a unique identification number to each worksite or restaurant. Track each call using a telephone interview log found in Appendix 5. Preparing your tools in advance will allow you to avoid unnecessary delays in your survey. Keep in mind that telephone interviews take time to complete. In pilot tests, each interview took between 5 and 12 minutes to complete. 8. Enter data. Enter data into a spreadsheet or database. 9. Analyze and interpret data. Using descriptive analyses will be helpful (see page 10). 10. Share your results and document your methods. Reporting your results is a critical part of the process. Remember to document your survey process. This will help others understand how you gathered the information and judge the strengths and weaknesses of the data. 2 Tobacco Control Monitoring and Evaluation Program

5 The questionnaires: What they are and how to use them This guide contains two survey questionnaires from the Wisconsin Tobacco Control Board s Monitoring and Evaluation Program (MEP). These questionnaires were designed for use by telephone interviewers. They can be easily adapted for use as questionnaires for face-to-face interviews but should not be used as mail surveys in their current form. Coalitions may add additional questions to the questionnaire as long as all MEP survey items remain unchanged. Keep in mind that adding questions will lengthen the time required for interviews as well as for data entry. We recommend that interviewers follow the order of the survey questions as presented on the questionnaire and recite them as worded, being careful to accurately record each response. The Questionnaires The questionnaires appear in the appendices. They contain core items to measure the smoking policies of worksites and restaurants in your county. Data from this questionnaire will be most useful at the statewide level if you do not change the questions in this core questionnaire. Consider adding questions to this core only if they are important to your program. Following this procedure ensures uniformity of responses. On page 13, we have included a list of definitions to clarify this process. Please ensure that interviewers are familiar with these definitions before proceeding with the survey. These data will prove valuable in our efforts to understand trends in worksite and restaurant smoking policies across Wisconsin. These data will not be used to evaluate your coalition. Rather, they increase our understanding of whether policies are changing statewide. Later this summer, we will provide instructions about how you can contribute your data to the statewide database. Tobacco Control Monitoring and Evaluation Program 3

6 Conducting the surveys: Step-by-step instructions Whom you choose to survey depends on 1) the purpose of your survey, 2) how you plan to use the data, 3) the resources you have available to conduct the survey, and 4) the resources you have available to enter and analyze the data. The following steps should clarify in greater detail how to conduct a telephone survey using the worksite or restaurant questionnaires we have provided. What is a worksite? A worksite is any private place of business operating within your county. For purposes of this survey, do not include restaurants, taverns, municipal or county buildings. A separate questionnaire for public buildings will follow. 1. Clarify the reasons you are conducting the survey. 2. Clearly identify the resources (time, people, funds, and skills) you have available. We recommend that each coalition designate an individual who is responsible for overseeing the survey process. Volunteers might include university or college students looking for service learning opportunities or members of a community organization with whom you might share the results of your data. 3. Identify the region of your survey. Will you survey worksites or restaurants in a city, group of towns, county, or region? 4. A. Identify your population. Worksites: A comprehensive business list is usually available either from the local Chamber of Commerce or from the County Development Office. Local sanitarians may also be a Q. What if a business has more than one branch within the county? A. Count each branch as a separate worksite. What is a restaurant? A restaurant is any business that serves food, has at least one sit-down eating area, and has alcohol sales that constitute less than 50% of total sales. Q. What if the respondent says that alcohol sales fluctuate over the course of the year, above 50% and below 50%? A. Complete the survey. It is best to go ahead with the survey if the person is not definite about where they fall. source of current lists of restaurants or worksites. Try to obtain an electronic file or database so that you can make the process more efficient. You can use this file to manage your list, print labels for each questionnaire, and track changes in telephone numbers or contact information. 4 Tobacco Control Monitoring and Evaluation Program

7 Restaurants: Obtain a list of all businesses with food licenses. To ensure accuracy, the list should be recent preferably updated within the past year. This list will comprise not only restaurants but also businesses such as taverns and gas stations with food booths. Again, an electronic file or database will allow you to manage information about these establishments. Over time, this might include information about your contacts with local restaurants or your own rating scale of how ready establishments are to change their smoking policies. B. Remove from the list businesses that do not meet the worksite criteria or establishments that do not meet the definition of a restaurant. Worksite survey: Do not include government and other public offices. There will be a separate questionnaire developed for those worksites. Before you conduct the survey, delete all restaurants from the list. A restaurant is considered a place of business that serves food, has at least one sit-down area, and whose alcohol sales if any constitute less than fifty percent of total sales. If you are in doubt that the business meets this definition of worksite, keep it on the list. Non-worksites will be screened out during the survey process. Restaurant survey: Remove from the list businesses that are not restaurants. A restaurant is considered any business that serves food, has at least one sit-down area, and whose alcohol sales, if any, constitute less than fifty percent of total sales. If you are in doubt that the business is a restaurant, keep it on the list. Non-restaurants will be screened out during the survey process. C. Looking at the list, decide whether you have the time, money, staff and volunteers to call all the worksites or just a sample of them A sample is a portion or a subgoup of a larger group called a population. Decisions about sampling depend on your population size, what you want to know, and the resources you have available. Sampling Two standard ways to draw a sample are probability sampling and nonprobability sampling. A probability sample is a sample whose members you choose randomly. This means that each member of the population has an equal chance of being selected. This increases the likelihood that your information will represent the characteristics of the entire group. If you want to generalize your findings to the entire population, or to be able to say that the For more specific information on selecting a sample, see Program Development and Evaluation: Sampling by Ellen Taylor-Powell at the University of Wisconsin Cooperative Extension publications web site: and choose Program Development and Evaluation from the menu at left. Tobacco Control Monitoring and Evaluation Program 5

8 sample statistically represents the larger population, you should use a probability sample. If your population is small, you may conduct a sample of the entire population. If you want to learn about individuals or particular cases within a population, nonprobability sampling is appropriate. For example, if you had a list of all worksites in your county and randomly selected every fifth worksite to include in your sample, you would have a simple random sample, and you could say that your sample fairly represented the population of worksites. In this example, it would be important to judge whether an alphabetical list might bias your sample by excluding particular establishments or worksites with names that reflect that industry. If possible, you may wish to stratify the worksite list by industry and perform a stratified sample to ensure each type of industry in your region is represented. In tests of the worksite survey, each interview required between 5 and 12 minutes. You can estimate the amount of interview time you will need for your target sample by multiplying the target number of completed interviews roughly by 15 minutes. This will give you a conservative estimate. D. How many surveys should you complete? Response rate refers to the proportion of interviews completed to the total sample you set out to interview. Again, your desired response rate will depend on the purpose of your survey and the resources available to conduct the survey. If you collect too much data and cannot enter and analyze it, it will not be worth much to you. However, if you collect too little data, you may not have enough information to generalize about worksite and restaurant policies in your region. Moreover, some counties may have very few restaurants, while others may have more than a thousand. In the latter case, you need to assess how you will use the data and whether you have the resources to survey all restaurants. For more information about choosing a sample that will allow you to calculate more precise statistics, refer to the UW Extension document mentioned in the above inset. 6 Tobacco Control Monitoring and Evaluation Program

9 5. Select and train interviewers. Interviewers should read Telephone Interviewer Instructions and The Basics of Proper Interviewing beginning on page 18. These will provide some useful general guidelines for conducting a telephone interview. Prior to initiating the calls, all interviewers should understand questions on the questionnaire and their interviewing role. Interviewers should read the questions out loud to ensure they understand the questions and response items. Ideally, interviewers will make calls in a quiet place, away from distractions. 6. Prepare to conduct telephone interviews. Using the list of worksites, create a unique identifier for each worksite you attempt to contact. We suggest using a series of sequential numbers 0001, 0002, so that you will be able to tell worksites from one another. For example, if you talk with two McDonald s franchise owners and their names are both Jim Smith, you will easily know from the unique identifier that they are two separate franchises. Also, if you want to share data with another group but you don t want to share identifying information, such as names, cities, and telephone numbers, you can easily remove those items from the database and leave the unique identifier to help you tell the respondents apart. After you have determined the number of worksites or restaurants you will survey, make an adequate number of copies of the questionnaire. Interviewers will document responses from each interview on the questionnaire form, and having multiple copies before starting will make the process run more smoothly. Each interviewer should have on hand a blank pad of paper to record notes about particular interviews. Decide ahead of time whether interviewers should or should not capture respondent comments that fall outside the standard questions. If you think this information is important (and it probably is!), you may want to keep a separate log of open-ended comments. Be sure to code those comments using the unique identifier on the survey questionnaire so that you can match the comment with the respondent. This will prove critical in later analysis. Tobacco Control Monitoring and Evaluation Program 7

10 7. Conduct telephone interviews. Worksites: Call worksites during times convenient for people who work in personnel offices daytime calls to businesses open during the day and evening calls to those open at night. Interviewers should ask to speak to the human resources manager, safety officer, or someone knowledgeable about smoking policies at each worksite. Depending on the size or management structure of the worksite, this may be the owner or manager. Restaurants: Call restaurants during times when restaurant owners or managers will most likely be available. Interviewers should ask to speak to the restaurant manager or assistant manager to complete the survey. If the appropriate person is unavailable, the interviewer should ask when that person will be available, record the alternate time on the call log, and offer to call back then. Keeping a call log is especially important when interviewers conduct only a few interviews at a sitting or when the interviewing task is shared among several people. We suggest that you divide respondents between interviewers and that interviewers keep separate call logs. To increase the response rate, attempt multiple calls when you do not get a complete response to survey items on the first call. Call back until the appropriate person is reached and the questionnaire is completed. Depending on your time and resources, determine a reasonable number of call-backs (we recommend five). Record data immediately as you move through the interview. If a respondent offers multiple answers, ask him or her to identify the choice that best fits the question. 8 Tobacco Control Monitoring and Evaluation Program

11 8. Record and enter data. As interviewers collect data from respondents, information will be directly marked on the paper copy of the survey instrument. It is probably easiest to circle the respondent s answer, being careful not to make the circle so big that the person entering the data cannot decipher the correct response. Once data collection is finished, enter these data into an electronic spreadsheet or database for analysis. You can use any type of spreadsheet or database program. We suggest using Microsoft Excel or Microsoft Access because they are commonly used and they can be used to generate descriptive statistics, explained below. If you are familiar with SPSS or SAS, you may prefer to calculate descriptive statistics using these programs. The first field in every spreadsheet should be a unique identifier usually a number assigned to each completed survey. You may want to put this identifier on the corresponding paper survey to make it easy to go back to the paper surveys to double check data entry. The individual responsible for entering the data should familiarize him or herself with the questionnaire. Code variable names Q1 for Question 1, Q2 for Question 2, etc. Use numeric codes for all of the responses because it is easier to type numbers in a spreadsheet rather than typing the words Yes or No for each question. For example, if the response to a question is Yes, the person entering the data will look at the questionnaire and see a 1 next to that option. A 1 will be entered in the spreadsheet as a Yes response. The numeric code for Don t Know is 8 and Refused is coded as 9. For questions that ask respondents to Please Describe, enter text information. For questions that ask for a name or other open-ended response, this information can be typed directly under the appropriate variable. For more specific information on entering your data using the provided Excel spreadsheet, please see the Analysis Tools section of this booklet. Once you have entered your data into the spreadsheet you will want to clean the data to make sure that everything has been entered correctly. Cleaning the data involves looking over each value to make sure that it is one of the possible responses for that particular question. For example, if the question asks Yes (1) or No (2) and your spreadsheet has a 3 marked, you will need to go back to the paper copy of the survey and change the entry to the correct response. You should also make sure that all of the responses that are numeric are indeed numeric and those that are text have text rather than numeric values. Finally, you should make sure that there are no missing values. If there are missing values, you will need to check the paper copy of the survey to determine if there was a data entry error or if the respondent did not answer the question. In cases where a respondent did not answer the question, those questions should be coded as 9. Tobacco Control Monitoring and Evaluation Program 9

12 9. Analyze and interpret data. * You will use descriptive statistics to analyze most of your data. Descriptive statistics are used to describe your data and allow for comparisons to other survey data. The following section provides some general information on descriptive statistics. The statistics you use will depend on the questions you would like your data to answer. Data can be categorical in the case of questions that ask for Yes / No responses, or they can be numeric in the case of How many seats does your establishment have? Counts and percentages are useful for a simple analysis of categorical data. Measures of central tendency (mean, median and mode) and measures of variability (range, mean deviation, variance, and standard deviation) are useful for numeric data. We explain these in more detail below. Numerical Counts or Frequencies Counts or frequencies tell us how many times something occurred or how many responses were recorded for each question. For example, you might indicate that 27 of the 50 restaurants surveyed allow smoking at the bar. In some cases, numerical counts are enough. In other cases, they serve as the basis for additional calculations. Or, your strategy might be to target restaurants that serve a lot of people and thus expose a lot of people to tobacco smoke. To target the appropriate restaurants, you might be interested in generating a statistic that tells you the number of restaurants in your sample that have more than 30 seats in a smoking section. This would represent a "count," or frequency. If this count tells you that 5 restaurants in your sample have more than 30 seats in a smoking section, you may choose to target those five. However, if the number tells you that there are no restaurants with more than 30 seats in a smoking section, you may choose to generate a statistic that gives you the number of restaurants who have smoking sections in a range of sizes (e.g., 0-5 seats, 6-10 seats and so on). You would then look at what those respondents mean when they say they have a "smoking section." Percentages A commonly used statistic the percentage expresses information as a proportion of a whole. For example, to obtain the percent of restaurants that allow smoking at the bar we would use the equation 27 / 50 = 54%. Thus, 54% of restaurants allow smoking at the bar. Percentages are a good way to show relationships and comparisons. For example, if we compare the hypothetical example that 54% of restaurants allow smoking at the bar in Dane County compared to 70% of restaurants statewide, this provides a quick assessment of Dane County s restaurant smoking policies compared to the rest of the state. When reporting a *Adapted from Ellen Taylor-Powell, Program Development and Evaluation: Analyzing Quantitative Data, University of Wisconisn Extension Cooperative Extension, Tobacco Control Monitoring and Evaluation Program

13 percentage, common practice is to indicate the number of cases from which the percentage is calculated. N indicates the total number of cases used while n indicates a sub-sample or sub-grouping of the total number. It is important to use the correct base or denominator when calculating a percentage. The base or denominator is the number used to calculate the percent. In most cases it is the N or n. An incorrect base will result in erroneous conclusions. Does 75% mean all worksites, only those that answered the question, or those for whom the question applied? For example, say 10 people did not respond to a certain question, 70 answered Yes and 20 answered No. If we use 100 as the base, we show that 70% answered Yes, but if we had used 90 as the base, the result would be 78% of those who responded said Yes. We can either say that 10% did not answer or omit the 10 no answers from the base. One would follow a similar situation if a particular question were not applicable to a respondent. In other words, you could report the percentage of respondents for whom the question is not applicable, or you could omit them from the base and calculate the percentage using the total number of applicable respondents who answered the question. Displaying the data for each question by count and percent is useful as it shows the distribution of responses for each item. Distributions are usually displayed in a table. The following table provides an example. Measures of Central Tendency Table 1. Frequency distribution of the restaurants in [City X] that allow smoking (n=500)* Frequency Percentage Anywhere in the restaurant % Only in a smoking section % Smoking is not allowed % Total % *Hypothetical data Measures of central tendency are used to characterize what is typical for a group. These are measures that allow one to visualize or identify the central characteristic of the measure. The most frequently used measures of central tendency are the mean, mode and median. Tobacco Control Monitoring and Evaluation Program 11

14 Mean Mode The mean, or average, is commonly used for reporting data, especially numeric data or measures that involve Likert-type rating scales. With narrative scales, we can assign a number value to each category and thus calculate the mean. For example, not important, slightly important, fairly important, and very important can be assigned 1, 2, 3, and 4. An average of the responses for this type of question provides information on how a majority of respondents feel about this particular issue. For questions in which the information collected is already numeric (e.g. number of seats in the smoking section of a restaurant) the mean is also useful. In this example the mean is obtained by totaling up the seats in the smoking section of all restaurants in a given county and dividing that number by the total number restaurants. If there are 1000 seats in the smoking sections of all restaurant and there are 100 restaurants, the calculated average would be 1000/100=10. Thus, the average number of smoking section seats for restaurants in this county is 10. It is important to keep in mind that the mean is affected by extreme values. Therefore, in our example if you have one or two restaurants with very large smoking sections, this will skew the mean to be artificially high. The mode is the most commonly occurring answer or value for a given question. For example, if most respondents report that they have 50 seats in their restaurants, then 50 is the modal number of seats for restaurants in that county. Median The median is the middle value. It is the midpoint at which half of the cases fall below and half fall above the value. Sometimes we may want to know the midpoint value in our findings, or we may want to divide a group of participants into upper and lower groupings. To calculate the median, arrange the data from one extreme to the other. In the case of the restaurant survey you could arrange the surveys by number of seats in restaurant smoking sections from the smallest number of seats to the largest number of seats. Proceed to count halfway through the list of numbers to find the median value. When two numbers tie for the halfway point, take the two middle numbers, add them and divide by 2 to get the median. Like the mode, one advantage to using median is that it is not affected by extreme values. Measures of Variability Measures of variability express the spread or variability in responses to a particular question. As stated earlier, the mean may mask important differences or be skewed by extreme values. Looking at variability often helps further clarify survey results. We can tell whether responses to a particular question are at all similar or whether there is a lot of difference between responses. 12 Tobacco Control Monitoring and Evaluation Program

15 Range The range compares the highest value and the lowest value to indicate the spread of responses. For example, the number of seats in the smoking sections of restaurants could range from 0 to 500 seats. The range can also be expressed as the difference between the highest and lowest points. In this example the range is 500 seats. Standard Deviation The standard deviation measures the degree to which individual values vary from the mean. It is the average distance the average score lies from the mean. A high standard deviation means that the responses vary greatly from the mean. A low standard deviation indicates that the responses are similar to the mean. When all answers are identical, the standard deviation is zero. SD = ( x x) n 1 2 Tobacco Control Monitoring and Evaluation Program 13

16 10. Share your results and document your methods. Reporting your results to coalition members and to the community can draw attention to critical areas for intervention. An example illustrates elements to include: Context and purpose for the survey: Why did you conduct the survey? Purpose of survey: Collect information about current worksite smoking policies in Doe County. How data will be used: Determine current worksite smoking policies and practices in Doe County. Focus outreach efforts on worksites where smoking policies are limited or nonexistent. Follow up survey in 2003 to document changes in policy. Methods: What methods did you use? Be as specific as possible: Date of survey: July September 2001 Instrument and method: Telephone interview using standard questionnaire developed by the Wisconsin Tobacco Control Board Monitoring and Evaluation Program. Source of data on population: Wisconsin Department of Development database of businesses in Doe County, December Respondents: Human resources managers or other worksite personnel knowledgeable of or responsible for establishing smoking policies. Survey region: Doe County. Sampling frame: 400 Doe County worksites, including private places of business and excluding restaurants, taverns, municipal offices, and county offices, in Doe County. Sampling method, if any: Random sample of 150 worksites. Businesses were assigned unique identification numbers. These numbers were placed on 400 pieces of paper, and 150 were drawn out of a hat. Results: What was your response rate? What were the most important results of your survey? Here you can talk about how different groups of respondents answered the questions. You should leave your interpretations and conclusions for the last section. Percentages, frequencies and cross-tabulations will help you describe your data. Conclusions and recommendations: What did you learn from conducting the survey? What recommendations do the data allow you to make? 14 Tobacco Control Monitoring and Evaluation Program

17 Appendix 1: Key terms and phrases Comprehensive List: a list that includes every single restaurant or worksite within a given county Confidentiality: maintaining the security of information gained during the interview process; not revealing or forwarding information to unauthorized sources. Current: updated within the past twelve months Employee: anyone who receives reimbursement for work performed at a given restaurant or worksite (include volunteers registered with the agency/establishment) Interviewee: the appropriate person at the restaurant (manager or assistant manager) or the worksite (human resources staff or Safety Officer) to interview to complete the questionnaire. Interviewer: the person designated by each coalition to make telephone calls to restaurants or worksites for the purpose of completing survey questionnaires. Instrument: a standardized tool for collecting data. This could be a survey questionnaire, a focus group protocol, or a group interview protocol. MEP: Monitoring and Evaluation Program, University of Wisconsin-Madison Questionnaire: Standard list of questions used to generate qualitative or quantitative information. Restaurant: an establishment that serves food, has at least one sit-down area to eat, and whose alcohol sales if any comprise less than fifty percent of total sales. Do not include movie theaters because smoking is never allowed in such establishments. Do include places such as gas stations that serve food and have a food booth. Role-Play: practice by means of a physical enactment the scenario of making telephone calls to conduct the Restaurant or Worksite survey. The trainer (coalition member or designee) will engage in role-play with the interviewer-in-training. The trainer will act as an interviewee while the interviewer asks survey questions as if it were a real interview. Role-plays will be repeated until the interviewer handles the situation accurately and comfortably. Sample: A portion or a subgroup of a larger group, or population. Smoking: includes smoking any tobacco product Smoking Policy: any rule, guideline or policy at a restaurant or worksite that addresses either verbally, in writing, or by some other method expectations and prohibitions regarding the practice of smoking any tobacco product. Worksite: a private place of business, as opposed to a government office or public office. Include all those that have at least five employees. Tobacco Control Monitoring and Evaluation Program 15

18 Appendix 2: Troubleshooting telephone interviews During the course of conducting the Restaurant or Worksite Survey any number of unanticipated events may occur. It will be helpful to examine some of those possibilities ahead of time. A few such situations are outlined below. If you do not find your particular situation listed here you may contact the Regional Evaluation Specialist for guidance. PROBLEM Reach an incorrect telephone number Telephone number has been disconnected or changed to a non-published number Appropriate person to interview is unavailable for an extended period of time There is static on the telephone line and you cannot hear the person at the other end The interviewee keeps straying from the topic of the survey Interviewee is unintelligible (mumbles, difficult to understand, a bad connection, etc.) Interviewee becomes rude or verbally aggressive Interviewee wants assistance with smoking policies AND wants to know WHEN they will be hearing back from the coalition SOLUTION Confirm the number you reached as the one you intended to dial (the number on your list) Double-check the number from your list If the same number as on your list, call Information for the correct number Call correct number Double-check the number from your list and call again If you confirm that you have reached a disconnected number, call Information for assistance Ask to speak to the staff member who is assuming the responsibilities of the person you called during his/her absence Try to explain to the person (even though you cannot hear him/her) that you are experiencing static and will call back at a later time, apologize for the inconvenience Gently remind him/her that when the survey is completed there will be an opportunity for questions and comments; gently redirect him/her back to the current question If the problem continues, gently explain that in order for you to complete the survey you have limited time for telephone calls to each business. Let the person know that s/he can contact a designated coalition member for additional information, but for the time being, the questions need to be completed Explain that you are having difficulty recording the person s answers and would like to call back at another time (ask when it would be convenient) In the meantime, make arrangements for another interviewer to call the person back Remind interviewee of purpose of the questionnaire and ask if s/he is willing to continue answering the questions If not, ask if there is another knowledgeable member of the staff with whom you could speak Take down the name, title and number of the contact person at the agency Explain that someone from the coalition is better able to answer this question and that you will pass it on. 16 Tobacco Control Monitoring and Evaluation Program

19 Appendix 3: Telephone Interviewer Instructions * Before you start. Have these instructions in front of you. You will also need a pen and some blank paper. Read over the instructions and the answers to respondents' questions to familiarize yourself with the procedures and with what to say in response to questions. Using the telephone. Familiarize yourself with the telephone. Make sure you know how to dial an outside line and to dial long distance, if necessary. Keeping a log of all calls. Have a clock or watch on hand to record the date and time of day you make calls. Use a telepohone interview call log for both the worksite and restaurant questionnaires. Fill in your initials and the date and time of the call. (Copies of the telephone call logs appear in Appendix 1.) Track the times you tried to call back, recording more convenient times to call back again. Also indicate to whom you talked (if anyone) and whether you completed the survey. If the respondent becomes incensed, or uses abusive language, be nice! Do not hang up! Keep cool! (Such negative responses are not likely to happen.) If they do, be patient-maybe the person had a bad day. Some responses that might help: Yes, I see. Uh-huh. Yes, I understand that you feel quite strongly about this matter. But we really do need the information. If all else fails, call for a supervisor or wait for the opportunity to say something to this effect: I think I can understand your feelings, and your not wanting to complete the interview. But, thank you very much anyway. Goodbye. When you hang up: Record the time and enter it for this interview. * Salant P and D Dillman. How to conduct your own survey. Toronto: Wiley & Sons, Inc., 1994, p Tobacco Control Monitoring and Evaluation Program 17

20 Appendix 5: The Basics of Proper Interviewing * Main Points 1. Read each question exactly as it is written and in the order in which it appears in the questionnaire. 2. Read slowly. 3. Use standard feedback phrases for acceptable responses. Examples are "Thank you. That's important information" and "I see." 4. Use standard cues or probes to help the respondent give more complete answers to the question. 5. Do not take anything home with you. All questionnaires, codebooks, etc., must remain in the office. 6. After you have left: We have an obligation to respondents to keep their interviews confidential. We feel very strongly that this obligation should be honored. Therefore, please do not tell anyone the substance of any interview or part of an interview, no matter how fascinating or interesting it was. Also, please avoid giving your own summary of findings. Just because 90 percent of your respondents feel a certain way does not mean that 90 percent of everyone else feels the same way. Confidentiality is essential. If you want a copy of results from this survey, let the supervisor know, and we will be sure you get them just as soon as they are available. Two Types of Questions 1. Closed-ended or precoded questions with predetermined response categories from which the respondent must choose. 2. Open-ended questions that ask the respondent to express reasons and feelings in his or her own words. Question Wording 1. Surveys work only if everyone gets the same stimulus. Therefore, read questions exactly as they are worded in the questionnaire. Read questions with no additions, deletions, or substitutions. 2. Often you will find questions that contain parentheses. An example is "How well do you think your (husband/wife) understands you?" Make a choice from parentheses based on what you have learned about the respondent from the household listing and other questions. 3. Read the entire question before accepting the respondent's answer. Question Order 1. Ask each respondent every appropriate question. 2. Don't skip a question because the answer was given earlier or because you "know" the answer. 3. In those situations in which the respondent has already provided information that probably answers the next question, you may preface the question with some combination of the following phrases: "I know we've talked about this," or "I know you just mentioned this, but I need to ask each question as it appears in the questionnaire." "You have already couched on this, but let me ask you... " "You've told me something about this, and. this next question asks... " * Adapted from interviewer training materials used at the Social and Economic Sciences Research Center, Washington State University. 18 Tobacco Control Monitoring and Evaluation Program

21 4. Avoid direct reference to past responses. Style As an interviewer, you must not "direct" the respondent toward an answer. Do not assume that an "answer" you received in passing is the correct answer to a specific question at a particular point in the interview. Do not direct the respondent by mentioning an earlier answer. If an answer is different from the one you expect, do not remind the respondent of an earlier remark or try to force consistency. The quality of your delivery your style also affects the quality of the information you collect. Emphasize underlined words to enhance meaning and to be consistent with the other interviewers. Keep your tone neutral, and avoid voice inflections that might bias results. Use a pleasant tone of voice that conveys assurance, interest, and a professional manner. 3. Read at a slow pace. Remember that although you may have read these questions many times, -the respondent is hearing them for the first time. He or she needs time to understand the questions and to decide on the answers. Question Clarification Sometimes respondents ask for additional information. When this happens, your response is called a CLARIFICATION. There are three kinds of clarifications: 1. Accurate repetition of the entire question, or part of the question. 2. Use of clarifications or definitions that are specified on the definitions sheet. 3. Use of the phrase, "Whatever [Insert the word or phrase about which the respondent asked.] means to you." Points to Remember 1. If you have any doubt that the respondent has heard the entire question, repeat all of it. 2. Upon request, repeat or clarify the item to which the respondent has referred. 3. When asked to repeat only one of several response options, repeat all the options given in the question. 4. Only give definitions specifically allowed in your interviewer's manual. Examples of Standard Probes and Clarifications 1. Whatever it means to you. 2. Whatever you think of as 3. What do you mean? 4. How do you mean? 5. Would you tell me more about your thinking on that? 6. Would you tell me what you have in mind? 7. What do you think? 8. What do you expect? 9. Which would be closer to the way you feel? Tobacco Control Monitoring and Evaluation Program 19

22 10. Are there are any other reasons why you feel that way? Examples of Special Probes 1. Would that be good times or bad times? 2. Would that be favorable or unfavorable? Examples of Neutral Ways to Preface a Probe 1. Overall Generally speaking Well, in general Yes, but Of course no one knows for sure Of course there are no right or wrong answers We're just interested in what you think Let me repeat the question... Pausing or Repeating a Question Thoughtful answers are more complete and more accurate. Give the respondent time to think by reading the question slowly and pausing as appropriate. As you gain experience, you will learn how long a pause should be. Keep in mind that the respondent has never heard the questions before, so a pause that seems long to you might be just right for the respondent. Don't rush your respondents pausing is another way of indicating that you expect thoughtful answers. 1. Repeat the entire question if the respondent's reply indicates that they didn't understand it, or if they need more time to think about their response. 2. If the respondent has clearly eliminated a response option, you do not have to include it in the repetition. 3. Repeat the entire question unless you are sure that only one part of it was misunderstood. Open-ended Responses After listening to a respondent's answer to an open-ended question, repeat the response that you have written down. Make sure your words are acceptable to the respondent. Ways to Probe for Answers on Open-ended Questions 1. When you cannot understand the respondent's reply: "What do you mean?" or "Could you tell me what you mean by that?" "Would you tell me more about your thinking on that?" or "Would you tell me what you have in mind?" 2. When the respondent gives an incomplete answer: "Would you tell me more about your thinking on that?" or "Would you tell me what you have in mind?" 3. When the respondent responds with a tentative "I don't know" or "I hope so": 20 Tobacco Control Monitoring and Evaluation Program

23 "What do you think?" or "What do you expect?" 4. When the respondent has narrowed the choices to two or to a range between two: "Which would be closer?" or "Which would be closer to the way you feel?" What is Feedback? Feedback consists of statements or actions that indicate to the respondent that he or she is doing a good job. 1. Effective interviewers give feedback only for acceptable performance, not "good" content. 2. Give short feedback phrases for short, oneor two-word responses (usually for closeended questions). 3. Longer, more thoughtful answers deserve longer feedback (usually for open-ended questions). 4. A brief pause followed by a feedback phrase makes the feedback more powerful. 5. Specific study information and interviewer task-related comments also act as feedback because they motivate the respondent. 6. Telephone interviewers should give feedback for acceptable respondent performance from 30 to 50 percent of the time. Examples of Positive Feedback 1. Short I see... Uh-huh/Um-hmm. Uh-huh/Um-hmm, I see. Thank you. Thanks. 2. Long That's useful/helpful information. It's useful to get your ideas on this. Thanks, it's important to get your opinion on that. I see, that's helpful to know. It's important to find out what people think about this. That's useful for our research. 3. Interviewer task-related comments Let me get that down. I need to write it all down. I want to make sure I have it right (REPEAT ANSWER). We have touched on this before, but I need to ask every question in the order that it appears in the questionnaire. It's important to find out what people think about this. That's useful for our research. Tobacco Control Monitoring and Evaluation Program 21

24 Restaurant Telephone Interview Log Interv iewer Initial s Interv iew comp leted Name of Restaurant MM Betty's Supper Club City Tomah Contact Name/ position Betty Long, Manager (or Jerry Long, Comanager) Phone number Call 1: Date and time (456) /04/ :15 AM (Call at 4:30 today) Call 2: Date and time 02/04/2001 4:35 PM (Call tomorrow pm) Dates of survey: Interviewer(s): Call 3: Date and time Call 4: Date and time Group conducting survey: Notes: 2/5/01 Jerry Long can also answer questions. Tobacco Control Monitoring and Evaluation Program 22

25 Worksite Telephone Interview Log Interv iewer Initial s Interv iew comp leted Name of Worksite City Contact Name/ position Phone number Call 1: Date and time Call 2: Date and time Group conducting survey: Dates of survey: Interviewer(s): Call 3: Date and time Call 4: Date and time Notes: MM Green Lawn and Garden Kaukauna Joe Green, Manager (456) /12/ :15 AM (Call at 4:30 today) :35 PM (Call tomorrow pm) They own Green Acres Nursery in Manitowoc. Tobacco Control Monitoring and Evaluation Program 23

26 Appendix 7: County codes: Wisconsin Adams 001 Ashland 003 Barron 005 Bayfield 007 Brown 009 Buffalo 011 Burnett 013 Calumet 015 Chippewa 017 Clark 019 Columbia 021 Crawford 023 Dane 025 Dodge 027 Door 029 Douglas 031 Dunn 033 Eau Claire 035 Florence 037 Fond du Lac 039 Forest 041 Grant 043 Green 045 Green Lake 047 Iowa 049 Iron 051 Jackson 053 Jefferson 055 Juneau 057 Kenosha 059 Kewaunee 061 La Crosse 063 Lafayette 065 Langlade 067 Lincoln 069 Manitowoc 071 Marathon 073 Marinette 075 Marquette 077 Menominee 078 Milwaukee 079 Monroe 081 Oconto 083 Oneida 085 Outagamie 087 Ozaukee 089 Pepin 091 Pierce 093 Polk 095 Portage 097 Price 099 Racine 101 Richland 103 Rock 105 Rusk 107 Sauk 111 Sawyer 113 Shawano 115 Sheboygan 117 St. Croix 109 Taylor 119 Trempealeau 121 Vernon 123 Vilas 125 Walworth 127 Washburn 129 Washington 131 Waukesha 133 Waupaca 135 Waushara 137 Winnebago 139 Wood 141 Tobacco Control Monitoring and Evaluation Program 24