Resource Guide for Community Survey Projects

Transcription

1 Resource Guide for Community Survey Projects Laura L. Payne Ph.D. Associate Professor and Extension Specialist OFFICE OF RECREATION & PARK RESOURCES Department of Recreation, Sport & Tourism University of Illinois at Urbana-Champaign

2 Acknowledgements The development of this resource manual was a group project and numerous people provided feedback and advice as this project moved from a concept to a tangible resource. Special thanks to Dr. Terry Schwartz, Director of the Winnetka Park District, and Craig Culp, Executive Director of the Northern Suburban Special Recreation Association (NSSRA), for their thoughtful and thorough feedback and guidance. I would also like to thank Robin Hall, Jarrod Scheunemann, and Jill Gurke for reviewing several drafts of this resource guide and helping out so much along the way. i

3 TABLE OF CONTENTS Purpose of this Resource Guide... 1 How to use this Resource Guide... 1 Planning for a Community Needs Assessment and/or Evaluation... 1 The difference between needs assessment and evaluation... 1 When to survey your community... 2 Creating a Quality Sample from the Population... 2 Sample defined... 2 Sampling strategies... 3 Random sample... 3 Stratified random sample... 3 Systematic random sample... 3 Convenience sample... 4 Obtaining a Random Sample... 4 Sample Size: How Large Should the Sample Be?... 5 Additional Sampling Considerations... 5 Response Rates and Reaching Diverse Sub-Groups... 6 Reaching diverse popluations... 6 Survey translation... 6 Oversampling in diverse neighborhoods... 6 Personal intercepts survey distribution... 6 A combination approach... 7 Optimizing response rates... 7 Personalizing the mailing... 7 Inform respondents ahead of the survey... 7 Follow-up post cards... 7 Handling non-response bias... 8 Methods for Collecting Data... 9 Deciding the Content of Your Survey Constructing Your Survey Measuring satisfaction levels Overall satisfaction Overall program participation and satisfaction Satisfaction with program features Importance-performance analysis Participant behaviors Demographic information Open-ended feedback Motivation for participating and benefits Tips for Writing Effective Survey Questions: Enhancing Reliability and Validity Mutually exclusive answer categories Exhaustive answer categories Balanced answer categories Notes about neutral or neither agree or disagree answer categories Managing, Analyzing Data and Report Writing Conclusion References Appendix A ii

4 Purpose of this Resource Guide The purpose of this resource guide is to provide easy to follow instructions and guidance for community parks and recreation staff in the process of conducting a community needs assessment and/or evaluation. This is a practical how to reference guide that is helpful for planning and implementing needs assessments and evaluations. How to use this Resource Guide This resource guide is meant to be helpful in the design and implementation of a community needs assessment and/or evaluation project that involves the use of surveys. In this guide, you will find information, practical examples and strategies for the following: determining types of questions to ask writing survey questions determining sample size acquiring a quality sample of respondents optimizing survey response rates managing non-response bias analyzing data and reporting results Overall, the purpose of this guide is to give you enough knowledge and insight to design and implement a project. In addition to using this resource guide, the Office of Recreation and Park Resources (ORPR) is happy to discuss and review with you portions of your project as well. Planning for a Community Needs Assessment and/or Evaluation The difference between needs assessment and evaluation The terms needs assessment and evaluation are often used interchangeably. Although there are some similarities between these concepts (e.g., both involve information gathering, use of surveys, asking questions, identifying a sample), they generally have different goals. Needs assessments focus on understanding needs of a population or the feasibility to develop a facility, park, or a set of programs. Questions are asked to determine peoples attitudes toward programs, facility development, and land acquisition or for funding various projects. Understanding attitudes involves asking questions about peoples feelings (e.g., about a proposed facility), 1

5 beliefs about a proposed project (e.g., will building a fitness center be a wise use of agency resources, is a fitness center needed in the community, etc.), and examines peoples behaviors (e.g., how likely it is a person/family would use a fitness center, attend a program, etc.). Evaluation on the other hand is more judgmental and focused on obtaining feedback regarding program quality, facility maintenance/upkeep, staff customer service, and other critical elements of the operation. The goal is to understand the agency s performance related to programs, parks, facilities, and staff. The agency is interested in understanding how satisfied people are with various features and operational elements of the organization (e.g., staff knowledge, facilities upkeep, program quality, parks upkeep, marketing effectiveness, customer service, etc). Since significant time, money and effort are required to develop and implement a quality needs assessment or evaluation, they are often combined into one large project with a dual purpose. This is a common occurrence and examples of evaluation/needs assessment surveys are included in the appendix to illustrate how this can be accomplished. When to survey your community: The timeframe for conducting a survey in your community can depend on a variety of factors such as assessing the type of capital projects the agency has planned, involvement of residents in the planning process, and the length of time since the last evaluation. Most community park and recreation agencies conduct a community needs assessment and/or evaluation every three to five years. The need to re-commit to a study will depend on any number of issues that need to be reviewed in the organization (e.g., expansion, grant seeking, land acquisition, etc). Oftentimes, public input must be captured to qualify for a grant, understand the willingness of the community to pursue a project, &/or purchase a parcel of land. Creating a Quality Sample from the Population Sample defined A sample is identifying a portion of a population from which to gather information. In this case, the information gathered is for a needs assessment and/or evaluation survey. Many people 2

6 mistakenly believe that everyone in a population must be surveyed. This would indicate that an entire population be surveyed, which is time consuming, costly and not necessary. Sampling strategies There are many different ways to obtain a sample. In this section, sampling strategies such as 1) random sample, 2) stratified random sample, 3) systematic random sample, and 4) the strengths and weaknesses of a convenience sample are defined and discussed. Random sample: This is the ideal sample to achieve. A random sample means that every person in the population has an equal chance of being selected to participate in the survey. Exercising the random sample method will help make your sample more representative of the population. It is important to note that since a random sample is more likely to be representative of the population, it is also more likely to allow you to generalize your survey findings to this population. Stratified random sample: Agencies are often interested in the opinions of sub-groups such as age groups, neighborhoods (e.g., east side vs. west side, school neighborhoods), or gender. In this case, you may want to divide or stratify your sample by these sub-groups. For example, in one study we determined the population of the east side of the community and the west side of the community and then drew a random sample proportionate to the neighborhood sizes. This helped to ensure our sample was balanced in terms of the number of people who got surveys on the east side and the west side. Systematic random sample: This approach is usually used when distributing surveys in person around the community. The community can be defined traditionally as XYZ Village or a major tourist attraction such as a zoo. Rather than give every person you see or every person you see who looks friendly and cooperative a survey, you select every Nth person to whom you would give the survey. People often use a number between 1 and 5. You can also develop a system for selecting a person if you approach a group of people. As suggested by Salant & Dillman (1994), a non-threatening approach is to use the following method: In order to give each person in your group an equal chance of participating, I d like to give the survey to the person who has had the most recent birthday. 3

7 Convenience sample: Convenience samples happen when surveys are distributed without a systematic or random sample approach. For example, standing in a park and distributing surveys only to people who appear to be friendly and cooperative would not result in a random sample. The biggest disadvantage to using a convenience sample is that equal chance of participation is not given to each member of the population, thereby significantly reducing the likelihood that your sample will be representative of the population you wish to survey. Another limitation of convenience sampling is that the findings of the survey may not be generalizable to the population, because the sample wasn t selected randomly. Obtaining a Random Sample This can be achieved in a number of ways. There are companies that conduct market research and specialize in providing (for a fee) random samples for research projects. These companies own and maintain large databases that can be accessed. For a fee, the sampling company can produce a random sample for projects. Here are examples of two reputable companies: Survey Sampling International LLC 6 Research Drive Shelton, CT USA Phone: Website: Marketing Systems Group: GENESYS Sampling Solutions 565 Virginia Drive Fort Washington, PA Phone: Website: 4

8 Sample Size: How Large Should the Sample Be? Assuming you use random sampling procedures, it may surprise you that you don t need more than 400 or so completed surveys to represent a population as large as 100,000 or more people. In order for your sample to have 5% +/- sampling error at a 95% confidence level (which means you are 95% confident that your sampling has only 5% error), follow the guidelines in table 1 provided by Salant & Dillman (1994): Table 1. Sample Size Needed Per Population Population Size Sample Size Needed for 5% +/- Error , , , , , , , ,000, Kish, Leslie Survey Sampling. New York: John Wiley and Sons, Inc. Additional Sampling Considerations In addition to using percent of acceptable error and confidence level to determine your sample size, you might also consider how many sub-groups you wish to divide your sample into for analysis. The more sub-groups you wish to divide your sample into, the larger it needs to be. There are no strict rules for how much larger your sample needs to be to accommodate subgroups, however Kish (1965) suggests that between 30 and 50 observations (i.e., completed surveys) are needed for each sub-group. For example, if you plan to divide your sample into four neighborhoods, it will be important to have between 30 to 50 completed surveys from each of those neighborhoods. 5

9 Response Rates and Reaching Diverse Sub-Groups Although you may only need approximately 400 completed surveys (based on a random sample) to represent a large population, keep in mind that most mailed surveys yield between a 30 and 50% response rate. Therefore, to ensure you get 400 completed surveys, it will be important to send out at least 800 to 1,000 surveys. Also, even if you feel confident you have a current and complete mailing list, there will be some unusable addresses in the mailing list and it is important to plan for some undeliverable surveys. Reaching diverse populations Communities are becoming more diverse each year. For example, suppose Sunnyvale, USA s Hispanic population has increased by 18% from 2000 to 2010 and now represents almost a quarter of the total population of Sunnyvale. Oftentimes, response rates on mailed surveys are lower from ethnically/racially diverse populations. There could be many reasons for this. For example, language could be a barrier to their participation. Another reason for non-response is that the topic may be perceived as not important or timely for the person. Whatever the reason, there are ways to improve the response rates from ethnically/racially diverse populations. The following are a few strategies that can be used to improve response rates: 1. Survey translation: In areas of the community that are highly populated by, for example, Latinos/Latinas, providing the survey both in Spanish and English is one way to improve the response rate. 2. Oversampling in diverse neighborhoods: It is common for the response rates in diverse areas to be as much as 50% lower than overall response rates. Therefore, a good strategy is to oversample in these areas to optimize the response rate. Oversampling can help reduce sampling error that can occur as a result of not having an adequate sample of a particular sub-group (i.e., older adults, families with children, ethnic/racially diverse households). 3. Personal intercepts survey distribution: Using trained field staff to take surveys directly into neighborhoods to be delivered personally at the respondents home is another strategy for increased response rates. There are many ways to do this. Field staff can divide the 6

10 neighborhood(s) into manageable geographic units, take surveys for a certain defined area and deliver the surveys personally. The surveys can be completed on the spot, dropped off and then picked back up at a designated date/time by field staff, or respondents can return the completed surveys in a pre-addressed postage paid envelope. The method chosen will depend on the human resources available and the budget for postage. Sometimes trained volunteers can be used as field staff. This is an efficient strategy that also builds interest and involvement by community members in the project. 4. A combination approach: Another strategy is to combine any of the strategies mentioned above. For example, you could oversample a particular sub-group and hand-deliver a Spanish language survey. Or oversample and provide the Spanish language survey. Optimizing response rates Following up with your sample is important to optimize your response rates. Don A. Dillman developed a process referred to as Total Design Method, which involves follow-up processes to improve response rates. 1. Personalizing the mailing: If possible, including a personalized cover note (produced using a mail-merge program) can help increase your response rate. 2. Inform respondents ahead of the survey: A pre-survey postcard, phone message, or message can be useful to inform respondents that a survey is coming so they are more likely to notice it and complete it. 2. Follow-up post cards: Sending a follow-up postcard seven days after your mailing the initial survey offers respondents a friendly reminder to complete and return the survey. An example of text that can be used for the follow-up postcard is provided in figure 1 below. 7

11 Figure 1. Sample Follow-up Postcard Thank you for taking part in the Sunnyvale Park District community recreation and park survey. If you have completed and returned the survey, thank you for your valuable input. If you have not had a chance to complete the survey, won t you take a few minutes to fill it out? We are looking forward to getting your feedback. If you have any questions, feel free to contact me at the number below. Remember, as soon as you return your completed survey, we will remove you from our mailing list, and you will be eligible for our door prizes, which are $25 gift certificates for Super Target stores. Again, thanks for your help! Laura L. Payne Handling non-response bias Sometimes the response rate for your survey is lower than you had hoped. If your survey resulted in a response rate that is less than 50%, it is important to check for non-response bias. This is especially important if your sample is not a random sample. An effective way to check for non-response bias is to select a portion of your non-respondents (for a sample of 1,000, contact 100 people; a sample of 500 contact 50 people) randomly and conduct brief phone surveys in which you ask some of your demographic questions (gender, age, race, education) and perhaps 1-2 of the key survey questions. Armed with this new data for nonrespondents, you can compare the composition of your sample of respondents with your smaller sample of non-respondents. If the average age, education level, race/ethnicity or use of the park and recreation agency s programs, services and facilities is significantly different than your sample, then you have an issue with non-response bias. If the two groups are not significantly 8

12 different (and ORPR staff can help you analyze your data for this difference), then you do not have a problem with non-response bias. Methods for Collecting Data There are many ways surveys can be collected from a sample (see table 2). If financial resources are limited, surveys might be distributed by hand or if the survey is relatively short, it can be completed by respondents at various community locations (e.g., parks, community centers, the library, post office, supermarkets). However, remember the importance of using the random sampling procedures outlined on pages 2-3 of this resource guide. Using the Internet and mailing surveys are another option. Below is a list of strengths and weaknesses of each data collection method to help you make the most informed decision for your project. Table 2. Data Collection Methods: Advantages and Disadvantages Data Collection Method Advantages Disadvantages Mailed Surveys Relatively low cost Fairly quick to assemble for mailing Respondent has time to give thoughtful answers A random sample can be achieved Personal Intercept: Personally Questions are answered by distributing surveys (and field staff sometimes picking them up) Field staff can designate a household respondent Higher response rates Online/Internet Surveys Low cost Quick responses Time is allowed for thoughtful answers Telephone Surveys Staff are available to answer questions Relatively quick to complete Lower response rates No one is present to answer questions about the survey Field staff/volunteers are required Can be costly if you don t use volunteers Samples are limited to Internet users No one is present to answer questions about the survey Must obtain accurate lists Response rates can still be low Can be expensive Surveys must be fairly short (15 minutes or less) Questions must be simple and very clear 9

13 Deciding the Content of Your Survey Once you ve found some assistance for developing your survey project, the next step is to determine what type of information you want to obtain through the written survey. You may check all of the following areas that apply: Type of Question Ways results can be Satisfaction Importance/ Participant Demo- Open- Motivations How you used Levels Performance Behaviors graphics ended & Benefits heard Analysis Effectiveness of program process Resource Allocation (efficiency) Staff Effectiveness Determine if goals & objectives were met Budget Justification Marketing (promotion and publicity) Staff Morale & Pride x x x x x x x X x x x X x x x x x X x x x x x x X x x x x x x X x x x x x x X x Constructing Your Survey You have now determined what areas and organization elements you would like to evaluate. The next section offers examples of written questions and scales that can be used to measure them. 10

14 To make it easy, you may duplicate any of these questions and use them for your own survey project. Measuring satisfaction levels There are many ways to measure customer satisfaction. You can measure overall program satisfaction and satisfaction with certain features of a program. In this section, we have provided several ways of measuring satisfaction. Overall satisfaction: Below is an example of how overall satisfaction can be measured. On a scale from 1 to 10, one being the lowest, and 10 being the highest, how satisfied are you with the program? Please circle one number below Overall program participation and satisfaction: Sometimes you want to get feedback from respondents on a wide array of programs, including satisfaction and participation. The following is an example of how you can gather a great deal of information from respondents efficiently and effectively. 11

15 In the past 12 months have you or anyone in your family participated in any of the following programs? Please check the appropriate box below. If you participated, check your satisfaction level with the experience. Overall Evaluation of Programs Have You Attended? a. Attended a special event (e.g., Fireworks Excellent Good Fair Poor No Opinion night at pool, fishing derby, 16 softball Yes No tournament) b. Aquatics Programs Yes No c. Arts Programs Yes No d. Adult Athletics Programs Yes No e. Youth Athletic Programs Yes No f. Camps (i.e., mini day, full day, athletic) Yes No g. Conservatory Classes & Events Yes No h. Fitness Programs (e.g., strength, aerobic, Yes No personal training) i. Fitness Center Yes No j. Golf Yes No k. Facility Rentals (i.e., Conservatory, Central, Yes No Golf Course, PlayPlex, RecPlex) l. Teens and Youth Programs Yes No m. Tots and Kids Programs Yes No Satisfaction with program features: Importance-performance analysis is a useful tool for measuring satisfaction with key program features. We recommend an importance-performance (I-P) analysis to measure satisfaction with program features such as staff, equipment, facilities, activity, variety, etc. A more detailed explanation of IP analysis is provided below along with a sample I-P scale from the Active Options (a community parks and recreation fitness/wellness program) program evaluation. 12

16 Importance-Performance analysis: An importance-performance (I-P) analysis was conducted of different attributes of the Active Options program. Respondents were asked to rate the importance of 18 program features on a seven-point scale from 1= not important, to 7=very important. Next, members were asked to rate how well Active Options performed on each of these attributes. Again, seven response categories were used, ranging from terrible to delighted. Average importance and performance scores were calculated for the 18 attributes. Usually, importance-performance scores are presented in the form of a scatter diagram that is divided into four quadrants: a) high importance and high performance (i.e., keep up the good work), b) low importance and low performance (i.e., low priority), c) high importance-low performance (i.e., put efforts here), and d) low importance-high performance (i.e., possible overkill). An example of an Importance Performance Scatter plot can be found in figure 2 below: 13

17 Figure 2. Importance Performance Scatter Plot However, one drawback of this method is that the charts are sometimes difficult to interpret and many features end up being clustered into one category. Gap scores provide another more straightforward method to determine priorities when analyzing the importance-performance ratings of program attributes. For more information on gap scores please see Appendix A. Participant behaviors: Behavioral data includes questions about 1) activities participants engage in within the agency/program, 2) activities they do at home or with friends, 3) how often they participate in sponsored programs/activities, and 4) how often they use recreation facilities and other services. Furthermore, participation data is important if you plan to do an individual impact evaluation. See impact evaluation section for further explanation. 14

18 Below is an example of an effective way to ask about recreation behaviors. 1. In the last 12 months, how often have you or anyone in your family used the following parks and facilities? Please check the answer that represents your use. Less than Never 4 + days per week 2 3 days per week 1 day per week 1 3 times per month once per month a. Central Community Center b. Majewski Athletic Complex c. Lions Recreation Center d. Lions Memorial Park e. RecPlex f. Mt. Prospect Golf Course and Community Center g. The Studio at Melas Park Another angle to participant behaviors is to determine outside of your agency s parks, facilities, programs and services, what organizations respondents use to pursue their (and their family s) parks and recreation pursuits. It may be useful to ask about other providers. Below is one example of how this information can be collected. 2. In addition to the Sunnyvale Park District, what other organizations do you and your family use for parks and recreation activities? Please check all that apply. Lawn Aqua XSport Fitness Public School based sports/ activities Church Another Community Park District Private/Parochial School based sports/activities Other Ice Arena Bally Total Fitness Other local public Golf Course Village Senior Center Other (please explain): Demographic information: Besides participant behaviors, it is also good to obtain some demographic information so you can create a profile of your users/participants/members. One 15

19 caution when determining what demographics to collect: many people are threatened by demographic questions, due to their personal nature (e.g., income, education, occupation, race). Therefore, as you select from the list of demographics below, be sure you know why you need that particular demographic and how you ll use it. For example, if your agency serves all middle class white residents of Sunnyville, USA, then you probably do not need to ask about race and income in your evaluation. You probably won t learn anything new (especially if there is recent demographic data available on the community you serve) and you may risk offending or threatening the survey respondent. Consequently, it may decrease your response rate because many people will not finish a survey once they see questions they perceive as threatening or offensive. Ask yourself these questions: 1. What will I gain from collecting information on age, marital status, occupation, disability status, or employment status? 2. Will, for example, married people versus widow/widowers answer the questions differently or in a way that indicates I need to examine their results separately, so I can consider their different opinions and preferences? If so, then you need to include marital status in your survey. Open-ended feedback: Although responses to open-ended questions are time consuming to analyze and report on, they are valuable because they give the program staff some insight into why the program was deemed successful or unsuccessful. For example, if survey respondents rated the knowledge of staff poorly overall, it would be insightful to include open-ended questions about weaknesses and suggestions for improving the program. The three most common open-ended questions are: What are the strengths of this program? What are the weaknesses of this program? What are your suggestions for improving the program? Motivation for participating and benefits: Obtaining information on benefits received from program participation and learning what motivates people to participate is helpful to learn how 16

20 well you have achieved your goals and objectives. This information can also be used to demonstrate that the program meets participant s needs. Although conducting a program evaluation may seem time consuming, remember, there are many resources available to assist you with every aspect of this important part of the programming process. Furthermore, the dividends of conducting an evaluation are useful for marketing, budgeting, staff morale and public relations. Tips for Writing Effective Survey Questions: Enhancing Reliability and Validity When writing a survey, there are some important tips to make your survey questions both valid and reliable. Reliability and validity are important concepts to keep in mind when constructing and implementing survey projects. Reliability and validity are complementary concepts that might be viewed as different sides of the same coin. As a reminder, reliability refers to consistently measuring a construct the same way under the same conditions over time. For example, when measuring peoples use of local parks, the way your question is written and how answer categories are constructed will affect how reliable the information is that you obtain from the question. Validity is a related concept that refers to the accuracy of the information you obtain from questions and answer categories on your survey. As a refresher, here are some friendly tips for effectively written survey questions which can help optimize the reliability and validity of your survey items. Mutually exclusive answer categories When constructing closed-ended questions with answer categories that use numbers, make sure your number categories do not overlap. For example, a common question could be: How often do you visit the Sunnyvale Recreation Center? 1-2 times per week 2-3 times per week 4-5 times per week 6 or more times per week 17

21 Two of the answer categories are not mutually exclusive; 1-2 times per week and 2-3 times per week overlap. Therefore, some people might check one category, and some the other category, which will damage the accuracy of your results. Exhaustive answer categories Another common mistake in questions that measure behavior is sometimes that the answer categories are not exhaustive. Take the example of the question above. Not every possible answer is accounted for because the question lacks an answer for never visiting the recreation center. The question and answer categories below adhere to both rules: mutually exclusive answer categories and exhaustive answer categories. How often do you visit the Sunnyvale Recreation Center? Never 1-2 times per week 3-4 times per week 5-6 times per week Everyday Balanced answer categories It is common to gauge peoples attitudes toward different parks and recreation programs, facilities and parks within a community survey. When constructing answer categories for attitude statements, make sure your answer categories are balanced. For example, the answer categories for the statements below are balanced. In table x, there are 2 categories to represent levels of agreement and 2 categories that represent levels of disagreement. 18

22 Statement Strongly Agree Agree Disagree Strongly Disagree I am satisfied with the variety of recreational opportunities offered by Sunnyvale Park Dist. Recreation programs, facilities & park areas are not important to my quality of life. The Sunnyvale PD is responsive to the recreational & park needs of the community An example of an unbalanced set of answer categories is in the question set below: Statement Strongly Agree Somewhat Agree Slightly Agree Disagree Strongly Disagree I am satisfied with the variety of recreational opportunities offered by the Sunnyvale Park Dist. Recreation programs, facilities & park areas are not important to my quality of life. The Sunnyvale PD is responsive to the recreational & park needs of the community Notes about neutral or neither agree nor disagree answer categories It is common when asking about attitudes toward different aspects of community parks and recreation for answer categories to include a category that represents a neutral opinion. This can often lead to people defaulting to select a neutral category, especially if your survey is long and they don t want to take time to think about the issue. It is often better (and of course this is situational and a judgment call to some extent) not to include a neutral category to optimize the opportunity for getting respondents to think about an issue or issues and make a value statement through their response. Otherwise, you may risk having a large proportion of your respondents answer neutrally and the information you get will be more limited. 19

23 Managing, Analyzing Data and Report Writing The process of data coding, data entry, analysis and report writing is a very important step in the survey research process and one that should be done with much care and attention to detail. The quality of the overall project may be judged in part by the readability of the report, presentation of results and how accurate and clean your data is in its final form. Therefore, if your agency needs assistance with this step in the process, the ORPR staff would be happy to consult with you and potentially assist with this step in the process. Conclusion In this resource guide, we ve discussed sampling, survey development, and important methodological considerations to optimize reliability, validity, representativeness and generalizability related to survey research projects. We hope this resource guide offers you practical tips and instruction for how to facilitate a survey research project. Please feel free to contact the Office of Recreation and Park Resources (ORPR) for more information and assistance regarding survey research projects. We are happy to help! Our phone number is: or by , please contact Robin Hall, Director of ORPR at rrhall@illinois.edu. 20

24 References Babbie, E. (1998). The practice of social science research. Belmont, CA: Wadsworth Publishing Company. Kish, Leslie (1965). Survey Sampling. New York: John Wiley and Sons, Inc. Salant, P. & Dillman, D. A. (1994). How to conduct your own survey. New York: John Wiley and Sons, Inc. 21

25 Appendix A: Gap Scores When analyzing the importance-performance ratings of program attributes, gap scores provide another more straightforward method to determine priorities. Gap scores are determined by subtracting the importance score from the performance score. A positive gap score indicates a good performance relative to the importance rating of the attribute. A negative gap score reveals that the performance rating was lower than the importance rating for a single attribute. In this analysis, four attributes were found to have the largest negative gap scores. These attributes are 1) times fitness classes are offered, 2) affordability, 3) locker room cleanliness, and 4) availability of fitness equipment. Again, positive gap scores indicate a high performance relative to lower importance for a single attribute. Attributes with high positive gap scores were 1) accessible to individuals with disabilities, 2) locker room size, 3) fitness test, and 4) availability of other program information. Table 3 below provides an example of gap score analysis. 22

26 Table 3. Gap Scores Summary of Attribute Means and Gap Scores Attribute Description Performance Mean Importance Mean Gap Score Accessibility Locker Room Size Fitness Test Availability of Info Fitness Test Schedule Variety of Equipment Facility Location Availability of Fitness Staff Number of Participants Variety of Classes # Classes Per Session Knowledge of Instructors Equipment Maintenance Helpfulness of Instructors Times Classes are Offered Fitness Equipment Availability Cleanliness of Locker Room Affordability