Factors Influencing Employee Survey Response Rates Ten variables that can affect employee response and survey validity Discussing the many factors that drive response rates to employee surveys Copyright 2012
Executive Summary The response rate of an employee survey determines the validity of the results, the higher an organization s response rate, the more likely the results will be representative of the population. This provides greater confidence that the results are truly representative of the attitudes and perceptions of the total population of employees. Many factors affect response rates to employee surveys. Some of these may be within the organization s control, while others are not. The exact impact of each factor can be extremely difficult to quantify, primarily because the quality of survey implementation is rarely consistent across organizations, or across units within a given organization. In addition, the various factors are likely to interact to affect response rate. For example, efforts to allay concerns regarding confidentiality of responses is likely to be more successful in improving response rates within an organization characterized by high trust in management than one where trust is low. Another consideration is the natural limit to survey response rates for organizations of any size. For an organization larger than a few thousand, that limit is typically in the low 90 percent range. This means efforts to improve survey response rates may have less impact as organizations approach their natural limit; that is, it will be much easier to improve the response rate from 45 percent to 55 percent than it will be to increase it from 75 percent to 85 percent. Factor 1. Administration Method Typically, the greatest determinant of survey response rate is the method used to administer the questionnaire. When the organization has a greater the degree of control over when and where employees complete the survey, the response rates are generally higher. Typical response rates for three frequently used administration methods are: Group administration: Employees gather in a central location, often at an assigned time, and are given an opportunity to complete the questionnaire. While participation in the survey is voluntary, it is not unusual for meeting attendance to be mandatory. Typical response rate (percentage): High 80s to low 90s. E mail / office mail: The employee receives the questionnaire either by e mail or office mail. They are expected to complete the questionnaire when they have an opportunity, typically on company time. Typical response rate: Low 70s to mid 80s. Postal mail: The questionnaire is sent via postal mail service to the employees' residence. They complete the questionnaire on their own time and return it to the organization by postal mail. Typical response rate: High 50s to mid 60s. 1
Factor 2. Length of Administration Period Generally, the longer a survey is open to participants, the greater the percentage who will respond. However, this process produces diminishing returns. Most responses occur fairly early in the administration period, with smaller spikes in response appearing after reminders are received by participating groups. Using software that is able to track responses and send follow up e mail reminders to non respondents is one way to continue response momentum. As a result, most large organizations to administer an employee survey over a period of 2 to 3 weeks. However, as seen in Figure 1, meaningful increases in response rates can be obtained by extending the administration period out as long as 6 weeks. Figure 1 2
Factor 3. Communication Besides the administration method, communication is the factor with the greatest potential to influence response rate. This includes the communication employees receive before, during, and after the survey administration period. Pre survey: The entire survey process must be effectively communicated to employees prior to the administration period. There is nothing more likely to dampen response rates as a survey showing up unannounced in an employee s in basket or e mail. The key considerations for presurvey communications are: Quantity: It is difficult to argue that there can be too much communication to employees prior to the administration period. Every available channel should be used to present a consistent, positive message regarding the survey process. Content: In order to maximize response rates, employees must understand: o Why is the business surveying employees? An explanation of why the survey is being administered and why the information is important to the business is essential. This is less important for organizations with established survey processes however there are always new employees who may not know. o What is the survey process? Ideally, this should include a timeline and a description of how decisions will be made regarding survey related priorities and actions. o What is expected of me? What role should the employee have in the process beyond simply filling out the questionnaire? o What can I expect from the business after the survey? An explanation of how the business plans to communicate the results and their use. During the survey: There is unequivocal evidence that reminders sent during the survey administration period boost response rates. Figure 1 shows when reminders were sent to employees during the administration period. In each case, there is a corresponding spike in responses immediately afterward. Post survey: Many organizations are diligent about communicating survey results to employees once they are available, often for the individual work group, department, or business unit, as well as for the overall organization. Where they frequently fall down is in communicating the actions that have been taken in response to the survey results, and explicitly linking those actions back to the survey process. If the organization doesn't make the connection between survey results and actions taken in the business apparent, most employees won't make the connection on their own. 3
Factor 4. Management Support Management support of the survey process indicates to employees that the results will be used to improve the business. Without active, visible support from all levels of management, the survey runs the risk of being perceived as a Human Resources exercise. An example of this impact is evident in Table 1. Response rates are shown for five organizations within a large government agency. It is clear in this case that greater management support was indicative of higher response rates. Table 1 Organization # of Employees Response Rate % Management Involvement* A 1,261 19.5 1.83 E 6,218 29.0 2.54 D 1,225 39.3 3.19 C 10,116 38.2 3.62 B 4,807 59.7 4.27 * As rated by Human Resources managers within that organization on a 1 to 5 scale. 1 = Very Uninvolved, 5 = Very Involved Factor 5. Trust in Management Although very difficult to quantify, organizations where employees distrust management seem to have substantially lower response rates than those where trust is high. And it s not just trust in senior management. Distrust of how middle managers or supervisors might misuse the survey process can also have a negative impact on response rates. Large scale efforts to increase participation in such organizations without directly addressing the trust concerns of employees may result in increased participation, but at the cost of data quality. Factor 6. Response to Prior Surveys One truism shared among survey practitioners is that the simple act of asking questions of employees raises their expectations that action will be taken in response to their input. 4
Over time, if employees perceive repeated inaction following the survey, they will be less likely to participate. Table 2 presents the response rates for three administrations of a survey in an organization of approximately 800 people. Despite strong recommendations to the contrary, no action was taken by the organization in response to any of the survey results. As can be seen, response rates dropped substantially. Table 2 Survey #1 Survey #2 Survey #3 Response Rate 89% 76% 67% Factor 7. Incentives and Accountability Incentives and accountability effectively represent the carrot and the stick for survey participation. Incentives may be given at individual and/or group levels. Having coffee and donuts for people when they take the survey can be considered a tradeoff for employees time and is not likely to be seen as an incentive. Alternatively, competitions or contests, with, for example, the business unit(s) with the highest response rate(s) receiving a prize or benefit can help to engage employees in the process. It is also possible to build accountability for participation into managers' performance objectives. This can be done as either achieving a target response rate from their teams, or as an increase in participation from the last survey cycle. This approach also has the added benefit of supporting management s ownership of the survey process (versus Human Resources). Factor 8. Questionnaire Length There is some evidence that increased survey length can have a negative impact on response rates. However, the communication about the survey, the questionnaire instructions, questionnaire format, and content can reduce the negative potential of a longer questionnaire. In any case, the impact is certainly not linear. Questionnaires seem to come in one of four lengths: Short, moderate, long and "Please tell me you re kidding. As long as you stay out of the last category, questionnaire length is likely to have minimal impact on response. Factor 9. Database Accuracy Use of electronic (e mail) administration, means the accuracy of the HRIS database from which the e mail addresses are drawn is increasingly more important as a factor in response rate. In many large 5
organizations, the HRIS system is rarely considered completely accurate. Estimated error rates in excess of 10% are not unsurprising. In turn, this can significantly limit the potential for participation. Factor 10. Mobile Technology This is a fairly new variable in surveying, but one that is likely to increase in importance over time. With more and more employees using mobile technology rather than standard desktop computers, incompatibility among the survey platform and mobile devices has the potential to be a significant limiting factor to response rates. 6
About On the Same Page Founded in 2002, On the Same Page (www.on the same page.com) is named for what we take pride in being on the same page with our clients. Working in collaborative relationships, our consultants develop and apply customized internal communication strategies that engage employees and drive business outcomes. We approach and solve problems like company insiders, so we can deliver client ready solutions that simplify the complex, convert challenges into opportunities and provide clarity out of uncertainty. Whether a client is in the midst of a merger, is transforming its operating model or needs to achieve higher employee engagement, we always start the process the same way: by listening. We get to know businesses and cultures intimately so we can uncover solutions that fit the particular work environment, support the company s objectives and meet and exceed goals. Our services include: Strategy and execution during major shifts in direction such as brand positioning, mergers and acquisitions and restructuring; Engagement strategies to enhance performance in safety, quality, productivity and customer service levels; Communication assessments, organizational surveys and key leadership meetings; Training and coaching for leaders, managers and frontline supervisors; Interim staffing for communication leaders or team members on leave; and Objective, third party business advice. 7 From smaller firms to Fortune 500s, and across a vast array of business sectors, our clients run the gamut. They include: Alpha Natural Resources; American Standard Brands; Cox; Hess; ITT Exelis; Mars; Merck; Nike; Novartis; PepsiCo; Textron; and The Vitamin Shoppe. (For a full client list, visit http://www.on the same page.com/clients/our_clients.asp). Under the leadership of our owner and CEO Tracy Benson, we ve experienced our own transformation. Over the past decade, we ve grown from a dedicated group of solo practitioners into a cohesive, dynamic team with a shared purpose and identity. We d love to help you turn the page and begin the next chapter in your company s growth. To explore how we can support you, please email wecanhelp@on the same page.com or call 914 304 8122. On the Same Page, LLC PO Box 751 * Katonah, NY 10536 * wecanhelp@on the same page.com * 914 304 8122 Visit us on LinkedIn * Facebook * Twitter