Measuring Employee Engagement: Off the Pedestal and Into the Toolbox Andrew Graham, School of Policy Studies, Queen's University December, 2015 I was recently asked by a senior HR advisor in a Canadian government about what was happening with respect to annual employee engagement surveys. This jurisdiction had been doing one for a number of years and was considering its strategic direction with them. What follows in the result of a brief survey of practice elsewhere across the country in government but also a review of recent findings and new practice in the private sector. Two main topics will be tackled: An emerging critique of the traditional annual employee survey, and What is trending that suggests a more varied approach in which the annual survey is just part of the mix. Hence, the theme of this note is that annual surveys, as they are used more, have developed some issues that are best addressed by moving into a mixed strategy. Take the survey off the pedestal standing alone and out there and put it into a broader employee engagement strategy. That being said, any public sector organization that has established and sustained a formal employee survey process continues to stand out. In a recent survey of American pubic agencies, only 30% had such instruments in place and in use. i This is an oversight piece, not complete and certainly not operational. In addition, some of the views are attributed and some are those of the author. Some Notable Canadian Practice Here are some examples of the Canadian public sector experience in measuring employee engagement: British Columbia Work Environment Survey: This program has been in place since 2006 and is now biennial. For more information: http://www.bcstats.gov.bc.ca/statisticsbysubject/employeeresearch/wes/ WESFAQ.aspx Alberta Corporate Employee Survey: Alberta has a long history of assessing employee engagement. It began a survey process in 1997. It has produced an annual Employee Engagement Index. Saskatchewan: There have been two whole- of- government surveys, the last in 2006. The government abandoned these, but there have been departmental surveys in most departments. Manitoba: Manitoba s Civil Service Commission conducts an employee engagement survey every three years. 1
Ontario: Ontario competed its last OPS Employee Engagement survey in February 2014. Canada: the 2014 Public Service Employee Survey was completed Oct. 2104. Results will be released in early 2015. Nova Scotia: The results of the 2013 Employee Engagement Survey are available at http://novascotia.ca/psc/employeecentre/employeesurvey/2013survey.as p Newfoundland: There is a series of biennial Work Environment Surveys, the last one available being 2011. Measuring Engagement Probably the most consistent message in general practice is that concerning the concept of employee engagement that should be measured through the use of a survey. From there, the similarities seem to end. Many names are used for this survey; an engagement survey, an attitude survey, work climate improvement survey, etc. Even more structures and survey questions are recommended concerning the best way to measure employee engagement. The famous Gallup study highlights twelve key elements that form the foundation of strong engagement. ii Another measurement example comes from Development Dimensions International (DDI). DDI has their own measure, E3, which assess three key elements of engagement (individual value, focused work, and interpersonal support) as well as provides a standard measure for employee satisfaction, which they feel is a result or consequence of engaging employees. But what emerges from a broad survey is how flimsy the linkage between the measurement of engagement and organizational performance actually are. Similarly, what creates engagement is contentious. Kieron Shaw reviewed employee engagement research of many entities including Gallup, Towers Perrin, ISR, and Accenture. Shaw s study struggles to provide any real clarification. In the end, Shaw states, It s arguably unfeasible to directly measure in the survey all the actions behind engagement, due to the fact that, there are potentially thousands of different individual actions, attitudes, and processes that affect engagement. iii Josh Bersin, in a searing discussion of the annual survey process, It s Time to Rethink the Employment Engagement Issue, iv commented on what is a major theme of this report: While this (holding an annual survey) is a good thing to do, most companies now tell us that this process is not keeping up. It s not detailed enough, it isn t real- time, and it doesn t consider all the work related issues which drive employee commitment. A new breed of engagement tools, vendors,, models, books and workshops has emerged. 2
Even more recently and certainly more stridently is the study by Converse Consulting of Calgary, Employee Engagement Surveys as a Statistical Confidence Game, the abstract of which states: Most employee research gets it wrong. It's a statistical confidence game placing the public relations objective of telling a good story above the practical objective of telling the truth by mangling statistical technique and the overselling the results. This economisting (sic) uses: statistical significance to identify 'important' differences that amount to trivial artifacts, regression models of engagement that are cherry- picked statistical fairy tales, and performance benchmarking yielding absurd claims of being one of the best companies to work for. Add to this, the belief that complex concepts can be reduced to a single number, and the result is evidence corruption, eliminating the possibility of rational decision making. Engagement surveys assign blame, promote tampering and destroy trust. The trend, therefore, is towards some revisionist thinking about the rightness of employee surveys. Issues with employee engagement surveys: 1. Not frequent enough: Employee engagement surveys are usually annual. Engagement varies through out the year and annual surveys do not capture them. Even a quarterly survey won t work because of the nature of the surveys. Surveys have lengthy action planning process. And because of this lengthy process, quarterly surveys can become a nightmare for HR and managers to coordinate and execute. 2. Single Scoring Leads to Issue Distortion: Nowhere is this more evident than in the treatment of so- called harassment percentages. In a number of organizations and jurisdictions, definitions were so vague or totally absent, that anyone would answer a question such as Have you ever been harassed in your workplace? with a resounding yes, even though there is no evidence of sexual 3. Aggregation reduces meaning: Often survey results are at such a level that their meaning is not clear or there is, in fact, a distortion of results at the working level. 4. Does not capture the specifics: Even though different employee engagement survey vendors claim that their survey captures the specifics, it is not possible by the very nature of the surveys. Most of them are multiple choice questions with some placeholder to capture user comments. To analyze the individual comments and then group them across an 3
organization, most of the specifics can be lost. Most importantly the context to user answers is seldom captured. Without lengthy discussions with teams post surveys, it is difficult to capture the specifics. Also employees usually feel there is no point in answering questions correctly since little to no action was taken from the previous year surveys. They may not participate at all or do not bother giving the right answers. 5. Lengthy or poor response planning: Because surveys do not capture the specifics, lengthy action planning is needed involving HR, senior managers, employees and managers. 6. Managers are busy and have no incentive to implement any actions: When there is a plan to implement, managers become busy with their regular work and the process looses momentum. 7. Lot of resources and monitoring: HR needs resources for implementing a survey, collecting data, distributing the data, action planning and monitoring. 8. Surveys Get Old: One key observation is that surveys can become useless after 4 or 5 years, especially if they remain the same. The conflict here, of course, is that you can only track trends by remaining consistent. On the other hand, by being consistent, you invite either gaming or form filling without meaning. Further, you do not create space to delve into shorter- term issues or trending ones. Other tools might need to be considered here. 9. Causality Not Clear: Even positive trending results over a number of years may be the result of a mix of factors, some of them changing even if the result is the same. This is the challenge of proving causality or of the survey s capacity to capture cause and effect relationships. The classic example is the reporting of job satisfaction in the face of a number of negative environmental factors such as poor supervision, inadequate resources, etc. All these issues notwithstanding, there is considerable support for surveying employee engagement in both government and private sector. In addition, the notion of the top 100 or 50 companies seems to be seductive, if more for corporate pride than to learn more about the workplace. Such misplaced popularity among some also threatens the credibility of the process within organizations. What s Trending? Bersin notes that While the survey is not going away yet, it is now being replaced by new tools and techniques that measure happiness, alignment and job satisfaction in real time. What appears to be forming is the notion that engaging employees to get overall feedback now includes a need to engage them in formulating policies that affect them, given them more real- time capacity to express their views on their role and engagement and use both push and pull techniques. Some of the trending developments, all with attendant vendors and consultants to help organizations along are: 4
Qualitative Research: This can take many forms, but always involved engaging employees in specific discussions, either associated with a quantitative survey (giving it meaning), focus groups on specific issues or task groups for individual changes. Shorter, Vertical, One- Topic Surveys: This is the opposite of the annual survey, which serve to compare engagement scores year to year. This type of survey is strategically focused on a single topic about which the organization needs to learn more with respect to employee attitudes. For instance, instead of one or two questions in a general survey about harassment, a vertical survey would focus entirely on this subject. Segmentation Analysis: Applying customer research tools that involve the use of cluster analysis to identify in more depth groups that require more specific responses. Social Media: Everybody is doing it, but organizations have to systematically open themselves up the twitterverse and other tools to get responses, quickly, through social media. Pulse Surveys: These are rapid response surveys on current topics. They are not in depth, but rather intended to gauge reaction to significant corporate events or shifts. Collaborative Platforms: This involves the creation of open survey web platforms that permit employees to engage in a conversation on a topic rather than just respond with a choice of 1 to 5. Text Analytics: Beyond having the new electronic platforms, new analytical capacity enables the employer to trend language and references that gauge overall employee mood, etc. Glassdoor (www.glassdoor.com/ )and related tools: Organizations need to monitor web- based rating sites to see how they are portrayed. These are anonymous sites that rate employers. With all their flaws, they can occasionally provide some insights, even with the tendency for the most disgruntled to use them. Some General Final Thoughts Many of the criticisms of the annual employee survey are valid, but none would suggest that any public organization that uses one should give it up. On the contrary, looked at from a broader perspective of good management practice, more should start doing them. Several key characteristics emerge as tests of a useful annual survey process: The survey is administered in a regular and predictable fashion. There is consistency in certain core questions that permit period- to- period comparison. Questions are well designed to get relevant information. The results have sufficient granularity that they can be matched up to accountable sub- units (departmental level for example) of the government. Assessment of the results involves either a set of further qualitative surveys or discussions at the executive and staff level of the meaning of the results. 5
There is an assignment focus of responsibility for the administration of the survey (normally Human Resources), but that the survey itself is owned by senior management. The survey is part of a broader engagement strategy that involves a mix of tactics, of which the survey is but one. Clearly, new tools are emerging, many of them using social media platforms. What these give an organization are the ability to go deep and go quickly to engage either staff response to events, changes or developments. Further, such media move in all directions. Therefore, they also permit organizations to be more open to staff engagement through input to ongoing internal blogs on specific topics or through a 21 st century version of the suggestion box. Organizations can either be passive, e.g., e- mail us your ideas or create an internal secure platform for discussion groups or active, e.g., pushing out pulse surveys, single questions on rapidly emerging issues. Whatever is done, this must be managed in a consistent way across the organization. Many government departments now have internal chat groups. These work especially well in communities of practice where people with similar jobs or backgrounds can share current experience on issues. All of this must be driven by a model of engagement for the organization. It is clear that governments with experience in this area and the private sector have moved well beyond job satisfaction as that model. The private sector experience, reinforced for the public sector by such concept of a public sector value chain, is that this model must be linked to the end goals of the organization, not simply on the current concerns of the individual employee. Good surveys then begin with a strategic review of the business combined with a mixed engagement measurement strategy. Many of the new tools are more current than an annual survey. Many are more precise, but limited to a specific topic. Combining the two is what makes the most sense. i Governing Institute, Employee Engagement, report to the International Public Management Association for Human Resources, available at http: ipma- hr.org/sites/default/files/.../adp- IPMA_EmployeeEngagement.pptx ii Thackray, J. (2001). Feedback for real. Gallup Management Journal, http://gmj.gallup.com/content/811/feedback- for- Real.aspx iii Shaw, K. (2005). Employee engagement, how to build a high- performance workforce. Melcrum Publishing Limited, ISBN: 0-9547741- 3-2. http://www.emeraldinsight.com/insight/viewcontentservlet?filename=published /EmeraldFullTextArticle/Articles/0500210701.html#b15 iv Bersin, Josh, It s time to Rethink the Employee Engagement Issue, available at http://www.forbes.com/sites/joshbersin/2014/04/10/its- time- to- rethink- the- employee- engagement- issue/ 6