ROI CERTIFICATION. Building Capability and Expertise ROI Institute, Inc.

Transcription

1 ROI CERTIFICATION Building Capability and Expertise with ROI Implementation 2013 ROI Institute, Inc.

2 Building Capability and Expertise with ROI Implementation Phone (205) ROI Institute, Inc.

3

4 Table of Contents Module 1 Setting the Stage Program Objectives Program Success Role of the ROI Implementation Leader Skill Areas for Certification Certification Projects Case Study Presentation ROI Project Implementation Plan Requirement Projects Global Communications Paradigm Shift in Programs Definition of Results-Based Programs How Results-Based are Your Programs? Human Capital Perspectives Increased Interest in the Value of Human Capital Three Journeys Take us to the Use of ROI Apex, Inc Comparison of Approaches to Measure the HR Contribution Measuring the HR Contribution: Status HR Accountability Progress Leading Edge Approaches to Measure the HR Contribution Recommendations for Measurement Categories Common Human Capital Measure Setting the Investment Level Human Resources Development Issues Human Capital Investment Benchmarks The ROI Methodology Valuing Human Capital Three Approaches Module 2 The ROI Methodology Reliance Insurance Company The Results Key Issues with This Level of Analysis The ROI Process Generates Six Types of Data ROI by the Numbers Why Use Impact and ROI Analysis? Applications Basic Elements Evaluation Framework Defining the Return on Investment

5 ROI Target Options Characteristics of Evaluation Levels Evaluation Framework and Key Questions Chain of Impact Business Alignment Linkage Matching Evaluation Levels with Objectives Evaluation Planning Data Collection During and After Program Isolating the Effects of the Program Example Use of Control Group Example Use of Trend Line Analysis Example Participant s Estimation Converting Data to Money Example Converting Data Using External Databases Cost of a Sexual Harassment Complaint using Historical Costs and Expert Input Example Linkage with Other Measures Tabulating Program Costs Intangible Benefits are Important ROI Process Flexibility Do Not Confuse the CFO Common Target Audiences Select Media Implementation Issues Key Implementation Actions Retail Merchandise Company Utility Services Company Matching Exercise: The Twelve Guiding Principles of ROI Module 3 Planning Evaluation Regional Public Utility Level 3 and 4 Objectives Nissan Motor Manufacturing Company Wachovia Bank Metro Hospital Regional Health Center Department of Internal Affairs International Car Rental Results-Based Approach Key Alignment Questions Developing Reaction Objectives Developing Learning Objectives Developing Application Objectives Developing Impact Objectives Developing Impact Objectives

6 Developing Level 3 and 4 Objectives Evaluation Targets Criteria for Selecting Programs Level 3 Evaluation Criteria for Selecting Programs Level 4 Evaluation Evaluation Planning Meeting Data Collection Plan ROI Analysis Plan Project Plan Global Financial Services, Inc Retail Merchandise Company The Business Case for EI Self-Test: How Results-Based Are Your Human Resources Programs? Module 4 Collecting Data Data Collection Issues Methods for collecting Program Data Classic Evaluation Instruments Applications of Data Collection Instruments Matching Exercise Data Collection Exercise Part Survey/Questionnaire Design Common Mistakes in Survey/Questionnaire Design Selecting Survey Questions What Makes an Effective Survey Questions? Types of Test Types of Objective Tests Steps to Developing Objectives Tests Interview Design Focus Group Guidelines Observations Guidelines Observation Methods Typical Sources of Performance Data Monitoring Performance Data Characteristics of Effective Instruments Factors to Consider when Selecting Data Collection Methods Sources of Information for Program Evaluation Factors to Consider When Determining Timing of Follow-up Data Collection Exercise Part Cyber International Leadership Development Program Impact Questionnaire Sales Culture at Progress Bank Developing ROI with Action Planning Action Plan Performance Contract Process Steps The Performance Contract Should Be

7 Response Rate Exercise Follow-up Questionnaire Content Issues Checklist Impact Questions for Follow-up Evaluation Performance Contract Chain of Impact Questions Increasing Questionnaire Response Rates Module 5 Isolating the Effects of the Program First Bank Isolating the Effects of a Program Matching Exercise Several Factors Contribute to an Improvement After a Program is Conducted Techniques to Isolate the Effects of Programs Financial Services, Inc Use of Control Groups Example Control Groups Problems Micro Electronics, Inc Healthcare, Inc Example Use of Trend Line Analysis Conditions for Trend Line Analysis Use Woody s, Inc National Bank Using Estimates to Isolate the Effects of a Program The Power of Estimates Key Issues with Estimates Credibility of Data Credibility of Outcome Data is Influenced by the: Isolation Methods Example Use of Participants & Managers Estimate of Training s Impact National Computer Company The Wisdom of Crowds Multi National, Inc Module 6 Converting Data to Money Examples of Hard Data Characteristics of Hard Data Examples of Soft Data Characteristics of Soft Data Converting Data to Money Five Steps to Convert a Measure to Money Converting Data to Monetary Values Data Conversion Issues Standard Values are Everywhere Examples of Techniques (Convert Data to Monetary Value) What Makes an Expert Credible?

8 Example Converting Data Using External Database Finding Data Link with Measures Converted to Money Classic Relationships Example Linkage with Other Measures Estimating the Value Turnover Costs Summary Converting Data to Monetary Values: Questions to Ask Short-Term Solutions When Estimating Time for Long-Term Solutions Converting Your Level 4 Measures to Money Total Fitness Co Data Conversion Four-Part Test Module 7 Tabulating Costs and Calculating ROI Reasons for Developing Cost Data Issues About Tracking Costs How Much Should You Spend on HR? Overall Cost Categories Tabulating Program Costs Recommended Items Prorating Cost Example of Prorating Overhead Allocation Example Costs Classification Matrix Cost Estimating Worksheet Federal Information Agency Cost Worksheet Different Approaches to Measure the Financial Payoff of Learning Defining the Benefit Cost Ratio Defining the Return on Investment Defining the Payback Period ROI Target Options A Rational Approach to ROI The Journey to Increased Accountability Cautions When Using ROI Guiding Principles Typical Intangible Measures Linked with Programs Identification of Intangible Measures: Timing and Source Issues with Intangibles Reporting Intangibles Program Costs Module 8 Reporting Results Communication Challenges

9 Communication Principles Audience Selection Questions Common Target Audiences Complete Report The Impact Study Serves Several Purposes: Select Media Impact Study Outline Communication with Senior Management Presentation Sequence Communication Progression ROI Impact Study: One-Page Summary Brief Reports Electronic Reporting Mass Publications Building a Macro Scorecard Seven Categories of Data Potential Reporting First University Use of Evaluation Data Delivering Bad News Analyze the Results of Communication Matching Exercise: Reporting Results Module 9 Forecasting ROI ROI Possibilities ROI at Different Levels Pre-Program Forecast ROI Model Retail Merchandise Company Steps for Pre-Program ROI Forecast Sensitivity Analysis for Expected ROI Values Input to Forecast Forecasting ROI from a Pilot Program Level 1 Measures Important Questions to Ask on Feedback Questionnaires ROI with Level 1 Data Retail Merchandise Company Level 2 Evaluation ROI with Level 2 Data Relationship Between Test Scores and Performance Retail Merchandise Company Retail ROI with Level 3 Data Advantages of Forecasting Forecasting Realities Module 10 Implementing ROI Barriers to ROI Use and Implementation

10 Overcoming the Barriers Implementation Issues Typical Barriers Building Blocks to Overcome Resistance Assessing the Climate for Results Identifying Champions Measurement and Evaluation Implementation Project Plan Responsibilities for Champions Responsibilities for Team Members Getting Team members Involved Participant Responsibilities Cost-Saving Approaches to ROI Suggested Evaluation Targets Worksheet Project/Program Selection Criteria Criteria for Selecting Programs for Levels 4 and 5 Evaluation Evaluation Procedures and Guidelines Management Influence Why Managers Don t Support Your Programs Management Action Target Group Scope Payoff CEO Commitment Checklist CEO Commitment Checklist Interpretation of Results Ten Commitments for Top Executives Why Programs Do Not Work The Transfer of Success to the Job Ideal Management Support Ideal Reinforcement Levels of Management Support Concerns About HR From a Key Manager Managers Workshop Objectives Steps to Develop a Partnership Partnering Principles Annual HR Review Agenda Action Plan Matching Exercise: Implementation Strategies Module 11 Case Studies International Car Rental The ROI Fact Sheet Daily Feedback Evaluation Questionnaire ROI Quiz

11

12 Module 1 Setting the Stage 1.1

13 1.2

14 The intent of this program is to: Program Objectives Reaction Objectives Provide participants knowledge and skills that are: Relevant to their job Important to their current job success Immediately applicable New to their understanding of accountability Relevant to their colleagues in similar job situation Learning Objectives Enable participants to: Describe the five critical components of a successful evaluation practice Describe the five levels of evaluation Describe the six types of data in the chain of impact Describe the ten steps in the ROI Methodology Follow the 12 guiding principles Plan and execute an ROI evaluation project Calculate and explain the difference in the benefit-cost ratio (BCR) and the return on investment (ROI) Communicate the results of an ROI study to a variety of stakeholders Implement the ROI Methodology within their organization Application Objectives Support participants as they: Build support for the ROI Methodology in their organization Complete their initial ROI evaluation project Plan and implement future ROI projects Revise/update internal evaluation strategy/practice Brief/teach other in the ROI Methodology Change the way they propose, implement, and evaluate programs, processes and initiatives Impact Objectives Enable participants to realize positive consequences as a result of applying what they learn such as: Improving program effectiveness Improving program efficiencies Expanding successful programs Redesigning or discontinuing ineffective programs Improving relationships with clients and executives Enhancing the influence of their function within the organization 1.3

15 Setting the Stage The ROI Methodology Planning Evaluation Collecting Data Isolating the Effects of the Program Converting Data to Money Tabulating Costs and Calculating ROI Reporting Results Forecasting ROI Implementing ROI 1.4

16 Program Success will be Measured by: Ratings achieved on end of course evaluation Increase in knowledge gain as reported on end-of course evaluation Demonstration of knowledge through: - Course exercises - Case study presentations - ROI project plan presentation - ROI implementation plan ROI project completion following ROI Methodology steps and guiding principles - Evaluation planning - Data collection - Data analysis - Report submittal Steps toward implementing (beyond ROI project) completed as planned. Technical Expert Consultant Problem Solver Initiator Designer Developer Coordinator Roles of the ROI Implementation Leader Cheerleader Communicator Process Monitor Planner Analyst Interpreter Teacher Skill Areas for Certification Planning for ROI calculations Collecting evaluation data Isolating the effects of solutions Converting data to monetary values Monitoring program costs Analyzing data including calculating the ROI Presenting evaluation data Implementing the ROI process Providing internal consulting on ROI Teaching others the ROI process 1.5

17 Certification Projects Item Due Date Case Study Presentation During Workshop Implementation Plan for the ROI Process End of Workshop ROI Project Plan End of Workshop Implementation complete 3-6 Months ROI project complete 6 Months Ideally Case Study Presentation Team based assignment Present results to executive audience (you own the study) Q and A session Critique the case study (you do not own the study) ROI Project Based on a planned or anticipated ROI impact study Individual or team based Provide a copy of Data Collection Plan during workshop Provide a copy of ROI Analysis Plan during workshop Ask for input from the group Ideally complete within 6 months Specific Motivational Achievable Realistic Time based Implementation Plan Requirement Must be within your control! 1.6

18 Projects No model or theory will be successful if it is not implemented properly. Three projects are planned during the ROI certification workshop that are designed to enhance the likelihood of the methodology being implemented and utilized in the organization. The first project, a case presentation, is a simulation of an ROI impact study presentation to a senior management team. The second project is an individual implementation plan to make evaluation work in the organization. The third project is a planned ROI study that should be completed after the certification workshop. Case Study Presentation Although face-to-face meetings and a formal presentation will not be the routine way to communicate the results from detailed studies, the first one or two studies should use this approach for distributing the data. The objectives of this meeting are threefold: 1. Explain the methodology and assumptions used in the study and secure buy-in. 2. Explain the results from the study and address any issues or questions concerning the data. 3. Drive improvement or changes suggested from the data. Essentially, the case study is a modified version of original impact study developed for a specific program. Certification participants will be divided into small teams. Each team is assigned a case study for analysis and presentation. Team members will assume the role of the consultants who developed the study. Each team should present the findings to the remainder of participants, who serve as the senior management team. The presentations contain two parts: Part A Study Results (15-20 minutes). Show the results obtained in the study using a format similar to the following: Background Reasons for the ROI analysis (why ROI?) Results at each level Relate the study to the parts of the ROI Process model Key issues in the original study Conclusions Questions and discussion 1.7

19 Part B Study Critique (5 minutes). Critique the study in terms of weaknesses and concerns. Use the ROI process model or guiding principles as a standard. What would you have done differently? Presentation Methodology. Be creative. Use flip charts, PowerPoint slides, overhead transparencies, or other media. Role of the Audience: Critique the data presented by each team from the perspective of a senior manager. Ask questions. Seek clarification. Bring up issues that would be of concern to senior management. ROI Implementation Plan The second project involves the individual implementation plan. It is important for each individual pursuing certification to implement parts of the process. There are many actions that an individual can take to help increase accountability to programs, projects, and processes. Each workshop participant should select a few items to complete in the next three to six months to help introduce the methodology to the organization and make it a routine process. It is important to avoid committing to too much or too little and to pursue only those actions that are under the control of the participant. Workshop participants should develop an implementation action plan. Using the form on the next page (or a similar form), participants should indicate what they will do to implement the ROI Process within their area of influence and with specific dates. The action plan may cover any of the following general areas: Establishing responsibilities Developing skills/knowledge with ROI Creating an implementation plan Completing ROI studies Preparing/revising evaluation policy/procedures/guidelines Briefing managers on the ROI Process Communicating progress/results Tapping into a network Building support for evaluation Changing perceptions about ROI Report progress in 4-6 months. Indicate what was planned and what was accomplished, with comments. ROI Study The third project is an ROI impact study. You should have an impact study in mind as you attend the certification workshop. Two detailed planning documents are completed during the certification workshop: a data collection plan and an ROI analysis plan. These completed forms are presented to the group on the 1.8

20 last day of the workshop. Following the presentation, the group will provide input to aid in fine tuning these plans. It is anticipated that each participant will complete the study as an important part of the certification. Using flip charts, overhead transparencies, PowerPoint slides, or other media, participants will present the following: Brief description of the program Drivers for ROI calculations (why ROI?) Data collection plan with emphasis on Level 3 and 4, focusing on issues where input is needed. ROI analysis plan with emphasis on methods to isolate the effects of the program and convert data to monetary values. Please distribute a copy of your plans to each participant. Role of the Audience: The audience should provide input/suggestions to enhance/improve the data collection and ROI analysis plans. This is an opportunity to practice suggested structure for providing feedback. 1. My thoughts for improving data collection are My thoughts for isolating the effects of your program are My thoughts for converting data are My biggest concern is My overall reaction is... Blank copies of both plans are included as well as completed samples. Your completed study should be sent to the ROI Institute to complete the certification process. Summary As a summary, the following table shows what must be accomplished with these three projects during and after the certification workshop. Workshop During Workshop 3-6 Months After Impact Study Presentation X Conducted Implementation Plan Develop Plans Implemented ROI Impact Study Draft Plans Complete Completed 1.9

21 Data Collection Plan Evaluation Purpose: Program: Responsibility: Date: Level 1 Broad Program Objective(s) REACTION AND PLANNED ACTION Measures Data Collection Method/Instruments Data Sources Timing Responsibilities 2 LEARNING AND CONFIDENCE 3 APPLICATION AND IMPLEMENTATION 4 BUSINESS IMPACT 5 ROI Baseline Data: Comments: 1.10

22 ROI Analysis Plan Program: Responsibility: Date: Data Items (Usually Level 4) Methods for Isolating the Effects of the Program/ Process Methods of Converting Data to Monetary Values Cost Categories Intangible Benefits Communication Targets for Final Report Other Influences/ Issues During Application Comments 1.11

23 The ROI Report Matrix Criteria for Certification When Implementing an Impact Study.The primary criteria used to evaluate the quality and acceptance of an impact study is the ROI Report Matrix. The ROI Report Matrix was developed to rate the evaluation methodologies and reporting of ROIImpact studies. The document was designed to assess studies on 30 relevant items that are encompassed by six categories. The instrument is available on the ROI Website along with other ROI tools. Four of the six categories of the Matrix are the major components of The ROI Process which provide the structure and approach for planning, conducting, analyzing, and reporting ROI/Impact studies: a) Evaluation Planning b) Data Collection c) Data Analysis d) Reporting (The Study Report must be submitted to a designated ROI Institute Certified coach.) The two other categories assess a study in terms of: a) General Information, b) Additional Factors to be addressed in a strong Study Report. Specific criteria are assigned to each of the 30 items so that they may be rated on the common scale shown below: Not at All/Not Included = 0 points given Weak = 1 point given Satisfactory = 2 points given Exceeds Requirements/Expectations = 3 points given Total overall score possible for the 30 items is 90 points. Use of the common scale allows the 30 items to be compared on designated Criteria with weighted scores and the six categories to be compared on averages. While utilizing the Matrix can be subjective, it does provide a useful structure to the review process. 1.12

24 ROI Report Matrix Name: Study under review: Date: Study Report Categories and Items Criteria Not at All Weak Sat Exceeds Notes General Information 1. Executive Summary Included? 2. Objectives of Study Clearly Stated? 3. Background Outlined with clarity? Evaluation Planning 4. Data collection and ROI analysis plan included Included? 5. Methodology for Impact Study Clearly Defined? 6. Levels of Evaluation Outlined and explained? 7. ROI Process Described/Illustrated? Data Collection 8. Response Profile Clearly defined data? 9. Satisfaction and planned action Provided for Reference? List method(s) used 10. Learning Data provided? List method(s) used 11. Application Data provided? List method(s) used 12. Business Impact Data provided? List method(s) used 13. General Information Provided for Reference? Data Analysis 14. Isolating the Effects of the Program Credibility of method(s) used? List method(s) used 15. Converting Data to Monetary Value Credibility of method(s) used? List method(s) used 16. Cost of the Program Fully Loaded? 17. Assumptions (Guiding Principles/ROI Explained for Reference? List assumptions and Guiding Principles followed Interpretation) 18. Results (L-1 and L-5) Credible Data provided? 19. ROI Calculation and Guiding Principles applied Credible Data provided? List assumptions and Guiding Principles followed 20. Intangible Benefits Data listed and detailed? 21. Barriers to Application in work setting Data provided? 22. Enablers of Application in work setting Data provided? Reporting 23. Conclusions Convincing support provided? 24. Recommendations Convincing support provided? 25. Appendices and Exhibits Provided for Reference Additional Factors Scoring for the Additional Factors Category 26. Did the study achieve the level of measurement it was designed to achieve? Score a 3 if it met measurement target, 0, 1, or 2 if it did not 27. Was the planning for this study adequate? Score 0, 1, 2, or 3 based on strength of the planning 28. Did the researcher provide clear linkage between the project and the results Score 0, 1, 2, or 3 based on strength of the linkage reported? 29. Overall appropriateness and quality of instruments used? Score 0, 1, 2, or 3 based on strength of instruments 30. How could this study be improved? Score 0, 1, 2, or 3 based on overall strength of study Total Score each column Total Overall Score, 1 30 Use this document to score your projects, have members of your certification cohort provide feedback, and obtain feedback from your ROI Institute coach. In order to receive Certification in the ROI Methodology, items 1 through 25 must be included in the study. 1.13

25 ROI Implementation Action Plan Program: Responsibility: Date: Action Steps Expected Consequences Date Issues/Comments Comments: 1.14

26 Global Communications* Sydney Mitchell has been serving as CEO for Global Communications for the past nine months. She has a reputation for being aggressive in meeting goals, yet is pragmatic and fair. In her previous organization, Sydney increased profits as well as customer satisfaction ratings while reducing staff and positioning the company as one of the best 100 companies to work for. Before making significant changes in Global Communications organization structure, Sydney is giving each function the first year to make strides toward meeting strategic objectives. These strategic objectives focus on increasing profits, market share, customer satisfaction ratings, and employee satisfaction ratings. Sydney has communicated these objectives very clearly throughout her first nine months and even holds monthly learning sessions with employees to help them understand the meaning and importance of each objective. With three months remaining in the year, Sydney has been meeting with executives of each function to get a status report. She has been relatively pleased with the results in the marketing, human resources, and distribution functions. Today she is meeting with the President of the GlobalCom University, Global Communications corporate university. Donald Hodges is the President of GlobalCom University. He was hand-picked by the past CEO and believes the university is making a difference. He always receives rave reviews from participants after each program. Donald is ready for Sydney. He has a very flashy PowerPoint presentation and the results of all of his program evaluations are ready to review. Sydney enters the room. Sydney: Hi, Donald. It s nice to see you. The place looks great and everyone seems really busy. Donald: Yes, Sydney. We re developing 12 new programs. Sydney: Really? What are these programs? Donald: Well, we re developing a new communications program as well as revising our orientation program to include our new benefits package. We ve also had requests from employees to offer programs they re interested in, including a dress-for-success program, a time management program, and a business etiquette program. And we re developing a leadership program similar to one I attended recently and really enjoyed. I think the managers will enjoy it as well. Sydney: Hmmmmm. How much time does it take to develop these programs? Donald: Oh, not long; about a week for each day of training at the most. We have our four program developers working on three programs each. I estimate it will take a few months to develop all 12 programs. Sydney: I see. A few months. Donald: Come on in the conference room, Sydney. I want to share our accomplishments thus far! * Source: Phillips, Patti P, Bottomline on ROI, CEP Press, Atlanta, GA,

27 Sydney: Great, I d really like to see. Donald boots up the presentation. He goes through all the preliminary issues, and then gets to the results of the past nine months. Donald: In the past nine months we have developed 10 new programs, offered 750 hours of training, had 1,500 employees attend training, and received an average of 4.5 out of 5 on the program satisfaction rating. So basically, we have developed new training, offered some of the new programs as well as some of the old favorites, and the employees attending training seem to think we re moving in the right direction. Sydney: Well, I see you ve been busy. Thanks for the update, Donald. Do we know about the success of these programs on the job? Donald: No, not specifically, but we are confident that they are adding value. Sydney: How do you know you re adding value? Donald: Because of the feedback we receive. Sydney: What kind of feedback do you receive? Donald: Many of the participants tell us that they have been very successful with what they have learned. Sydney: So, you ve actually had a follow-up of each program? Donald: No, not exactly. We just receive random comments. Sydney: So you have no organized way of knowing about the success of your programs? Donald: Well, it s not a formal follow-up, but we still receive good feedback. Sydney: I see. Well, thanks, Donald. I d like to meet with you next Monday to discuss your contribution to the organization. Questions for discussion: 1. From Donald s perspective, how do you think the meeting went? 2. From Sydney s perspective, how do you think the meeting went? 3. What is the fate of GlobalCom University? 1.16

28 Activity Based Characterized by: 1. no business need for the program 2. no assessment of performance issues 3. no specific measurable objectives 4. no effort to prepare program participants to achieve results 5. no effort to prepare the work environment to support program 6. no efforts to build partnerships with key managers 7. no measurement of results or cost benefit analysis 8. reporting on programs is input focused Paradigm Shift in Programs Results Based Characterized by: 1. program linked to specific business needs 2. assessment of performance effectiveness 3. specific objectives for application and business impact 4. results expectations communicated to participants 5. environment prepared to support program 6. partnerships established with key managers and clients 7. measurement of results or cost benefit analysis (ROI) 8. reporting on programs is output focused Definition of Results-Based Programs Programs are initiated, developed, and delivered with the end in mind. A comprehensive measurement and evaluation system is in place for each program. Impact and ROI evaluations are regularly developed. Program participants understand their responsibility to obtain results with programs. Support groups (management, immediate support, staff support, etc.) help to achieve results. How results-based are your programs? Take the assessment entitled How Results-Based Are Your Programs? When taking this assessment, try to be candid in selecting the appropriate response. Score your assessment using the guidelines provided. Compare your scores with others. What is considered to be an adequate score? What are the potential uses of this survey? 1.17

29 Traditional View Human capital expenses are considered costs. The HR function is perceived as a support staff. HR is involved in setting the HR budget. Human capital metrics focus on cost and activities. Human capital metrics are created and maintained by HR alone. There is little effort to understand the return on investment in human capital. Human capital measurement focuses on the data at hand. Human capital measurement is based on what GE and IBM are measuring. HR programs are initiated without a business need connected to them. Overall reporting on human capital programs and projects is input-focused. Questions for discussion: Human Capital Perspectives 1. Has this shift occurred in your organization? Explain. Emerging View Human capital expenditures are viewed as a source of value. The HR function is perceived as a strategic partner. Top executives are increasingly involved in allocating HR budget. Human capital metrics focus on results. Top executives are involved in the design and use of metrics. The use of ROI has become an important tool to understand the cause-and-effect relationships. Human capital measurement focuses on the data needed. Human capital measurement is based on what is needed in the organization. HR programs are linked to specific business needs before implementation. Overall reporting on human capital programs and projects is output-focused. 2. What must happen for this shift to occur? 3. What are the implications of this shift? 1.18

30 Increased Interest in the Value of Human Capital is Driven by: The increasing cost of human capital Consequences of improper or ineffective HR practices Linkage of human capital to strategic initiatives Increased accountability of all functions Top executive requirement for HR contribution, and human capital ROI Balanced Scorecard Economic value added Benefits vs. costs Strategic accountability Value based PROFITABILITY ROI Vital Signs Performance standards Effectiveness Bottomline contribution Shareholder value Evaluation Program impact The Measurement Landscape is confusing Three Journeys Take Us to the Use of ROI 1. The need to change the HR measurement mix 2. Setting the investment level for human capital 3. Valuing human capital Each is explored next! 1.19

31 Apex, Inc.* John Hamilton, Senior Vice-President of Human Resources at Apex Inc., a large business products firm, was perplexed and a little confused after attending a meeting with the Chief Executive Officer, Rob Peterson. The CEO had asked John to develop a plan to show the senior executive team the contributions of the human resource function and the value of the company s human capital. Although the CEO fully supports the majority of HR programs and services, he questioned the value of a few HR initiatives and was concerned about comments from the senior team. As Peterson said, Many of our senior executives are questioning the actual value of the human resource function and suggesting it could be out-sourced or parts of it eliminated. What they need to see is a more direct link between these programs and the bottom line. In addition, Peterson asked about the total amount of the company s investment in human capital, with a follow-up question, Is it too much or too little? As John reflected over the meeting, he was aware of the excellent HR programs that have been created. Some have even won awards from professional associations. He knew that the senior team places value on employee feedback, taken annually, which provides an assessment of job satisfaction, motivation, and commitment from the employees. This process has provided valuable information to help plan HR initiatives. John is also concerned that Peterson had mentioned return on investment in his conversation. John was aware of the ROI concept but has not seen it applied to the human resource function and wondered if it was even feasible. These concerns left John confused as he began to tackle the assignment. He must present his recommended plan for showing the value of human capital and the contribution of the human resource function to the senior management team at the next senior staff meeting. He knew he must have specific approaches to show the team to get their buy-in. Otherwise, he would have a tough challenge ahead as he entered the budgeting cycle. Questions for discussion: 1. What range of possibilities exists for John? 2. What specific types of measures would you suggest for an overall measurement scheme? 3. How can John win the support from the senior team? * Source: Phillips, Jack J. and Phillips, Patti P., Proving the Value of HR: How and why to Measure ROI. Society for Human Resource Management, Alexandria, VA

32 Comparison of Approaches to Measure the HR Contribution Relative Relative Value of Approach Measurement Focus Cost Information Feedback surveys Attitudes/Perceptions Moderate Moderate HR satisfaction surveys Human capital measures HR Auditing HR case studies HR cost monitoring Competitive HR benchmarking Key HR indicators Attitudes/Perceptions Moderate Moderate Value of skills, capabilities, performance of employees Efficiency/ Existence of practices Qualitative description with data Program/Function/Activity costs Performance measures/ practices Program/Function performance measures HR macro studies Multiple key indicators overall function HR management by objectives HR profit centers Goal setting for HR performance measures Profit contribution of programs /services High Low Low Low High Moderate High Low High Moderate Low Low Low High Moderate High Moderate High ROI methodology Benefits vs. Costs High High Balanced scorecard Balanced measures customer, financial, process Questions for discussion: 1. Which approaches are you using now? Moderate Moderate 2. Which approaches are you planning to use in the future? 3. Which approaches are you planning to discontinue? 1.21

33 Measuring the HR Contribution: Status Approach Currently Using Will Use in the Near Future 1-2 Years Will Use Long Range 3-5 Years Comments HR auditing HR case studies Feedback surveys HR management by objectives Competitive HR benchmarking HR cost monitoring Key HR indicators HR satisfaction surveys HR profit centers HR macro studies ROI methodology Balanced scorecard Human capital measures 1.22

34 HR Accountability Progress HR Accountability Early Approaches Solid Value- Added Approaches Leading Edge Approaches ROI Methodology HR Profit Center Balanced Scorecard HR Macro Studies Human Capital Measurement Competitive HR Benchmarking HR Satisfactory Surveys HR Cost Monitoring HR Key Indicators HR Auditing HR Case Studies Feedback Surveys MBO in Personnel 1960 s 1970 s 1980 s 1990 s 2000 Leading Edge Approaches to Measuring the HR Contribution Balanced Scorecards HR Profit Center Human Capital Measures HR Macro Studies ROI Methodology Most promise as an immediate tool Recommendations for Measurement Categories Select an approach in each of these categories: Attitudinal Data Comparative Data Human Capital Measures Benefit/Cost Analysis (ROI) 1.23

35 1. Innovation and Creativity Innovation Creativity 2. Employee Attitudes Employee satisfaction Organizational commitment Employee engagement 3. Workforce Stability Turnover and termination Tenure and longevity 4. Employee Capability Experience Learning Knowledge Competencies Educational Level 5. Human Capital Investment HR department investment Total human capital investment Investment by category 6. Leadership 360 feedback Leadership inventories Leadership perception 7. Productivity Unit productivity Gross productivity 8. Workforce Profile Demographics 9. Job Creation and Recruitment Job growth Recruitment sourcing and effectiveness Recruiting efficiency 10. Compensation and Benefits Compensation Employee benefits Variable compensation Employee ownership 11. Compliance and Safety Complaints and grievances 12. Employee Relations Absenteeism and tardiness Common Human Capital Measures Work/life balance 1.24

36 Charges and litigation Health and safety Questions for discussion: 1. How do these measures compare with your HR measures? 2. How would you measure the following? Innovation Learning Leadership Gross Productivity Setting the Investment Level 1 Let Others Do It! Motivating Forces Cost control Lack of infrastructure Instability Access to expertise Short-term focus Survival Approaches 1. Hire fully competent employees 2. Use contract employees 3. Outsource major functions 1.25

37 2 Invest the Minimum Motivating Forces Low cost industry High labor use Strong competition Employees are dispensable Approaches 1. Pay minimum wages 2. Provide few benefits 3. Keep training simple 4. Expect turnover and address it Human Resources Development Issues Focus Costs per Employee Time for Payback Risk for Payback Training Job-Related skills Low Short Low Education Development Preparation for the next job Cultural change and continuous learning Moderate Medium Moderate High Long High 1.26

38 3 Invest with the Rest! Motivating Forces Desire to have best practices Benchmarking is acceptable Benchmarking is used in all parts of organization Benchmarking can be low cost Approaches 1. Locate existing reports 2. Participate in existing projects 3. Create a custom project 4. Search the literature Benchmarking is low risk Human Capital Investment Benchmarks 1. Human Resource Expenses (HR Department Costs/Budget) a. As a percent of operating costs b. As a percent of revenue c. Per employee 2. Total Investment in Human Capital (Total HR expenses plus all salaries and benefits of non-hr staff) a. As a percent of operating costs b. As a percent of revenue c. Per employee 3. HR Expenses by Function 1.27

39 a. Recruiting and selection cost as a percent of total HR b. Recruiting and selection cost per new employee hired c. Training/Learning/Development costs as a percent of total HR d. Training/Learning/Development costs per employee e. Training/Learning/Development costs as a percent of compensation f. Compensation costs as a percent of total HR g. Compensation costs as a percent of operating expenses h. Compensation costs per employee i. Benefits costs as a percent of total HR j. Benefits costs as a percent of operating expenses k. Benefits costs as a percent of compensation l. Benefits costs per employee m. Employee relations costs as a percent of total HR n. Employee relations costs per employee o. Compliance and fair employment costs as a percent of total HR p. Compliance and fair employment costs per employee 4. HR Expenses by Process/Programming a. Analysis and assessment costs as a percent of total HR b. Design and development costs as a percent of total HR c. Implementation and delivery costs as a percent of total HR d. Operations and maintenance costs as a percent of total HR e. Measurement and evaluation costs as a percent of total HR 5. Selected HR Costs a. Turnover cost per employee leaving b. Turnover cost as a percent of compensation c. Accident cost per incident d. Safety cost per employee e. Absenteeism cost per absence f. Absenteeism cost as a percent of average wage rate g. Health care cost per employee h. Health care cost as a percent of total benefits 1.28

40 1 Determining What to Benchmark 7 Initiating Improvement from Benchmarking 2 Building the Benchmarking Team 6 Distributing Information to Benchmarking Partners Phases of the Benchmarking Process 3 Identifying Benchmark Partners 5 Analyzing the Data 4 Collecting Benchmarking Data 4 Invest until it Hurts! Motivating Forces Fad chasing Happy employee dilemma Quick fixes Retention concerns Competitive strategy Union demands We can afford it! Approaches 1. Pay above-market wages 2. Provide abovemarket employee benefits 3. Implement most new fads/ programs 4. Provide all types of employee services The Relationship Between Investing and Performance Financial Performance Under-Investing Optimal Over-Investing Investment in Human Capital 1.29

41 5 Invest as long as there s a Motivating Forces Need to show HR contribution Increasing cost of human capital Secure funding Business partner Improve processes Approaches 1. Measure success of each HR program 2. Collect up to six types of data 3. Use ROI routinely 4. Involve stakeholders payoff! 5. Use the data The ROI Methodology Capture Costs Of Solution Data Collection Data Analysis Reporting Evaluation Planning LEVEL 1: Reaction and Planned Action LEVEL 3: Application and Implementation Develop/ Review Objectives Of Solution Develop Evaluation Plans and Baseline Data Collect Data During Solution Implementation LEVEL 2: Learning and Confidence Collect Data After Solution Implementation LEVEL 4: Business Impact Isolate the Effects of Solution Convert Data to Monetary Value Calculate the Return On Investment LEVEL 5: ROI Generate Impact Study Identify Intangibles Intangible Benefits 1.30

42 The Development of ROI has been methodical Training and Learning Organization Development HR Programs Change Initiatives Technology Implementation Quality / Six Sigma Meetings and Events Coaching Valuing Human Capital Three Approaches 1. What we know from Logic and Intuition Automation has limitations People are necessary Stock market mystery Accounting dilemma Last source of competitive advantage Superstar Phenomena 2. What we know from Macro Level Research HR Effectiveness Index Gallup Studies The Service Profit Chain Watson-Wyatt Studies Deloitte &Touche Studies.... and many others 3. What we know from ROI Analysis Micro Analysis Tool 3-5,000 studies per year 41 Countries / 25 Languages Variety of Applications (2,500) ROI Networks ROI Standards ROI Best Practices Superstar Characteristics People are the difference Good and great Great places to work Most admired companies Valuing Human Capital The Complete Picture Micro Analysis (ROI Studies) Macro Analysis (Relationships) Logic & Intuition (Intangibles) 1.31

43 Ideas, Thoughts, and Planned Actions: 1.32

44 Module 2 The ROI Methodology 2.1

45 2.2

46 Reliance Insurance Company At the end of a monthly staff meeting, Frank Thomas, CEO of Reliance Insurance Company, asked Marge Thompson, Manager of Training and Development, about the Communications Workshops that had been conducted with all supervisors and managers throughout the company. The workshop featured the Myers-Briggs Type Indicator (MBTI) and showed participants how to interact with, and understand, each other in their routine activities. The MBTI classifies people into one of 16 personality types. Frank continued, I found the workshop very interesting and intriguing. I can identify with my particular personality type, but I m curious what specific value these workshops have brought to the company. Do you have any way of showing the results of all 25 workshops? Marge quickly replied, We certainly have improved teamwork and communications throughout the company. I hear people make comments about how useful the process has been to them personally. Frank added, Do we have anything more precise? Also, do you know how much we spent on these workshops? Marge quickly responded by saying, I m not sure that we have any precise data and I m not sure exactly how much money we spent, but I can certainly find out. Frank concluded with some encouragement, Any specifics would be helpful. Please understand that I am not opposing this learning effort. However, when we initiate these types of programs, we need to make sure that they are adding value to the company s bottom line. Let me know your thoughts on this issue in about two weeks. Marge was a little concerned about this CEO s comments, particularly since he enjoyed the workshop and had made several positive comments about it. Why was he questioning the effectiveness of it? Why was he concerned about the costs? These questions began to frustrate Marge as she reflected over the year and a half period in which every manager and supervisor had attended the workshop. She recalled how she was first introduced to the MBTI. She attended a workshop conducted by a friend, was impressed with the instrument, and found it to be helpful as she learned more about her own personality type. Marge thought the process would be useful to Reliance managers and asked the consultant to conduct a session internally. With favorable reaction, she decided to try it with the top executives, including Frank Thomas. Their reaction was favorable. Then she launched it with the entire staff and the feedback was excellent. She realized that the workshops were expensive because over 600 managers had attended. However, she thought that teamwork has improved, but there is no way of knowing for sure. With some types of learning you never know if it works, she thought. Still, Marge was facing a dilemma. Should she respond to the CEO or just ignore the issue? 2.3

47 Questions for discussion: 1. Is this situation typical? Explain. 2. What are the basic issues in this case? 3. How could this problem be avoided in the future? 4. What should Marge do? Be specific. 2.4

48 Matching Evaluation Levels with Objectives Instructions: For each objective listed below, indicate the level of evaluation at which the objective is aimed. o Level 1: Reaction o Level 2: Learning o Level 3: Application o Level 4: Business Impact o Level 5: Return on Investment Objective After completing this program or project, participants should: 1. Decrease citizen complaints by 20% in one year. 2. Use problem-solving skills to uncover product defect causes. 3. Be able to demonstrate the five steps to diffuse a conflict, given two individuals in a heated argument. 4. Rate the facilitator 4 out of 5 on presentation skills. 5. Decrease the amount of time required to develop a proposal. 6. Achieve a 20% ROI on year after program implementation. 7. Perceived the content to be relevant to job situations (4.5 out of 5). 8. Decrease security breaches by 25% in six months. 9. Conduct a proper investigation using the seven-step process in 95% of complaint situations. 10. Score an average of 75 or better on new strategy quiz. 11. Conduct a meeting with direct reports to explain the new ethics policy. 12. Use all ten negotiation skills in at least 50% of negotiation situations. Evaluation Level How do Level 3 and Level 4 objectives provide benefit? To whom? 2.5