Today s Session: Today s Sponsors: Measuring Learning Effectiveness with Integrated Analytics

Transcription

1 Today s Session: Today s Sponsors: Measuring Learning Effectiveness with Integrated Analytics March 20, 2012

2 Tips for the Webinar: Tweeting? Please use this tag: #IntegratedAnalytics (800) or (805)

3 Today s Speaker Kim Lennon Senior Product Manager, SumTotal Systems Leads reporting & analytics across the SumTotal Human Capital Management suite. 15 years experience building and analyzing customer business data warehouses Interfaces with analysts, customers, prospects and partners Spearheads product design and product roadmaps relating to reporting & analytics

4 Measuring Learning Effectiveness with Integrated Analytics Presented by: Kim Lennon Sr. Product Manager

5 Agenda Why Analytics - Why Now Key Performance Indicators Power in Numbers Drive the Story Home Improve Your Workforce Today Tool-Talk About SumTotal Discussion Page 5 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

6 Page 6 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

7 Top 5 Learning & Development / Business Priorities Entering 2012 Budget Need to increase productivity, reduce costs and still provide superior service Aligning operations, practices and systems across different business units Aligning L&D with business goals Attracting, engaging, and retaining employees with critical skills and competencies IDC, 2011 Page 7 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

8 Typical Pain Points Without Analytics CEO How the business is being impacted based on talent and hr initiatives Key player in business strategy showing how silos of information can be brought together effectively to increase revenue (reduce costs) SVP of HR / Chief Learning Officer Interested in efficiency, effectiveness, resource utilization and on-time delivery Interested in visibility and actions on people behind mandatory compliance training? What is ROI and business impact of learning initiatives? Line of Business Training Managers Ability to support the demand of business, having to support a large number of employees with a small staff Interested in how to increase effectiveness of his organization with proper trainings, competency gaps and development plans to help increase revenue, and whether or not there are any risks that need immediate attention? Are teams following HR practices and policies and completing on time? Create proactively talent strategy and pay for performance culture Page 8 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

9 Who Needs What Audience Decisions They Make Analytics Needs Executives Drive compliance Ensure training is efficient Make sure that employees are developed Overall metrics, financial information, compliance and training efficiency. Line of Business Managers Training Managers Drive compliance Develop people Ensure training costs are reasonable Ensure that people are completing and learning something What courses (type) should be offered Which audiences consume the most/learn the best Which vendors content performs well What should we stop doing Training compliance, skills development to meet job needs and completion of mandatory programs. Training volumes, completion rates, vendor effectiveness, facility efficiency, program effectiveness, overall costs and financial efficiency. Page 9 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

10 Poll #1: What is the most important learning measure for your management? 1. Consumption who is taking what? 2. Efficiency how much are we spending/saving? 3. Satisfaction are learners getting what they want? 4. Impact are we driving business results? Page 10 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

12 Decide How You Will Use the Data What is the question you are answering by building a scorecard, dashboard, or report? What decisions or actions do you hope to take based on data analysis findings and recommendations? Has this been done before? Page 12 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

13 Use Data to Drive Decision Making Make data business relevant Use formats that resonate Link your measures Take an iterative approach Tell a story Page 13 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

14 Why Measure Performance? Control & Monitor People Report Externally & Demonstrate Compliance Learn & Improve Page 14 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

15 KPI Checklist 1. Which strategic objective is this indicator relating to? 2. What question do you want to have an answer to? What are your information needs? 3. Who is asking this question? Who is the information customer? 4. What will they do with the information? Why are they asking? Additional considerations: What is the data collection method? What is the source of the data? What is the formula / scale / assessment method? How often, when and for how long do you collect the data? Who collects the data? What is the target or performance threshold(s)? What are the costs for collecting the data? Justified? Page 15 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

16 Actionable Example An organization has an objective to retain talent and has clearly defined what constitutes talent and has an agreed common enterprise-wide metric, and the measures shows that strategically critical employees are walking out the door, then this should trigger an intervention. Page 16 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

17 Individual Program KPI $ Average value of trainings delivered % Contribution to profit % Contribution to revenue # Training days per months # Participants per training % Customer satisfaction or retention rate # New customer inquiries received % Improved customer service level due to training % Training schedule adherence # Training delay # Time to prepare a training % Chargeable ratio Page 17 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

18 Organizational KPI % Training Penetration Rate (enrollment) % Certification Rate $ Expense Reduction $ Total Training Spending as % of Company Revenue % of HR budget spent on Training $ Average training costs per employee $ Expense Reduction % Training program that match organizational requirements $ Total Training Spending as % of Company Revenue Page 18 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

19 Compliance KPI % Completion rates for mandatory programs % Certification acquisition rates % Certified in each business unit % Regulatory compliance on time % of employees that have gone through training Page 19 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

20 Measures / Metrics Quantitative Fill Rates by class, course, curriculum, location, modality Min, Max, Avg. participants enrollments in class, course, program Enrolled vs. Attended ratio by class, course, program, curriculum Class cancellation rates (% cancelled) Completion rates by learning modality (% OLT offered vs. completed, % ILT offered vs. completed) Qualitative Instructor performance Courseware quality Quality of online delivery Effectiveness of on-the-job support tools Criticality of training to the job Page 20 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

21 Categorization of Training Measures Effectiveness / Alignment Are we achieving the business goals for a program? Are we having an impact on revenue and cost? Which individual programs, audiences, vendors, and media are having the highest/lowest impact? Efficiency (ROI) Which of our programs are most cost efficient? How can we reduce the cost per hour for a program? How does our organization benchmark against others? Compliance Are we meeting our compliance goals - where, where not? How do we increase compliance quickly? What is our risk of falling out of compliance? Page 21 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

22 Phillips-Kirkpatrick Model Level 5: ROI Level 4: Results Level 3: Behavior Level 2: Learning Level I: Training Reaction Page 22 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

23 Poll #2: What Level Is Your Training Strategy? 1. Training Reaction 2. Assessment 3. Behavior 4. Business Results 5. ROI Page 23 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

25 HIPAA Effectiveness Example Through training effectiveness measurements you can see how each department scored on various topic areas covered under HIPAA. For example, OR nurses may have scored a 90% on information regarding patient s rights however, the ED nurses may have only gotten a 63% score on that same information. Now that you have the information, what do you do? One possibility is remediation for staff in those departments that did not meet a certain scoring threshold. They could be directed to retake the training. Another possibility is to use your staff trainers to reinforce the topic with the specific group that scored poorly. Page 25 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

26 Manufacturing Effectiveness Example A key cost and quality objective in manufacturing is minimizing error rates during the manufacturing process. Measuring how learning management affects this objective is therefore key. With a talentbased learning system, measuring how learning management impacts this result is a simple process. Page 26 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

27 Manufacturing Effectiveness Example cont. Set an organization goal: Establish an organization-wide goal to reduce manufacturing error rate by twenty percent (20%) Cascade goal to everyone involved: Next, the organizational goal is cascaded down to every employee involved in the manufacturing process Execute training: Create and execute the training programs that are designed to reduce manufacturing error rates Setup metric that compares average test scores against goal achievement: Since all data for both goal achievement and training results are held in one system, it is easy to run a report that compares average test scores in the training program to goal achievement by employee Page 27 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

28 Efficiency Example The CEO of a company is trying to control spending and costs. She asks the Chief Learning Officer (CLO) for a training budget. That budget is prepared and submitted. The CEO wants some evidence to justify not cutting the budget this year. Reasonable business decisions dictate that tools that help the business perform better and improve the bottom line are good investments. The CLO meets with the CEO and tells her that 80% of the workforce received training in the prior year, that nearly 300 classes were taught and that anecdotal evidence (smile sheets) suggest people really liked the training. Would you approve the budget? Page 28 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

29 Efficiency Example cont. Did the training help increase revenues? Did it help decrease costs? What kind of impact did it have on the average employee s job performance relative to the salary paid to the employee? Page 29 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

31 Intelligence Through Analytics Now and Past Competetive Advantages Standard Reports Ad Hoc Reports Drill Down (Through) Alerts Statistical Analysis Forecasting Predictive Modeling Future Prescriptive Modeling Degree of Intelligence Page 31 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

32 Don t Do All the Heavy Lifting Yourself Web based / desktop based Mobile (phone / tablet) Export formats Office Integration Dashboard and scorecards Alerts and subscription delivery External data imports Database Versions Rich design environment (WYSIWYG) Drill-down & drill-through Actionable Intelligence Page 32 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

33 Analytics Tools of the Trade External applications Spreadmarts - heavily siloed Microsoft Excel workbooks Business Intelligence vendors Hardware / Software / Model Company business intelligence platform (ERP Analytics) Data warehouse supporting business areas such as finance Embedded reporting applications (HCM Analytics) Existing reporting platform in your Talent Management or Learning Management Platform Page 33 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

34 Challenges with Analytic Tools Landscape Off the Shelf Analytics Technology: Need to manage separate role structure and security policies from talent and learning applications Requires separate extracts from talent/learning systems and manual mapping/merging of data typically requiring IT resources Inability to link back to talent or learning applications for traceability and action Navigate department/division silos in order to obtain HRMS, LMS, and business data ERP Analytics Solution: Need to manage separate role structure and security policies from talent and learning applications Appending talent and HR analysis and data warehouse is still a large project because they don t have it Inability to link back to talent or learning applications for traceability and action Relatively expensive component of implementation timeline to achieve integrated vision HCM Analytics Solution: Need to manage separate role structure and security policies from talent and learning applications Generally no capability to integrate external business metrics without customization Inability to link back to talent or learning applications for traceability and action Limited breadth of data for talent and/or learning data Page 34 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

35 Strategic Workforce Analytics Talent/ Custom Portal Pre Built Key Performance Indicators (KPIs) Learning Hiring 3rd Party systems - SAP, Oracle, PeopleSoft Role Based Dashboards Performance Compensation Legacy & Custom Data Warehouse Interactive & Ad-hoc Reporting External Business Data Alerts & Actions Integrated Strategic HCM - Talent Data Warehouse Succession Career Development Flat Files (CSV, XLSX) Most comprehensive Strategic HCM-Talent Data warehouse 200+ HCM interlinked business objects across learning, performance, succession, compensation hiring and career development Rationalized data and security across these dimensions and processes Pre-built & Ad-hoc key performance indicators for most industry needs Pre-built role based dashboards and 50+ KPIs most critical to companies Flexible and quick ad-hoc KPIs and reports to enable dynamic analysis Strategic workforce analysis integrated with external business data Measure the impact of strategic HCM processes based on impact to any business drivers and data like financial, sales, customer satisfaction Easy to add any external data source and create integrated reports Actionable Intelligence for execution excellence Drill down into the detail transactional information from any analysis Drive closed loop execution and continuous improvement with integration transactional actions from reports Page 35 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

36 Page 36 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL You can't manage what you don't measure!

37 Poll # 3: When do you think your organization will deploy actionable analytics? Already using analytics to make a difference Within next six months From six to twelve months Not on the radar but should be Page 37 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

39 Comprehensive Strategic HCM Solution Best In Class and Complete Coverage for All HR Processes #1 Leader in Learning* Market Leader in Talent* Market Leader in WFM* Market Pioneer in Payroll* Learning Mgmt Talent Mgmt and Acquisition Workforce Mgmt Core HR & Payroll Most Comprehensive Learning Management Support for all types of learning elearning, OJT, Virtual learning, Trainings, Extended enterprise Manage different global divisional learning and training rules and policies Specialized SMB Maestro Learning Solution Most Comprehensive Talent Management Performance Management 360 Feedback Competency Management Career Development Integrated Learning Succession Planning Compensation Management Recruiting and Onboarding Complete Talent Profile Most Comprehensive Workforce Management Time and Attendance Tracking Absence Management Vacation Rules FLSA, FMLA and Others Scheduling Manual and Optimizations Full integrated Data Acquisition Devices with different access controls Comprehensive Payroll Ability to provide complex Payroll solution for global organization Integrated benefits management Core HR Self Service HR Transaction Management Expense Management Expense Tracking and Reimbursement *Forrester, Ventanna Research and Gartner Page 39 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

40 About SumTotal Worldwide Leader in Strategic Human Capital Management Solutions Market Share (users) SumTotal Worldwide Leader in Strategic HCM 3,500+ Customers Over 43+ Million End-Users Leader in SMB, Mid-Market & Enterprise Solutions Global Scalability Page 40 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

41 Barriers to Integration Existing HCM Investments Can your vendor can meet your current and future needs? Pure SaaS Enterprise-class Multi-tenant Rapid Upgrades & Advancements Cloud Services Private or Shared Options Massively Scalable Infrastructure Compliance Management Extensibility On-Premise Localized Deployment Oftentimes the complexities of integration between cloud and on premise systems is overlooked until it is too late. Page 41 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

43 It s your turn! Ask our expert Kim Lennon klennon@sumtotalsystems.com Free white paper from SumTotal Systems: "Using Analytics to Achieve the Greatest Value for Workforce Initiatives." Page 43 - March 19, 2012 PROPRIETARY AND CONFIDENTIAL

44 Upcoming Webinars: March 22: Successfully Marketing a Global Online Curriculum March 28: What Audiences Remember about Online Presentations April 3: Blended Learning: The On-Time Toolkit for Training April 18: Emotional Intelligence: Increasing Leadership Capacity & Performance For More Info / To Register / To Access Archive: Type Webinars into Search Box at TrainingIndustry.com Type On Demand into Search Box at TrainingIndustry.com

45

46 Thank You! On behalf of TrainingIndustry.com, thanks to: Today s Speaker: Kim Lennon Today s Sponsor: SumTotal Systems All of you for attending Questions or Comments? Please contact Tim Sosbe: tsosbe@trainingindustry.com