Presenter: Doug Reynolds, Development Dimensions International

Presenter: Doug Reynolds, Development Dimensions International

Data-Driven Talent Management: Using assessment and technology to run better organizations Presented by: Doug Reynolds, Ph.D. Senior Vice President & CTO Development Dimensions International (DDI)

Big Data: Old Wine or New Opportunity? Presented by: Doug Reynolds, Ph.D. Senior Vice President & CTO Development Dimensions International (DDI)

Big Data about people is hot!

Anatomy of a trend Reference to Moneyball Uninformed overgeneralization Pivot point

Anatomy of a trend Review of common problems

Anatomy of a trend Discovery of known knowledge Nuggets of insight from new technologies

What is Big Data for HR? A visit to the dust bowl A sea of spurious correlation High school science New insights from new tools

A few definitions Big data: Large quantity and variety of data generated through internet-based systems. Digital exhaust: trace information remaining after use of an online tool, often irrelevant to the purpose of the tool. HR analytics: organizationally relevant statistics regarding people associated with the organization

Overview Why are we talking about this now? The promise of big data in HR A few examples The answer

A few important trends Labor Market Management Technology Talent is a differentiator Aging workforce Skill gaps Automation Globalization Outsourcing Interoperability Services architecture Cloud computing

A (simplified) view of the evolution of business software

Start with an inefficient business process Input Step 1 Step 2 Step 3 Step 4 Output

design software to improve it Input Step 1 Step 2 Simplify, automate, increase availability, globalize, etc. Step 3 Step 4 Output

It doesn t take much to get started

Competitors may capture different parts of the process Company 1 Company 2

The next challenge: add value beyond your step Interconnect with other software tools to automate more of the business process

Acquire, merge, or build to own the whole process Attempt to support the whole process: Build more pieces Buy your neighbor Sell out Or, go out of business

Once you own one process, you get hungry for others Process: A B C D

Once you own one process, you get hungry for others Process: A B C D

Interconnections allow for more insight and strategic value A B C D Within Process Across Processes

In the HR context: Within Process Across Processes

Big data about people Recruit Promote Hire Manage Train

The promise of big data for talent management Strategic Impact Insight Process Automation

Candidate Source Assessment Data for Individual and Group Manager Satisfaction with Quality of Candidates Ratio Offers to Acceptance and Diversity of New Hires Candidate feedback on the hiring process Confidence of Hiring Manager in the New Hire and Confidence of the New Hire that they are in the Right Job Job Performance and Engagement Predictive Hiring Analytics Source Assessment Final Interview Job Offer First Day on the Job 6 Months on the Job 1 year on the job

A pervasive issue: assessment rigor Talent quality? 1 2 3 4 5

Examples of Assessment-driven Analytics

About DDI 40+ Years in 3 markets: Personnel Selection Executive Assessment Leadership Development 42 offices in 26 countries 4M+ tests delivered/yr 1,500 exec assessments/yr Multiple software delivery systems

Example 1: selection testing Selection test for graduate hiring Relevant and effective across cultures Strong security (difficult to cheat on) Strong predictor of performance Available anytime, anywhere Brief

Test features Figural reasoning: measure of reasoning ability, critical thinking, and problem-solving Non-verbal/graphical items No translations Applicable regardless of candidate reading level Culture-free/fair for all candidate groups Allows for comparisons across cultures/countries

Internet-based computer adaptive testing (CAT) CAT addresses several common issues: Test Security Cheating Length of Candidate Experience Items drawn from extensive bank; low item exposure rates Different combination of items for each candidate; no single key available to be used for cheating Compared to traditional tests, shorter test time but superior precision

CAT: development process Calibration Research 200,000+ candidates for entrylevel professional jobs, globally Items researched via internet delivered test forms Test timing and question functioning developed from response data Ensured the test is inclusive to all candidates globally Then, criterion-related validation study conducted

Results from CAT validation Criterion Validity Coefficient Composite Performance 0.36 (0.29)** Gathering Information 0.18 (0.14)** Reviewing and Analyzing Information 0.34 (0.27)** Decision Making 0.33 (0.26)** Strategic and Operational Agility 0.20 (0.16)** Innovation 0.20 (0.16)** Potential 0.27 (0.21)** Adaptability 0.25 (0.20)** Note. N=596 ** p < 0.01. Validity coefficients have been corrected only for unreliability in the criterion using a reliability estimate of 0.63. Values in parentheses represent uncorrected validity coefficients.

Interview scores by score group 2500 2000 1500 1000 500 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 RT Next scores step scores with passing with passing RS score CAT RT Next scores Step scores with or without or without RS score passing CAT

Assessment-driven metrics Employer brand index Recruiting effectiveness metrics

Test scores + HRIS data: Turnover survival analysis

Example 2: Tech-facilitated assessment center Key characteristics: Day-in-the-life format Technology driven Live interactions Deployed globally

COMPETENCIES PERSONALITY PATTERNS Karen Gates Vice President, Operations Started: 02.04.2000 Previous Position: Director, Operations Education: MBA, Wharton Business School Known Aspirations: VP of Eastern Region Sample Output Interpersonal Skills P Compelling Communication Leadership Effectiveness Inventory P Cultivating Networks Adjustment 72 S Navigating Politics Ambition 87 D Influence Sociability 25 Business Management D S P Building Organizational Talent Driving Execution Financial Acumen Interpersonal Sensitivity Prudence Inquisitiveness Learning Orientation 88 100 98 95 P P Operational Decision Making Entrepreneurship Leadership Challenge Inventory P Establishing Strategic Direction Volatile 33 Leadership S P S Leading Change Coaching and Developing Others Selling the Vision Argumentative Risk Averse Imperceptive Avoidant Arrogant 10 15 13 24 55 P Empowerment/Delegation Impulsive 11 Personal Competencies Attention Seeking Eccentric 53 69 P Executive Disposition Perfectionistic 24 S Passion for Results Approval Dependent 72

Multi-organization analyses Strengths 8000+ Executives Strategic Leaders Coaching Establishing Strategic Direction Financial Acumen Entrepreneurship Building Talent HRIS Company Profiles Strong Managers Strengths Driving for Results Vary by: Executive Level? Disposition Communicating Industry? with Impact Decision Making Experiences? Customer Personality Focus profile? Prior Assessments Company Performance

Level differences in leadership High Customer Focus Coaching Med Low Empowerment Business Savvy 2 nd Level 3 rd Level 4 th Level

Vacancy rate + competency analysis Expected vacancies: 10 Directors 66 Managers 184 Supervisors

Cultivating Innovation scores by position Strength Ready Ready with Development Not Ready at this Time

What if priorities change? Competency analysis: Driving Efficiency Strength Ready Ready with Development Not Ready at this Time

Big Data: Old wine or new opportunity

The answer: Big data: Old wine and New opportunity (with some significant challenges ahead)

Big Data about people Challenges in practice: Software is often an expensive hollow shell People data can be of poor quality Interpretations can be terribly flawed Managers defer to gut instinct

Big data: big skills required Complex data analysis and modeling Knowledge of people & systems in organizations Theory building and testing Communication and action planning

Big Data about people Opportunities: Interconnections across steps add new insights and strategic value Strong theory, modeling, hypothesis testing are essential to extract meaning and order from huge complexity Insight about people can be packaged to better inform organizational strategy

Concluding thoughts: Multi-disciplinary approach is important New opportunities are emerging for the role of assessment Potential for better people strategy if we can overcome common barriers We have an obligation to respond to the popular trend

Thank Thank you! you. Big Data: Old Wine or New Opportunity? Doug Reynolds July 23, 2013

Post-SIOP survey: What are you most excited about?

Vacancy rate estimation + competency analysis Years Projected Preliminary Talent Gap Projected Talent Gap Projected Talent Gap after internal sourcing Additional talent needed through internal talent identification, development and promotion Current Year Forecast Year Current Headcount Planned Headcount Current Vacancies % promoted out of role per year Retire % per year Resign, Terminate % per year % positions historically filled successfully through Internal Promotions Target % filled through external hires 2013 2016 3 Director 30 45 3 18 5% 4% 1% 22 15% 18 40% 10 Manager 150 300 12 162 7% 6% 2% 188 35% 122 30% 66 Supervisor 300 550 30 280 12% 8% 4% 461 40% 277 20% 184 Total 480 895 45 460 671 417 260 Summary Year Talent Supply Talent Demand Talent Gap 2013 435 480 45 2016 635 895 260 Projected Talent Gap, given current rate of expansion, promotions, retirments, resignations -- and targeted % of external hires