Presenter: Doug Reynolds, Development Dimensions International

Transcription

1 Presenter: Doug Reynolds, Development Dimensions International

2 Big Data and Talent Management: Using assessment and technology to run better organizations Presented by: Doug Reynolds, Ph.D. Senior Vice President & CTO Development Dimensions International (DDI)

3 Big Data: Old Wine or New Opportunity? Presented by: Doug Reynolds, Ph.D. Senior Vice President & CTO Development Dimensions International (DDI)

4 Big Data About People is Hot!

5 Anatomy of a trend Reference to Moneyball Uninformed overgeneralization Pivot point

6 Anatomy of a trend Review of common problems

7 Anatomy of a trend Discovery of known knowledge Nuggets of insight from new technologies

8 What does big data mean for I-O and HR? A visit to the dust bowl A sea of spurious correlation High school science New insights from new tools

9 A few definitions Big data: Large quantity and variety of data generated through internet-based systems; often involves data from multiple systems (3Vs). Digital exhaust: trace information remaining after use of an online tool, often irrelevant to the purpose of the tool. HR analytics: organizationally relevant statistics regarding people associated with the organization

10 Roadmap for the discussion Why are we talking about this now? The promise of big data in HR A few examples Old wine? New opportunity?

11 A few important trends Labor Market Management Technology Talent is a differentiator Aging workforce Skill gaps Automation Globalization Outsourcing Interoperability Services architecture Cloud computing

12

13 A (simplified) view of the evolution of business software

14 Start with an inefficient business process Input Step 1 Step 2 Step 3 Step 4 Output

15 design software to improve it Input Step 1 Step 2 Simplify, automate, increase availability, globalize, etc. Step 3 Step 4 Output

16 It doesn t take much to get started

17 Competitors may capture different parts of the process Company 1 Company 2

18 The next challenge: add value beyond your step Interconnect with other software tools to automate more of the business process

19 Acquire, merge, or build to own the whole process Attempt to support the whole process: Build more pieces Buy your neighbor Sell out Or, go out of business

20 Once you own one process, you get hungry for others Process: A B C D

21 Once you own one process, you get hungry for others Process: A B C D

22 Interconnections allow for more insight and strategic value A B C D Within Process Across Processes

23 In the HR context: Within Process Across Processes

24 Big data about people Recruit Promote Hire Manage Train

25 The promise of big data for talent management Strategic Impact Insight Process Automation

26 Candidate Source Assessment Data for Individual and Group Manager Satisfaction with Quality of Candidates Ratio Offers to Acceptance and Diversity of New Hires Candidate feedback on the hiring process Confidence of Hiring Manager in the New Hire and Confidence of the New Hire that they are in the Right Job Job Performance and Engagement Predictive Hiring Analytics Source Assessment Final Interview Job Offer First Day on the Job 6 Months on the Job 1 year on the job

27

28 A pervasive issue: assessment rigor Talent quality?

29 Examples of Assessment-driven Analytics

30 Example 1: selection testing Selection test for graduate hiring Relevant and effective across cultures Strong security (difficult to cheat on) Strong predictor of performance Available anytime, anywhere Brief

31 Test features Figural reasoning: measure of reasoning ability, critical thinking, and problem-solving Non-verbal/graphical items No translations Applicable regardless of candidate reading level Culture-free/fair for all candidate groups Allows for comparisons across cultures/countries

32 Internet-based computer adaptive testing (CAT) CAT addresses several common issues: Test Security Cheating Length of Candidate Experience Items drawn from extensive bank; low item exposure rates Different combination of items for each candidate; no single key available to be used for cheating Compared to traditional tests, shorter test time but superior precision

33

34 CAT: Development Process Calibration Research 200,000+ candidates for entrylevel professional jobs, globally Items researched via internet delivered test forms Test timing and question functioning developed from response data Ensured the test is inclusive to all candidates globally Then, criterion-related validation study conducted

35 Results from CAT validation Criterion Validity Coefficient Composite Performance 0.36 (0.29)** Gathering Information 0.18 (0.14)** Reviewing and Analyzing Information 0.34 (0.27)** Decision Making 0.33 (0.26)** Strategic and Operational Agility 0.20 (0.16)** Innovation 0.20 (0.16)** Potential 0.27 (0.21)** Adaptability 0.25 (0.20)** Note. N=596 ** p < Validity coefficients have been corrected only for unreliability in the criterion using a reliability estimate of Values in parentheses represent uncorrected validity coefficients.

36 Interview scores by CAT score group RT Next scores step scores with passing with passing RS score CAT RT Next scores Step scores with or without or without RS score passing CAT

37 Assessment-driven metrics Employer brand index Recruiting effectiveness metrics

38 Test scores + HRIS data: Turnover survival analysis

39 Example 2: Tech-facilitated assessment center Key characteristics: Day-in-the-life format Technology driven Live interactions Deployed globally

40

41 COMPETENCIES PERSONALITY PATTERNS Karen Gates Vice President, Operations Started: Previous Position: Director, Operations Education: MBA, Wharton Business School Known Aspirations: VP of Eastern Region Sample Output Interpersonal Skills P Compelling Communication Leadership Effectiveness Inventory P Cultivating Networks Adjustment 72 S Navigating Politics Ambition 87 D Influence Sociability 25 Business Management D S P Building Organizational Talent Driving Execution Financial Acumen Interpersonal Sensitivity Prudence Inquisitiveness Learning Orientation P P Operational Decision Making Entrepreneurship Leadership Challenge Inventory P Establishing Strategic Direction Volatile 33 Leadership S P S Leading Change Coaching and Developing Others Selling the Vision Argumentative Risk Averse Imperceptive Avoidant Arrogant P Empowerment/Delegation Impulsive 11 Personal Competencies Attention Seeking Eccentric P Executive Disposition Perfectionistic 24 S Passion for Results Approval Dependent 72

42 Multi-organization analyses 16,000+ Executives Strategic Leaders Strengths Establishing Strategic Direction Financial Acumen Entrepreneurship Building Talent HRIS Company Profiles Strong Managers Strengths Driving for Results Vary by: Executive Level? Disposition Communicating Industry? with Impact Decision Making Experiences? Customer Personality Focus profile? Prior Assessments Company Performance

43 Level differences in leadership High Customer Focus Coaching Med Low Empowerment Business Savvy 2 nd Level 3 rd Level 4 th Level

44 Example using alternative methods 16,000+ Executives Strategic Leaders Strong Managers Vary by: Personality profile?

45 Machine learning: not your typical PSYCH 650 class

46 Example: Random Forests Strategic Profile (Y/N)? Lo Ambition Hi Ambition Hi Risk averse Lo Sociability Med/Lo Risk Averse Hi Sociability Med Risk averse Lo Risk Averse

47 Example results: Random Forests & Logistic Regression Rank Random Forest Logistic Regression 1 Independent Thinker/ Strong Decision Maker Strong Interpersonal Relations Independent Thinker/ Strong Decision Maker 2 Energetic: Drives Self and Others 3 Strong Interpersonal Relations Conflict Averse (-) 4 Strategic/Creative B/W Thinking Style 5 Self-Promoting Emotionally Detached 6 Emotionally Unpredictable Thoughtful/Planful Error rate: 12.5% Error rate: 19% = Similar Rank = Close (Top 10 in opposing model) = Dissimilar Rank

48 Big Data: Old wine or new opportunity

49 The Answer: Big data: Old wine and New opportunity (with some significant challenges ahead)

50 Big Data About People Challenges in practice: Software is often an expensive hollow shell People data can be of poor quality New data sources are not well understood Interpretations can be terribly flawed Managers defer to gut instinct

51 Big Data: Big Skills Required Complex data analysis and modeling Knowledge of people & systems in organizations Theory building and testing Communication and action planning

52 Big Data About People Opportunities in practice: Interconnections across steps add new insights and strategic value Strong theory, modeling, hypothesis testing are essential to extract meaning and order from huge complexity Insight about people can be packaged to better inform organizational strategy

53 Post-SIOP survey: What are you most excited about?

54 SIOP Taskforce on Big Data Key areas for action: Meaning and definition of big data Training in research methods Theory generation & testing Interdisciplinary linkages Applications to validation practices Legal and ethical issues Education and awareness within SIOP Overcoming resistance

55 Will this trend leave I-O behind? In the world of Big Data, companies can assess people on real world performance. A flurry of new companies are now skipping I/O psychology and helping recruiters source candidates by analyzing their social data. While a test is a good way to understand someone, so is looking at everything they ve ever posted on the internet. Josh Bersin, Forbes, Oct 2013.

56

57 Concluding Thoughts: New opportunities are emerging for I-O and the role of assessment Potential for better people strategy if we can overcome common barriers We have an obligation to respond to the popular trend

58 Thank you. Thank you! Big Data: Old Wine or New Opportunity? Doug Reynolds February, 2014