Using Assessment Center Methodology to Accelerate Executive MBA Leadership Development

Transcription

1 Using Assessment Center Methodology to Accelerate Executive MBA Leadership Development Jeff Kudisch, Ph.D. Ralph J. Tyser Distinguished Teaching Fellow Managing Director, Office of Career Services University of Maryland, College Park, USA Presentation to the 35 th International Congress on Assessment Center Methods October 20-21, Singapore

2 A Quick Look at the Smith School 2,950 Undergraduate Students 970 Part-Time MBA Students 225 Full-Time MBA Students 180 Executive MBA & MS Students U.S., China, Europe 100 PhD Students Custom Executive Education

3 Comprehensive School 150 Full-Time Faculty 50 Part-Time Faculty 6 Academic Departments Accounting and Information Assurance Decision, Operations Information Technologies Finance Logistics, Business and Public Policy Management and Organization Marketing

4 Smith School Rankings #19 Undergraduate Program (10 th among public universities) - U.S. News & Word Report, 2010 #20 MBA Program, U.S. (5 th among public business schools) - Financial Times, 2010 #15 Part-time MBA Program - U.S. News & World Report, 2010 #22 EMBA Program (#15 Management Skills Rank) - Wall Street Journal, 2010 #14 World Research Ranking - Financial Times, 2010 #3 Intellectual Capital - BusinessWeek, 2008 #8 Top-Rated Schools by Recruiters (UMD) - Wall Street Journal, 2010 (#7 Accounting, #21 Business)

5 Setting the Stage: Business Education at a Crossroads Managers and recruiters are increasingly questioning the value and relevancy of MBA programs 1 Clear gaps between what s being delivered in MBA curricula and what s needed Newly minted MBAs lack essential leadership skills, teamwork, critical thinking, creativity and innovation, and cultural awareness Especially problematic given global evidence that organizations don t have the talent to lead them into the future 1 See Rubin, R. S., & Dierdorff, E. C. (2009). How relevant is the MBA? Assessing the alignment of required curricula and required managerial competencies. Academy of Management Learning & Education, 8, Also see Datar, S. M., Garvin, D. A., & Cullen, P. (2010). Rethinking the MBA: Business Education at a Crossroads. Boston, MA: Harvard Business School Publishing.

6 Fast Facts A recent survey of 425 senior executives in North America, Europe, and Asia showed that attracting and keeping skilled talent and developing employees into capable leaders were among the top ten business issues cited. 1 A 2005 survey of 750 CEOs indicated that 79% believed the ability to develop leaders was the most important or one of the top 5 factors in achieving a competitive advantage. 2 A recent Center for Creative Leadership (CCL) survey of 2,200 leaders from companies in the United States, India and Singapore identified that the four future key skills ranked most important for future success by respondents were found to be among the weakest for today s leaders: leading people, strategic planning, inspiring commitment, and managing change. 3 1 Accenture consulting study as cited in McGee (2005, August 16th). Why promote from the outside when the talent is inside? Information Week 2 see Center for Creative Leadership.; 3 see Taylor, S. (2010, February). Bridging the leadership gap. Chief Learning Officer -

7 Business Education at a Crossroads: The Path Forward Business schools must rethink accepted beliefs and assumptions and transform their curricula to better leverage handson exercises, experiential learning, and opportunities for self-reflection. Adding courses on business ethics and financial crises won t solve the problem Key question: Beyond coursework, what techniques can we use to enhance students self-awareness and proficiency on essential competencies required for successfully leading during challenging times?

8 Use of ACs in Business Schools Learning Outcome Assessment 1980 s AACSB partnered with DDI Undergraduate-Level Efforts Evaluating career readiness of business majors (Riggio et al., 1997) Predicting early career success of business majors (Waldman & Korbar, 2004) Graduate-Level Efforts: Guiding Leadership Development University of Tennessee MBA and EMBA programs University of Maryland EMBA program

9 Here was our response for our 1 st EMBA cohort Data-driven executive coaching Paper and pencil psychological assessments 360-degree feedback Leadership and development seminars Action Learning Projects Here is where we are now (our 10 th cohort) Assessment Center-driven executive coaching Behavioral and paper and pencil psychological assessments 360-degree feedback Targeted leadership development seminars Action Learning Projects linked to team assessments and team coaching

10 Assessment Centers Individuals participate in a series of situations that simulate the real world Behaviorally-oriented process used for a variety of organizational/human resources purposes Candidates evaluated on multiple competencies across multiple techniques Use of multiple assessors (raters) and integration session to reduce rater bias and thus increase rating accuracy Effective tools for predicting future overall job performance, management potential, performance in training, and career advancement; valid across cultures. Entry-level to executive applications; used increasingly around the world. Pictured: Stress Interview (OSS, WWII) Popular Simulations: Written: In-basket exercise Case analysis Interpersonal: Oral Presentation Group Discussion exercise Interview Simulation

11 Sample Organizations Using Assessment Center (AC) Methodology a Denotes Capital One runs Executive Development Centers (top-tier) and Leadership Development Centers (next tier) see Delahoussaye, M. (2002, March). When tomorrow comes. Training; b denotes Microsoft is using ACs as part of 35 th their selection process for Country Managers (GMs). International Congress on Assessment Center Methods, Singapore

12 Purpose of the Smith School Acceleration Center The Smith School is using the AC, in conjunction with the other Executive Coaching components, to help you diagnose and enhance your leadership strengths and development needs. The data may also be used to facilitate administrative issues related to enhancing the effectiveness of your EMBA endeavors.

13 Our EMBA AC Process

14 Seven Competencies Measured EXPRESSION CLUSTER Interpersonal Sensitivity Team Orientation COGNITIVE CLUSTER Judgment LEADERSHIP CLUSTER Process Leadership Persuasion & Influence Confrontation Initiative

15 Executive Summary Strengths: Proficiencies: Developmental Needs: Sample AC Report Judgment, Initiative, Confrontation Interpersonal Sensitivity, Persuasion & Influence Process Leadership Unable to Rate: Team Orientation Narrative Competency Summaries Listed below is a detailed, narrative summary of your performance across the various competencies. Evaluations of each competency are supported by behavioral examples (e.g., critical incidents) of performance. Interpersonal Sensitivity. Mr. Participant exhibited slightly below average interpersonal sensitivity in the Group Discussion Exercise. Although he acknowledged others ideas (e.g.., Yeah ; Yes, that s fine ), his comments appeared mechanical in nature. He also displayed weak active listening skills at times (e.g., he had a tendency to listen to respond versus listening to understand - he interrupted and talked over others to assert his views rather than learning what was said; he looked down when others were talking), and he used an insensitive, directive tone at times (e.g., he sometimes disagreed with others in an abrupt manner - "We can do that later..."; "But it said..."; "We need to..."). Overall, rather than coming across as affiliative, his style could be best described as detached, analytical, and task-oriented.

16 Benefits Positive participant reactions Jolts ; high impact insights Ah hah moments Accelerated coaching Richer development plans Cohort-level needs assessment data Accelerated on-the-job leadership development Enhanced Smith Brand Driving applications to the program

17 EMBA Reactions The simulation was fantastic, would like to add additional role play simulations to the first Assessment center experience though to really gain an insight in a persons leadership style. I enjoyed the experience, and learned from it!... This should be a very valuable part of the EMBA. Only gave it a four because I wish it was more thorough. I would like to have done the full assessment center, even if it had to be broken up over 2 days (or a day and a half). Other than that, it was excellent. I am able to communicate much better and more effectively with senior management. Through the executive coaching program I raised significantly my EQ by increasing my self-awareness and controlling my emotion and funned them in the appropriate manner to achieve results.

18 Would Do vs. Can Do Signs (Indicators of Behavior) Samples (Actual Behaviors) 1. What a personality test described: 1. What we observed in the AC: EMBA is insightful concerning how others feel and think. He is quick to respond to social nuances, and is likely viewed by others as reasonably perceptive and observant. He is pleasant to be with and usually easy to have around. EMBA displayed rather weak interpersonal sensitivity On the positive side, he occasionally complimented others On a less positive note he frequently interrupted others before they finished speaking; instead of responding to others' comments, at times he would transition to his own ideas; although he facilitated consensus on some decisions, at times he disagreed with others ideas and subsequently redirected the conversation without regard for their opinions he usually disagreed in an abrupt, brash manner using a harsh, rather arrogant tone Taken together the data suggest that while he can be personable, his assertive, competitive, task-oriented demeanor may prevent such behavior from consistently showing (e.g., he may come across as impatient toward others) The data further suggest that EMBA may lack emotional intelligence (e.g., he demonstrated limited empathy toward others feelings/needs and appeared to lack self-monitoring skills when interacting with others).

19 Would Do vs. Can Do Signs (Indicators of Behavior) Samples (Actual Behaviors) 2. What a cognitive ability test described: EMBA s performance places her in the 99th percentile, meaning that her problem solving capabilities are better than 99 percent of [upper division students in fouryear colleges] When compared to a sample of mid- to upper-level managers who have completed a similar assessment battery, her score is [again] in the 99th percentile Overall, she should have no problem at all recognizing key issues, making inferences, and coming to appropriate conclusions based on information given to her. In short, she should be an excellent problem-solver. 2. What we observed in the AC: EMBA demonstrated reasonable, though inconsistent, judgment in the LGD varied dramatically, from instances of excellent judgment, to instances of poor judgment could have enhanced her performance by doing a better job of consistently articulating the logic/rationale behind her ideas versus appearing opinionated, and by slowing down to ensure that all other team members were able to follow her thought processes. Doing so would also have had a positive impact on her ability to sway others, and would have ensured that she avoided premature closure on issues. On a final note she could have also enhanced her performance by better leveraging financial data to support her arguments (e.g., while she had very good grasp on most issues, she confused some of the P&L data).

20 Impact on EMBA Admissions Another element of my EMBA experience and how it will help advance my ability to execute on strategy is what I will learn through the assessment center and executive coaching experience. The psychological evaluation and the interpersonal approach is evidence that the program uses leading research and knowledge to develop the intangible qualities of leadership. This bespoke and experiential approach is what I value most out of the program, and I hope to devote as much energy as I possible can to maximize my development in this area.

21 Cost Challenges As high as $10-14K for executive-level assessments Time Staff Resources Top Management Commitment Intellectual Capital Demonstrating ROI

22 How We Overcame These Challenges Custom Program Success with Fortune 500 firm Top Management Intrigue & Buy-In Saw as a unique value proposition Narrowed the focus to key competencies Used fewer exercises Leveraged In-house expertise Raised EMBA tuition

23 Toward ROI: Initial Research Findings To what extent does participation in the EMBA Acceleration Center influence or shed light on executive coaching success? Data collected from 3 of our EMBA cohorts 100 executives, 13 coaches 18-month engagement (5 sessions as hours each) Survey methodology - measured throughout the coaching engagement; solicited coach and EMBA reactions

24 Executive Characteristics What characteristics of an EMBA (assessee), if any, help explain differences in his/her success in executive coaching? Limitation = Behavior change measured after session 4 and 5; Small Sample based on coach ratings; 4 items: α=.92 (n =46) Size The executive Definitely learned and applied new behaviors throughout coaching Acted on the majority of recommendations from previous meetings Presents evidence of behavior change Seems more confident about what s/he is doing on the job as it related to areas addressed in coaching r Judgment, behavior change (time 4) r Judgment,, behavior change (time 4) =.31 p<.05 r Interpersonal, behavior change (time 5) r Interpersonal,, behavior change (time 5) =.33 p<.05 r OAR, behavior change (time 4) r OAR,, behavior change (time 4) =.31 p<.05

25 Executive Characteristics (cont.) Findings consistent with past research that suggests that people with certain characteristics are more likely to accept feedback and participate in developmental activities c.f., judgment, personality, conscientious, goal orientation (Byham 2005; Jones & Whitmore, 1995; Kudisch & Ladd, 1997; Kudisch, Lundquist, & Smith, 2001; Poteet & Kudisch, 2003; Atchley, Smith, Hoffman, 2003) Further evidence that the rich get richer From a broader talent management perspective, the findings highlight the potential link between assessment center methodology and executive coaching for growing leaders (c.f., Poteet & Kudisch, 2008) Can we expect cross-cultural consistency in these results?

26 Executive Characteristics (cont.) What EMBA (assessee) characteristics help explain differences in his/her engagement in executive coaching? Engagement rated by Executives after sessions 2-5 Consisted of behaviors during the coaching session: Actively engaged in the session Came prepared, made it a priority And behaviors outside the coaching session: How hard they worked how much time they put in whether they accomplished what they planned to do 3 items, average α =.75; Since your last coaching session, how hard did you work towards your objectives laid out in coaching?

27 Role of Cognitive Ability Those with better reasoning and problem solving skills are more accepting of data (e.g., feedback) and are more likely to act on it Those with greater cognitive skills should value the feedback provided in coaching and engage more in the coaching process Especially in those coaching programs that are very assessment-oriented Watson-Glaser Critical Thinking Appraisal; 80 items, α =.75

28 Interesting & Unexpected Finding Hierarchical linear regression, controlling for coach Executive Characteristics (measured at T1) effects on engagement from T1 T5. Contrary to predictions, cognitive ability was negatively related to engagement! T2: β=.54, p=.00 T3: β=.22 p=ns T4: β=.23, p=ns T5: β=.52, p=.00 Why? Doesn t measure cognitive openness or flexibility Tapping an Alpha Male profile? 1 Bright, analytical, interested in learning about business, technology, etc., but no interest in introspection or curiosity about people. 1 1 See Ludeman & Erlandson (2004). For more see Gettman, H., Wouters, K., Stevens, C., Kudisch, J., Edinger, S., Russell, J., & Tesluk, P. (2010, April). Factors that influence executive engagement in coaching (and does it even matter?). Paper presented at the 25th annual conference of the Society of Industrial and Organizational Psychology, Atlanta, Georgia.

29 Here was our response for our 1 st EMBA cohort Data-driven executive coaching Paper and pencil psychological assessments 360-degree feedback Leadership and development seminars Action Learning Projects Here is where we are now (our 10 th cohort) Assessment Center-driven executive coaching Behavioral and paper and pencil psychological assessments 360-degree feedback Targeted leadership development seminars Action Learning Projects linked to team assessments and team coaching

30 An Integrated EMBA & Intensive Leadership Development Program An Integrated, Personal Leadership Development Plan Assessment & Coaching Courses The Right Tools Practice and Impact Action Learning Projects Leadership Mastery Seminars Targeted Skill Development

31 What s on the Smith Horizon Using more assessment methodology in our EMBA admissions process Expanding the assessment center methodology and coaching Using more assessment methodology in our part-time and full-time MBA programs Importance of handling feedback in a culturally sensitive manner given student diversity (40% FT MBA International) More systematically exploring our ROI Smith Executive Coaching Consortium

32 Questions/Co mments? Feedback and follow-up are most welcome:

33 References Atchley, E. K., Smith, E. M., & Hoffman, B. J. (2003, September). Examining the relationship between performance, individual differences, and developmental activities: Getting more bang for the buck from DPACs. In T. Maurer (Chair), Where do we go from here? Accepting and applying assessment center feedback. General session presented at the 31st annual meeting of the International Congress on Assessment Center Methods, Atlanta, GA. Byham, T. M. (2005). Factors affecting the acceptance and application of developmental feedback from an executive assessment program. Unpublished doctoral dissertation. University of Akron. Jones, R. G., & Whitmore, M. D. (1995). Evaluating developmental assessment centers as interventions. Personnel Psychology, 48, Kudisch, J. D. & Ladd, R. T. (1997, April). Factors related to participants acceptance of developmental assessment center feedback. Paper presented at the 12th Annual Conference of the Society for Industrial and Organizational Psychology. St. Louis, MO. Kudisch, J. D., Lundquist, C. & Smith, A. F. R. (2001, October). Reactions to dual-purpose assessment center feedback: What does it take to get participants to buy into and actually do something with their feedback? Paper presented at the 29th International Congress on Assessment Center Methods, Frankfurt, Germany.

34 References Ludeman, K., & Erlandson, E. (2004, May). Coaching the alpha male. Harvard Business Review, Poteet, M. L., & Kudisch, J. D. (2003, September). Straight from the horse s mouth: Strategies for increasing feedback acceptance. In T. Maurer (Chair), Where do we go from here? Accepting and applying assessment center feedback. General session presented at the 31st annual meeting of the International Congress on Assessment Center Methods, Atlanta, GA. Poteet, M. L., & Kudisch, J. D. (2008, September). Using executive coaching to leverage assessment center feedback: Tips for success. Paper presented at the 34th annual meeting of the International Congress on Assessment Center Methods, Washington, DC.