How to develop emotional intelligence tests - GECO - GERT

Transcription

1 How to develop emotional intelligence tests - GECO - GERT Marcello Mortillaro Katja Schlegel

2 Geneva Emotional Competence (GECO) Test Emotional intelligence in the workplace Marcello Mortillaro Katja Schlegel

3 Take home message Try to use a theoretically informed approach to measure EI that is based on: - Emotion theory - Specific models for each task

4 Two streams of measurement of EI As a trait via self-report questionnaires (trait EI) As a cognitive ability via performancebased tests (ability EI)

5 Ability Emotional Intelligence EI is conceived as a cognitive ability that can be measured with performance-based tests, i.e., tasks that have a correct answer. Ability EI is similar to other types of intelligence in the sense that people can be ranked on emotion-related tasks.

6 MSCEIT Problems Consensus based scores: 1) validity of the criterion; 2) difficult to differentiate the subjects with a really high EI score; 3) creative response style is disadvantaged Likert scales rather than «pick the right answer» Not specific to the work setting

7 Features of the new test - Ability EI - Performance based - Organized in sub-tasks based on EmotionTheory - Builds upon recent advancements (e.g., situational judgment tests STEU - STEM) - Concrete real(istic) scenarios - Good discrimination of individual with high EI - Response options built according to appropriate theoretical models

8 Emotion interaction process Expresser Event Appraisal Physiology Expression Motivation Subjective feeling Emotion Understanding Observer Expresser Perception Appraisal Appraisal Subj. Feeling Physiology Motivation Self (monitoring) Observer s Emotion Expression Action Expression Emotion Recognition Emotion Regulation Emotion Management

9 GECO Test Emotional Intelligence in the workplace Structure Emotion Recognition Emotion Management Emotion Understanding Emotion Regulation

10 Critical incidents interviews Interviews with 38 managers & HR employees to identify emotional scenarios at the workplace and actual and possible reactions Anger - Fear - (Inappropriate) Happiness - Sadness Initial transcription pool (195 items, 3-5 sentences each) Used for Emotion Management, Emotion Understanding, and Emotion Regulation

11 Emotion Management The ability to respond in an effective way to somebody s else emotions In this subtest, the respondent chooses the option that describes the behavior that they would most likely act in response to somebody s else emotional behavior

12 Emotion Management The items describes interactive real-worklife scenarios in which another person experiences one emotion (anger, fear, sadnes, inappropriate happiness) that needs to be coped with. The participants choose how they would react.

13 Emotion Management Response options represent 5 conflict management strategies (Thomas, 1992) Avoidance Competition Collaboration Compromise Accommodation

14 Emotion Management Scoring of correct response based on its adaptiveness The adaptiveness of the behavior varies according to objective factors (the specific context, availability of resources, etc.) and is defined according to conflict management theory (e.g. urgency of the response, type of relationship, etc.)

15 Emotion Management The emotions are explicitly stated in the item in order to avoid confusions and overlap with other sub-tests (e.g., emotion understanding)

16 Emotion Management Your colleague, with whom you get along very well, tells you that he is getting dismissed and that you will be taking over his projects. While he is telling you the news he starts crying. He is extremely sad and desperate. You have a very important meeting coming up in 10 minutes. What do you do? 1. You take some time to listen to him until you get the impression he calmed down a bit, at risk of being late for your meeting. (compromising) 2. You cancel your meeting, have a coffee with him and tell he has every right to be sad and desperate. You ask if there is anything you can do for him. (accommodation) 3. You tell him that you are sorry, but that your meeting is very important. You say that you may find some time another day this week to talk about it. (avoidance) 4. You suggest that he joins you for the meeting with your supervisor so that you can plan the transfer period together. (collaboration) 5. You cancel your meeting and offer him to start discussing the transfer of his projects immediately. (competing)

17 Emotion Management development overview From the transcriptions we generated an initial pool of 75 items First expert check: clarity of the scenarios, and correct correspondence options/strategies (54 items) Second expert check (16 emotion researcher): pick the best Mechanical Turk studies: Different instructions The theoretically defined solution must be the most frequently chosen (minimum 30%) 44 Items (at least 2 per emotion X strategy combinations) Expert ratings (17 managers): believability score

18 Emotion Understanding The ability to understand which emotion is felt by somebody in a certain situation In this subtest, the task for the respondent is to choose the label that describes the emotion that most likely is experienced in one scenario.

19 Emotion Understanding Items created according to the Component Process Model (Scherer, 2001), an appraisal theory of emotions that predicts a specific pattern of cognitive evaluations (appraisals) for each target emotion. Appraisals include novelty, conduciveness, coping potential, etc.

20 Emotion Understanding For example key appraisals of anger are Goal Hindrance /Obstructiveness External causation High Coping Ability Violations of norms

21 Emotion Understanding We generated items (n=48) according to appraisal theory for 15 emotions Expert rating (10 emotion researchers):at least 5 chose the target emotion and no other label was chosen by 3 or more experts. Mturk study: accuracy used to choose two items per emotion trying to have variance in the «difficulty level» 29 Items:3 for shame, 2 for the other emotions (despair was excluded)

22 Emotion Understanding John has to give a presentation in front of many people in a language that he does not speak very well. What is the most likely emotion that John feels? Anger Irritation Contempt Disgust Fear Anxiety Guilt Pride Joy Interest Relief Boredom Sadness Shame

23 Emotion Regulation The ability to use cognitive strategies to modulate one s own personal emotional experiences

24 Emotion Regulation Several researchers reviewed a large variety of possible emotion regulation strategies and provided different taxonomies Strategies vary in term of appropriateness to the situation and have different adaptive values

25 Emotion Regulation Items describe negative emotional realistic situations The respondent is asked to imagine herself in those situations The alternatives represent different cognitive emotion regulation strategies that vary in terms of their adaptive value/appropriateness

26 Emotion Regulation Participants read scenarios in which they experience negative emotions (fear, sadness, or anger) 4 response options (2 adaptive and 2 maladaptive regulation strategies Choose 2 out of 4 options Adaptive responses are scored as correct Garnefski & Kraaij, 2006 Adaptive Acceptance Positive refocusing Refocusing on planning Positive reappraisal Putting into perspective Maladaptive Self-blame Other-blame Ruminating Catastrophizing

27 Emotion Regulation We generated 44 items from the transcriptions of the interviews and all strategies equally represented within and between emotions Expert check (emotion researchers): check for the intended strategy in the response options Mturk study: excluded items with less than 15% score in one of the 3 possible scores - 0, 0.5, 1 31 items

28 Emotion Regulation You successfully completed a very important project that took a lot of your time. When you return to your daily business, your boss tells you that he is unhappy that you neglected your other projects. You are very irritated about the lack of acknowledgement of your boss. What do you think? 1. You think that you should have been better organized and have worked on all projects at the same time. (self-blame) 2. You think that you have to accept that bosses are never fully satisfied. (acceptance) 3. You think about the very positive feedback by the client for whom you completed the project. (positive refocusing) 4. You think that your boss is always unfair to you and that you should consider quitting if this continues. (catastrophizing)

29 Emotion Recognition The ability to recognize which emotion is expressed by somebody on the basis of his/her nonverbal behaviour Adaptation of the Geneva Emotion Recognition Test

30 Emotion Recognition Development of a short version of the Geneva Emotion Recognition Test (GERT, Schlegel et al., 2014) 42 items: 14 emotions, 3 items each

31 Construct validation study N=149 students at the University of Geneva Situational Test of Emotional Understanding (STEU; MacCann & Roberts, 2008) Situational Test of Emotion Management (STEM; MacCann & Roberts, 2008) Big Five Inventory (BFI; John, Donahue, & Kentle, 1991) Cognitive Emotion Regulation Questionnaire (CERQ; Garnefski & Kraaij, 2006) Emotion Recognition Index (ERI; Scherer & Scherer, 2011) Intelligence test NV5R - General Reasoning (Thiébaut & Bidan-Fortier, 2003)

32 Construct validation study Correlations Emotion Regulation Emotion Management Emotional Understanding Emotion Recognition STEM (Emotion management) ns.36**.29**.28** STEU (Emotional understanding) ns.48**.56**.33** ERI (Emotion recognition) ns.35**.29**.41** NVI5-R (general reasoning) ns.32**.29**.38** CERQ (adaptive regulation).38** ns ns ns Big Five- Extraversion.36** ns ns ns Big Five- Agreeableness.25**.16* ns ns Big Five- Conscientiousness ns ns ns ns Big Five- neuroticism -.32** ns ns.23** Big Five- openness.17* ns ns ns

33 Test Difficulty & Cronbach s Alpha M=.45, α= items 20 items M=.66, α=.59 M=.66, α= items M=.56, α= items Higher difficulty than other ability EI tests Reliability for emotion understanding is low, but IRT analyses are needed

34 Summary The GECO is a performance-based test to measure 4 emotional competencies in the workplace It is available online, administration time ~30 min First evidence for good psychometric quality and construct validity Next step: Investigation of predictive validity in samples of employees and managers

35 Wrap-up The new generation of EI test should be - Performance based - Informed by emotion theory in its articulation in sub tests - For each sub-test appropriate theoretical framework should be identified - Items should be built as close as possible to real(istic) scenarios using SJT

36 Thank you