When underconfident behavior is norm: some experimental evidences from the calibration analysis 1

Transcription

1 When underconfident behavior is norm: some experimental evidences from the calibration analysis 1 Pablo Fajfar Department of Economics and Mathematics, School of Economics, Buenos Aires s University Argentina. Correspondence to: pffajfar@yahoo.com.ar Natacha Gurman School of Psychology - Buenos Aires s University Argentina. Correspondence to: natachagurman@hotmail.com Abstract The over/underconfidence behavior has been explained as a bias of the cognitive process in the decision maker. Such bias has been mainly justified by the difficulty of the task or the problem to decide upon. It s the so called hard- easy effect (Lichtenstein and Fischhoff, 1977). In this paper we demonstrate that underconfidence dramatically increases whenever one's decision consequences are immediately known. In this way, we state that a closer time lapse between expectations and actual performance may trigger an alert signal in subjects. Such signal emotionally activates the loss aversion effect and makes most subjects turn into underconfident. JEL Classification: C91; D03; D81. Keywords: Underconfidence bias; decision consequences; alert signals. Resumen El comportamiento de Sobre o Subconfianza es explicado como un sesgo cognitivo en el proceso de toma de decisiones del sujeto. Tal sesgo, ha sido justificado principalmente por la dificultad del problema o tarea a resolver; el así llamado, efecto difícil fácil (Lichtenstein and Fischhoff, 1977). En este trabajo demostramos que la Subconfianza se incrementa pronunciadamente cuando las consecuencias de una decisión son conocidas tempranamente. En este aspecto, establecemos que la cercanía temporal entre las expectativas y los resultados consecuentes a ellas, generan una señal de alerta en los sujetos. Dicha señal, activa emocionalmente el efecto de evitar las perdidas subjetivas, transformando así a la gran masa de los sujetos en Subconfiados. Clasificación JEL: C91; D03; D81. Palabras Claves: Sesgo de Subconfianza; Consecuencias de una decisión; Señales de alerta. 1 A preliminary version of this paper was presented in the International Association for Research in Economic Psychology (IAREP) and Society for Advancement of Behavioral Economics (SABE) Joint conference - July 2009, Halifax Canada-.

2 1. Introduction Calibration is defined as the difference between the subjective performance expectations and the actual performance in a decision-making process (Oskamp, 1965; Lichtenstein, Fischhoff & Phillips, 1982; Yates, 1990). When the subjective performance expectations for any decision maker are greater than their actual performance, they are defined as overconfident subjects. On the other hand, when the subjective expectations are lower than the actual performance they are defined as underconfident subjects. Over/underconfident behavior has been explained as a bias of the cognitive process in the decision maker (Björkman, 1994; Gigerenzer, Hoffrage & Kleinbölting, 1991; Juslin, 1993a, 1993b, 1994). Such bias in the cognitive process has been mainly justified by the difficulty of the task or the problem to decide upon. The main previous in this way were Lichtenstein and Fischhoff (1977) who found that in terms of general knowledge item questionnaire, overconfidence increases/decreases whenever the difficulty of the task increases/decreases. That means, for lower actual performance -lower level of correct answers- they find overconfidence, and for higher cases they find underconfidence. This finding was the so called hard- easy effect. 2 Most of the contemporary studies about over/underconfidence in calibration analysis keep focusing their attention in terms of the hard- easy effect, that is, the difficultly of the task (Griffin & Tversky, 1992; Keren, 1991, 1997; McClelland & Bolger, 1994; Pulford & Colman, 1997; Juslin, Winnan & Olsson 2000; Merkle 2008). However, none of these studies pay attention to temporal effects involved in decision-making process. Because of this, we also consider significant to take into account the time lapse between the moment when the subjective expectations are formed and the moment when the actual performance becomes known. In this way, Gilovich, Kerr & Husted Medvec (1993) demonstrated that people tend to lose confidence in their prospect for success the closer is the moment of truth. 3 According to this, suppose the following event: You have just finished a grade examination and you must declare your expected performance! In which of these scenarios would you be less confident? Scenario I: You ll know the results of your actual performance in 10 minutes. Scenario II: You ll know the results of your actual performance next week. It is understandable that you will be less confident in the first one. Why this? First, people have loss aversion (Kahneman, Knetsch & Thaler, 1991). We take from this idea that it is reasonable that nobody wants to see him/her fail! In this sense, no one wants to know that he/she performed lower than he/she expected about him/herself. That is; people tend to avoid the narcissist wound that overconfidence could cause (Sigmund Freud, Introduction to Narcissism, 1914). Second; a closer time lapse between subjective performance expectations and actual performance may trigger an alert signal in subjects. The signal states; keep in mind that your actual performance will be given to you in a few minutes and remember that you will try to avoid seeing yourself as an overconfident subject. This signal emotionally activated 2 In the hard - easy effect, the difficultly of the task is explained by the actual performance on subjects. So, a higher actual performance is synonymous of an easy task, and a lower, of a hard task. 3 As the moment of truth, authors established the moment to perform a given task. 2

3 and reinforced the loss aversion effect and makes most subjects turn into underconfident. Underconfidence makes sometimes people happier! Underconfidence is sometimes a subjective gain! In the following experiment we demonstrate that underconfidence dramatically increases whenever one's decision consequences are immediately known. In other words, we present a scenario where underconfident behaviors are the norm. 2. Our Experiment One hundred and fifty five voluntary students (83 males and 72 females) of Economics and Social Sciences from the University of Buenos Aires, Universidad Católica Argentina and Universidad Abierta Interamericana were requested to answer a 34 multiple-choice questionnaire from the Baires s test of verbal performance (Cortada de Kohan, 2003). Students were randomly recruited by public electronic announces via web pages - made at the three Universities. The experiment sessions were made in the lab of the School of Economics from the Buenos Aires University. The Baires s test contains 34 multiple choice items with four options per item, where there s only one correct answer. The first 17 items present a noun and four possible definitions. The 17 remaining items, present a noun and four synonymous options. For every correct answer, for Baires s test performance, students were paid $1 without discounting any monetary value for the incorrect ones. At the end of the questionnaire we asked the participants to report the number of questions they believed they answered correctly i.e., the money they expected to have accumulated Experiment Treatments Two treatments have been put in practice. In treatment 1 (TR1-79 subjects) the participants knew they would be informed of their actual performance - i.e., the money they have accumulated- once the task was finished. In treatment 2 (TR2-76 subjects) the participants knew they would be informed of their actual performance after a week. In both treatments participants receive their payments -i.e., the money they have accumulated for the correct answers- a week after the test. 3. Hypothesis Starting with the fact that a subject is overconfident / underconfident if the expected numbers of correct answers are greater / smaller than the actual correct ones (Lichtenstein, Fischhoff & Phillips, 1982), we state that: Even when the difficulty of the task or the problem to decide is the same in treatment 1 (TR1) as in treatment 2 (TR2); underconfidence dramatically increases whenever one's decision consequences are immediately known! According to this, we expect to find a greater number of underconfident subjects in treatment 1 (TR1). 4 The experiment instructions are presented in Appendix section. 3

4 4. Experiment Outputs Table 1 presents the main descriptive statistics from the TR1 & TR2. Table 1 Experimental results. Data: Overall data: Calibration Coefficient (Mean) p-value = (3.82) (4.00) Overconfidence Cases: 11 Subjects; 14 % 35 Subjects; 46% Calibration Coefficient (Mean) p-value = (2.3) (2.33) Percentage of Correct Answers (Mean) 61.5% 57.73% p-value = Percentage of Expected Correct Answers (Mean) 68.45% 67.31% p-value = Underconfidence Cases: 60 Subjects; 76 % 34 Subjects; 45% Calibration Coefficient (Mean) p-value = (2.97) (2.6) Percentage of Correct Answers (Mean) 67.6% 63% p-value = Percentage of Expected Correct Answers (Mean) 54.21% 52.5% p-value = Calibres Cases 8 Subjets; 10% 7 Subjets; 9% Percentage of Correct Answers (Mean) = Expected 61% 62.6% p-value = (Standard Deviations between brackets) Treatment 1 ;TR1 (N=79) Treatment 2 ;TR2 (N=76) Diferences (Wilcoxon/Mann-Whitney) As shown in Table 1, the mean of the calibration coefficient (measured as the difference between the expected numbers of correct answers and actual correct ones) were statistically different between TR1 and TR2 p< In TR1, the difference between the expected number of correct answers and actual correct answers were = In TR2, the difference were = Notice that in TR2 an asymptotical tendency to the perfect calibration state was observed. However TR1 s subjects presented a dramatically bias of underconfident behavior. 5 The proportion between over / underconfidence cases were 14 % / 76 % in TR1, against 46 % / 45% in TR2. It is clear that the numbers of underconfidence cases were significantly greater in TR1 than in TR2. Moreover notice that the mean of the calibration coefficient was not statistically different for over / underconfidence cases between treatments. In this sense, the percentage of expected correct answers and actual correct answers i.e. percentage of correct answers- were statistically the same between over/underconfident subjects in both treatments. 6 That means; the TR1 underconfidence bias is explained by the number of underconfident subject s cases, but not by the underconfidence intensity i.e. calibration coefficient magnitude-. For a robustness explanation about the previous results, two probabilistic models were constructed. In the first one (model 1), we estimated the probability of underconfident behavior using as explanatory variables the age of the subjects, the gender, the percentage of actual correct answers, and the treatment effect as a categorical variable. The latter variable adopted 1 if the subject belongs to TR1 and 0 for TR2. 5 The actual correct answers were statistically greater than the expected correct answers. 6 The correlation between the expected numbers of correct answers and actual correct answers were statistically greater for TR2 subjects. In this way, for the overconfidence cases the Pearson correlations were 0.87 p<0.000 in TR2 against 0.61 p<0.05 in TR1. For the underconfidence cases, the Pearson Correlations were 0.87 p<0.000 in TR2, against 0.65 p<0.000 in TR1. 4

5 In the second one (model 2), we replicate model 1 estimation using the percentage of expected correct answers as explanatory variable. 7 Table 2 reports the marginal effects change in probabilities- for model 1 estimation. Table 2; Probit regression, reporting marginal effects Number of obs = 140 LR chi2(4) = Prob > chi2 = Log likelihood = Pseudo R2 = prob_u~r df/dx Std. Err. z P> z x-bar [ 95% C.I. ] age gender* pcorrect treatm~t* obs. P pred. P (at x-bar) As shown in Table 2, age had a negative effect on the probability of underconfident behavior p< 0.06, but with a small real impact. 8 Gender did not have a statistical influence. The percentage of actual correct answers significantly increased the probability of underconfident behavior p< In this way, the higher the actual performances on subjects, the higher their underconfident behavior probability. Notice that those results are consistent whit the hard- easy effect. 9 At last, the categorical variable of treatment effect reflected that underconfident behavior increases 26 % in TR1 p< In this way, the cognitive impact of knowing that your decision consequences would take place immediately dramatically increased the probability of underconfidence behavior even when the difficulty of the task were the same between people. Graph I presents the probability response curves of underconfident behavior between TR1 & TR2 using the percentage of correct answers as explanatory variable. In there, the treatment effects - as a categorical probability level variable- reflects that the time elapsed between the moment when the subjective expectations are formed and the moment when the actual performance is known makes a difference of the behavior on subjects. 7 Notice that the expected number of correct answers and the actual correct ones had a serial correlation. 8 The probability of underconfident behavior decrees 1.7 % when the age of the subjects increases. 9 Remember that in the hard - easy effect, an easy / hard task is explained by the actual performance on subjects. So, a higher actual performance is synonymous of an easy task, and a lower of a hard task. 5

6 Graph I; Probability Response Curves of Underconfident Behavior between TR1 -TR2 (Probit Estimation) Probability of Underconfident Behavior TR1 TR Percentage of Correct Answers The vertical axis expresses the probability of underconfident behavior as a result of the model 1 estimation (binary probit model estimation). The horizontal axis expresses the percentage of correct answers as a difficulty task indicator (0.1 the harder and 1 the easier). For every correct answers percentage, the probability of underconfident behavior is statistically greater in TR1. Table 3 reports the marginal effects change in probabilities- for model 2 estimation. Table 3; Probit regression, reporting marginal effects Number of obs = 140 LR chi2(4) = Prob > chi2 = Log likelihood = Pseudo R2 = prob_u~r df/dx Std. Err. z P> z x-bar [ 95% C.I. ] age gender* expcor~t treatm~t* obs. P pred. P (at x-bar) 6

7 As shown in Table 3, age and gender did not have a statistical influence on the probability of underconfident behavior. However, the percentage of expected correct answers significantly decreases the probability of underconfident behavior p< In this way, the higher the expectations on subjects, the lower their underconfident behavior probability. Notice that these results are consistent with the confidence behavior theory. That s, people who expected a higher subjective performance are naturally more confident less underconfident- than people who expected a lower performance. Nevertheless, the categorical variable of treatment effect reflected that underconfident behavior increases 36 % in TR1 p< In this way, and according with model 1, the cognitive impact of knowing that your decision consequences would take place immediately dramatically increase the probability of underconfident behavior. Graph II presents the probability response curves of underconfident behavior between TR1 & TR2 using the percentage of expected correct answers as explanatory variable. In there, the treatment effect - as a categorical probability level variable- reflects that the time elapsed between the moment when the subjective expectations are formed and the moment when the actual performance is known makes a difference of the behavior on subjects. Graph II; Probability Response Curves of Underconfident Behavior between TR1-TR2 (Probit Estimation) TR1 Probability of Underconfident Behavior TR Percentage of Expected Correct Answers The vertical axis expresses the probability of underconfident behavior as a result of the model 2 estimation (binary probit model estimation). The horizontal axis expresses the percentage of expected correct answers as a subjective expectation indicator (subjective performance expectation). For every expected correct answers percentage, the probability of underconfident behavior is statistically greater in TR1. 7

8 5. Conclusions and Discussion Proposals Conclusions In this paper we have demonstrated that underconfidence dramatically increases whenever one's decision consequences are immediately known. In others words, we have shown that over/underconfident behavior depends on the time elapsed between the moment when the subjective expectations are formed and the moment when the actual performance is known. At first, we find that even when the difficulty of the task or the problem to decide is the same between people, the cognitive & emotional impact of knowing that your decision consequences would take place immediately dramatically increased the propensity to underconfident behavior. In that sense, we reported several questions to the so called hard- easy effect in calibration analysis who establish that the difficulty of the task is the main justification for over/underconfident behavior. About this, our question are; why the difference between TR1 and TR2? Second, we find that the greater underconfidence behavior consequence of knowing that your decision consequences would take place immediately can be established as a norm rather than the underconfidence intensity in underconfident subjects. In this way, our experimental results showed that the greater underconfidence bias is explained by the number of underconfident subject s cases, but not by the calibration coefficient magnitude. That means; the closer are the times lapse between expectations and actual performance, the greater are the number of people who adopted underconfident behavior as a rule. Discussion Proposals What kind of explanations can be found for this experiment? It s reasonable that nobody wants to see him/her fail. People have loss aversion and the probability of an ex -post overconfidence scenario, as a failure emotional dilemma, could cause a narcissistic wound for everyone. In this way, the time proximity between the subjective performance expectations and actual performance may trigger an alert signal in subjects. The signal states: Keep in mind that your actual performance will be given to you in a few seconds, minutes, hours...; and remember that you will try to avoid themselves as overconfident. Such signal emotionality activates and reinforces the loss aversion effect and makes most subjects turn into underconfident. Underconfidence makes sometimes people happier! Underconfidence is sometimes a subjective gain! Confidence and expectations are two of the main topics in economic theory. The more we learn about them, the more questions arise. In this sense, confidence and expectations are mainly subjective perceptions rather than simple objective calculus. 8

9 References: Björkman, M. (1994). Internal cue theory: Calibration and resolution of confidence in general knowledge. Organizational Behavior and Human Decision Processes, 58, Cortada de Kohan, N. (2003). BAIRES. Test de Aptitud Verbal. Madrid: TEA. Freud, Sigmund (1911). Formulations on the two principles of psychic happen. Vol.12 Editorial Amorrortu. Freud, Sigmund (1914). Introduction to Narcissism. Vol. 14 Editorial Amorrortu. Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, Gilovich, T., Kerr, M. & Husted Medvec, V. (1993). Effect of temporal perspective on subjective confidence. Journal of Personality and Social Psychology, 64, Griffin, D., & Tversky, A. (1992). The weighting of evidence and the determinants of confidence. Cognitive Psychology, 24, Juslin, P. (1993a). An ecological model of realism of confidence in one s general knowledge. Acta Universitatis Upsaliensis: Studia Psychologica Uspaliensia, 14. Stockholm: Almqvist & Wiksell. Juslin, P. (1993b). An explanation of the hard easy effect in studies of realism of confidence in one s general knowledge. European Journal of Cognitive Psychology, 5, Juslin, P. (1994). The overconfidence phenomenon as a consequence of infor experimenter guided selection of almanac items. Organizational Behavior and Human Decision Processes, 57, Juslin, P., Winnan, A., & Olsson, H. (2000). Naïve Empiricism and Dogmatism in Confidence Research: A Critical Examination of the Hard Easy Effect. Physiological Review, 2, Kahneman, D. Knetsch, J.L. & Thaler R. (1991). Anomalies: The endowment effect, loss aversion and status quo bias. The journal of Economic Perspectives, Vol. 5, No. 1, Keren, G. (1991). Calibration and probability judgments: Conceptual and methodological issues. Acta Psychologica, 77, Keren, G. (1997). On the calibration of probability judgments: Some critical comments and alternative perspectives. Journal of Behavioral Decision Making, 10, Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20,

10 Lichtenstein, S., Fischhoff, B. & Phillips, L.D. (1982). Calibration of subjective probabilities: The state of the art up to In D. Kahneman, P. Slovic, & A. Tversky (Eds.). Judgment under uncertainty: Heuristics and biases (pp ). New York: Cambridge University Press. McClelland, A. G., & Bolger, F. (1994). The calibration of subjective probabilities: Theories and models In G. Wright & P. Ayton (Eds.), Subjective probability (pp ). Chichester, England: Wiley. Merkle, E., C. (2008). The disutility of the hard easy effect in choice confidence. Psychonomic Bulletin & Review. Oskamp, S. (1965). Overconfidence in case-study judgments. The journal of Consulting Psychology, 29, Puldford, B. D., & Colman, A, M. (1997). Overconfidence: Feedback and item difficulty effects. Personality and Individual Differences, 23, Yates, J. F. (1990). Judgment and decision making. Englewood Cliffs, NJ: Prentice Hall. 10

11 Appendix Experiment Instructions Instructions for TR1 (three sessions; 79 subjects) You are welcome and we thank you for your participation. In this event, you can earn money. The way to do it depends on your performance. You have assigned an ID number. We only need your age and gender as personal information. Your payments would take place next week by your ID number assignation. However, your performance would be given to you today once you finished the task proposed. Proposed task: You must answer 34 questions of a multiple choice item test where there s only one possible correct answer per item. The first 17 items present a noun and four possible definitions. The 17 remaining items, present a noun and four possible synonymous. For each correct answer you will accumulate $1 (Argentine Currency), without discounting any monetary value for the incorrect ones. You must answers the 34 questions! At the end of the questionnaire you must declare the number of questions that you believe have answered correctly. That is, the money you expected to have accumulated. We give you a help assistance sheet in where you can check the answers that you believe are correct. Instructions for TR2 (three sessions; 76 subjects) The instructions given in this treatment were the same than the first one (TR1), however we excluded the following sentence in the first paragraph: However, your performance would be given to you today once you finished the task proposed. 11