A Comparison of Three Probabilistic Models of Binary Discrete Choice Under Risk

Transcription

1 A Comparion of Three Probabilitic Model of Binary Dicrete Choice Under Rik by Nathaniel T. Wilcox * Abtract Thi paper compare the out-of-context predictive ucce of three probabilitic model of binary dicrete choice under rik. One of the model i the conventional homocedatic latent index or trong utility model that i widepread in applied econometric: Thi model i context-free in the ene that it error part i homocedatic with repect to deciion et. The other two model are alo latent index model, but their error part i heterocedatic with repect to deciion et, and in that ene are context-dependent model. Context-dependent model of choice under rik arie from everal different theoretical perpective. Here I conider my own contextual utility model (Wilcox 2009) and the deciion field theory model (Buemeyer and Townend 1993). A new experiment i performed on 80 ubject. Two-third of the data i ued to etimate model at the individual level, and thee etimate are ued to predict the remaining third of choice. The data i divided up o that the deciion et in the etimating data and the prediction data have interetingly different context. The context-dependent error model conitently outperform the context-free error model in prediction. JEL Claification Code: C25, C91, D81 Keyword: rik, dicrete choice, probabilitic choice, heterocedaticity, prediction. Preliminary Draft, March Pleae do not cite without aking me. * Economic Science Intitute, Chapman Univerity, and Department of Economic, Univerity of Houton. Phone , fax , [email protected]. I thank Stacey Jolderma for her excellent reearch aitance. 1

2 Beginning with Moteller and Nogee (1951), dozen of experiment on dicrete choice under rik etablihed that dicrete choice under rik appear to have a trong probabilitic, random or tochatic component. Thee experiment involve repeated trial of binary choice pair, and reveal ubtantial choice witching by the ame ubject between trial. In ome cae, the trial pan day (e.g. Tverky 1969; Hey and Orme 1994; Hey 2001) and one might worry that deciion-relevant condition may have changed between trial. Yet imilarly ubtantial witching occur even between trial eparated by bare minute, with no intervening change in wealth, background rik, or any other obviouly deciion-relevant variable (Camerer 1989; Starmer and Sugden 1989; Ballinger and Wilcox 1997; Loome and Sugden 1998). Since Kahneman and Tverky (1979) introduced Propect Theory, mot reearch on choice under rik ha concerned it tructure, that i the functional form or repreentation that decribe how lottery characteritic (outcome, event and their likelihood) are functionally combined to repreent binary preference direction. Econometrically, that dicuion concern the functional form taken by the fixed part of the latent index in a traditional dicrete choice model. However, there i reurgent interet in the tochatic part of deciion under rik. Thi ha been driven both by theoretical quetion and empirical finding. Theoretically, ome or all of what pae for an anomaly (ay, an apparent violation of expected utility or EU theory) can be attributed to tochatic model rather than the tructure in quetion (Wilcox 2008). Thi old point goe back at leat to Becker, DeGroot and Marchak (1963a,1963b) obervation that violation of the betweenne property of EU are precluded by ome probabilitic verion of EU (random preference) but allowed by other (trong utility). But thi general concern ha been reurrected by many writer; Loome (2005), Gul and Peendorfer (2006) and Blavatkyy (2007) are jut three relatively recent (but very different) example. 2

3 In thi paper I compare three probabilitic model of choice under rik. One of the model i the conventional homocedatic latent index or trong utility model that i widepread in applied econometric: Thi model i context-free in the ene that it random part i homocedatic with repect to deciion et. The other two model are alo latent index model, but their error part i heterocedatic with repect to deciion et, and in that ene thee two model are context-dependent. Context-dependent model of choice under rik arie from everal different theoretical perpective. Here I conider my own contextual utility model (Wilcox 2009) and the deciion field theory model of Buemeyer and Townend (1993). A new experiment i performed on 80 ubject. Two-third of the data i ued to etimate model at the individual level, and thee etimate are then ued to predict the remaining third of choice. The data i divided up o that the deciion et in the etimating data and the prediction data have interetingly different context. The context-dependent error model conitently outperform the context-free error model in prediction. 1. Preliminarie In my experiment, each choice pair i a et of two option,,,,. The option afe pay m dollar with certainty, while the option riky pay h dollar with probability q and l dollar with probability 1, where. Subject chooe between riky and afe in each pair preented to them. I call the vector of outcome,, the context of each pair. Figure 1 how an example pair where, 90,1/6,40, 50 and the context of the pair i 40,50,90. One familiar interpretation of each pecific context i that it name a pecific Machina/Marchak triangle (ee e.g. Machina 1987) repreenting all poible 3

4 pair of lotterie compoed olely from the three poible outcome 40, 50 and 90. Figure 2 how thi repreentation of the example pair in a Machina/Marchak triangle. I conider a cla of probabilitic choice model of the form V ( riky) V ( afe) (1) P Pr( riky) F, (, ) D riky afe where V ( riky) V ( afe) i a deciion-theoretic repreentation of the difference between the value of the option riky and afe, i a cale (or invere tandard deviation) parameter, D ( riky, afe) i an adjutment to the cale parameter in heterocedatic model, and F i a cumulative ditribution function or c.d.f. where F(0) = 0.5 and F(x) = 1 F( x). While my focu i on aumption about the function D ( riky, afe), I firt dicu the value difference V V ( riky) V ( afe). The function V need to be a deciion-theoretic repreentation of lottery value with theoretical breadth and empirical trength. Rank-dependent utility or RDU, developed by Quiggin (1982), Chew (1983) and many other, fit thi bill. Under RDU, the value of two-outcome option like riky, and ingle outcome option like afe, are (2) V riky w( q) u( h) [1 w( q)] u( l) and V afe u(m), where u (z) i the utility of outcome z; and w (q) i the weight aociated with the probability q of receiving the highet outcome h in the pair {riky,afe}. The RDU value difference between riky and afe in a pair i thu (3) RDU w( q) u( h) [1 w( q)] u( l) u( m). 4

5 RDU net the expected utility or EU repreentation: EU i jut that pecial cae of RDU where w( q) q. Therefore we develop all choice model below in term of RDU. To convert thoe into EU-baed model, jut replace w (q) by q in 3 to get (4) EU qu( h) (1 q) u( l) u( m), the EU value difference between riky and afe. Special experimental deign choice alo make the RDU repreentation inditinguihable from both Tverky and Kahneman (1992) cumulative propect theory (or CPT) and Savage (1954) ubjective expected utility (or SEU) repreentation. Cumulative propect theory differ from RDU only in it treatment of negative outcome or, more correctly, outcome below ome reference point (put differently, CPT poit lo averion), and my experiment pair contain only large poitive outcome of $40 to $120. In general, RDU i not a ubjective expected utility model ince the weight aociated with an outcome will in general change when the rank order of an outcome differ in two different lotterie, regardle of whether the event() generating that outcome are held contant acro thoe two lotterie. But if the mapping between event and outcome rank i held contant acro all lotterie, then SEU i inditinguihable from RDU. My experimental pair intentionally atify thi requirement a well. 1 Thi implie that the RDU repreentation in eq. 2 will be very broad, equivalent to (or neting) all of RDU, CPT, SEU, EU and EV (expected value). If we wihed to ditinguih between thee repreentation, thi deliberate confounding would be a bug, but here it i a feature 1 More concretely: In the experiment, lotterie riky all have probabilitie q of receiving their high outcome that are in ixth, generated by the roll of a ix-ided die. All lotterie are contructed o that q = k/6 i alway the roll 1 or 2 or k. So w(k/6), the weight on the high outcome h in riky, can alway be thought of a the ubjective probability of the event the die roll i 1 or 2 or k, while 1 w(k/6), the weight on the low outcome l in riky, can alway be thought of a the ubjective probability of the event the die roll i k+1 or k+2 or 6. 5

6 ince my interet lie with the cale adjutment D ( riky, afe). By experimental deign, the RDU repreentation of V V ( riky) V ( afe) will encompa thi wide et of deciion-theoretic repreentation, o all inference concerning D ( riky, afe) will be robut for thi et of deciion-theoretic repreentation. Deciion theory know the firt probabilitic model a the trong utility or SU model (Debreu 1958; Block and Marchak 1960; Luce and Suppe 1965), and econometric know thi a the homocedatic latent index model. It impoe the retriction D ( riky, afe) 1 on eq. 1, and with RDU it i (5) P rdu Pr( riky) F( RDU ). A i well-known (Luce 1959 ), if we chooe the logitic c.d.f. (x) a F(x), thi i equivalent to a binary logit with RDU lottery value: (6) P rdu exp( Vriky ) Pr( riky) exp( V ) exp( V riky afe ), with V riky and V afe a given in eq. 2. McFadden and other developed thi model in economic, and it appear widely in experimental/behavioral applied theory (e.g. McKelvey and Palfrey 1995; Camerer and Ho 1999). I ue the logitic c.d.f. a F in all my etimation for thi reaon, o that my reult peak clearly to thee application. The contextual utility or CU model (Wilcox 2009) et D( riky, afe) u( h) u( l), and with RDU it i 6

7 RDU (7) P rdcu F. u( h) u( l) Contextual utility make comparative rik averion propertie of the RDU repreentation and it tochatic implication conitent within and acro context. For repreentation uch a RDU and EU, utility function u (z) are only unique up to a ratio of difference: Intuitively, contextual utility exploit thi uniquene to create a correpondence between tructural and tochatic definition of comparative rik averion. To ee thi, conider any pair on a context. Under RDU and contextual utility, the choice probability in eq. 7 can be rewritten a (8) P rdcu F [ ( l, m, h) w( q)], where ( l, m, h) [ u( m) u( l)]/[ u( h) u( l)]. Thi probability i decreaing in the ratio of difference ( l, m, h). Conider two ubject Anne and Bob: Aume that they have identical weighting function (which include the cae where both have EU preference) and identical cale parameter. Alo aume that Bob i globally more rik avere than Anne in Pratt ene that Bob local abolute rik averion u ( z) / u ( z) exceed that of Anne for all z. The latter aumption, and imple algebra baed on Bob Anne Pratt (1964) main theorem, then implie that ( l, m, h) ( l, m, h), on all context, and a a reult (8) implie that Bob will have a lower probability than Anne of chooing riky on all context. Wilcox how that it i mathematically impoible for trong utility to hare thi property, and thi i the primary motivation behind the contextual utility model. The final model i deciion field theory or DFT (Buemeyer and Townend 1993). It et D( riky, afe) [ u( h) u( l)] w( q)[1 w( q)], and with RDU it i 7

8 (9) RDU P rddft F. [ u( h) u( l)] w( q)[1 w( q)] Note that eq. 9 i DFT only for pair like thoe found in thi experiment, where every pair conit of a two-outcome rik veru a ure outcome. In general, the function D( riky, afe) varie in a complex but theoretically well-pecified manner with deciion et. Notice too that in thi pecial cae DFT hare CU main property: Holding contant cale parameter and weighting function, globally greater rik averion (in the ene of Pratt) will imply a lower probability of chooing riky in all pair on all context. DFT ha another attractive property: A q approache zero (or one) that i, a afe (or riky) get cloer to tochatically dominating riky (or afe) the probability of chooing the (nearly) tochatically dominating alternative approache certainty. Buemeyer and Townend (1993) derive deciion field theory from a ophiticated computational logic, but a imple intuition can be given for the model. Suppoe that a deciion maker computational reource can effortlely and quickly provide utilitie of outcome, and alo uppoe the deciion maker wihe to chooe according to relative RDU value; but uppoe he doe not have an algorithm for effortlely and quickly multiplying utilitie and weight together. The deciion maker could proceed by ampling the poible utilitie in option in proportion to their deciion weight, keeping running um of thee ampled utilitie for each option, and top (and chooe) when the difference between the um exceed ome threhold determined by the cot of ampling. In eence, the choice probability in eq. 9 reult from thi kind of equential ampling deciion procedure, which can be traced back to Wald (1947). 8

9 The upercript in eq. 5, 7 and 9 (rdu, rdcu and rddft) index pecific combination of a deciion-theoretic repreentation and a tochatic model: The prefix rd denote a latent index baed on the RDU repreentation, while the uffixe u, cu and dft denote the three probabilitic model. Correponding EU-baed denotation are euu, eucu and eudft. Call thee pecification, and let pec tand for any one of them. The purpoe of thi tudy i to compare the out-ofcontext predictive power of thee pecification. The next ection decribe the experiment that collect the data for thi purpoe. After that, we will return to comparing the pecification. 2. Experimental Deign and Protocol The ubject in thi ummer 2008 experiment were 80 Univerity of Houton undergraduate tudent, recruited widely from regitered tudent by mean of a ingle undergraduate literv announcement. Each ubject wa individually cheduled for three eparate eion on three eparate day of their own chooing, almot alway finihing all three eion within one week. Only one ubject had to be replaced due to noncompletion of the three-day protocol. On each day, each ubject made choice from the 100 choice pair hown in Table 1, o that each made 300 choice in all by the end of their third day. On each day, the 100 choice pair were plit into two halve of 50 pair each, eparated by about ten to fifteen minute of other tak (demographic urvey, item repone urvey, hort tet of arithmetic and problem-olving ability, and o forth). Only rarely did any day eion lat more than an hour, and mot eion were ubtantially horter than thi. At the concluion of each ubject third day, one of their 300 choice pair wa elected at random (by mean of the ubject drawing a ticket from a bag) and the ubject wa paid according to their choice in that pair (thi i called random tak election). If the ubject choice in the elected pair wa riky, the ubject elected a ix-ided 9

10 die from a box of ix-ided dice (rolling them until atified if they wihed), and their elected die wa then rolled by the attendant to determine the payment. Here i the reaoning behind the protocol feature. I want to etimate utilitie and weight without aggregation aumption. Deciion theorie are about individual, not aggregate, and aggregation mutilate and detroy many obervable propertie of deciion theorie (Wilcox 2008). A large amount of choice data from each ubject i needed to etimate utilitie and weight with any preciion at the individual level. A ubject will become bored, and will become carele, if he make hundred of deciion at one itting. So the deciion are divided up acro three day, and on each day into two part eparated by unrelated tak providing a break from deciion. The eparation acro three day, in particular, introduce a rik that ome ubtantial event altering a ubject wealth or background rik will occur between day, which could arguably undermine the aumption that utilitie of outcome and hence choice probabilitie are tationary throughout the protocol. Thi i a rik I am willing to run to mitigate ubject boredom with hundred of choice tak, and I can check whether ditribution of riky choice proportion acro ubject appear to be tationary acro ubject three day of deciion. Figure 3 how thee ditribution. Although the firt day ditribution appear to have lightly le diperion, no parametric or nonparametric tet find any ignificant difference between thee three daily ditribution. The within-ubject difference between riky choice proportion on the firt and third day ha a zero mean by all one-ample tet. Finally, the firt principal component of riky choice proportion on the three day account for 95% of their collective variance, with no remotely ignificant econd principal component aociated with any particular day. There i no ign of nontationarity of choice probabilitie acro the three day. 10

11 Random tak election i meant to reult in truthful, motivated and unbiaed revelation of preference in each pair: That i, ubject hould make each of their 300 choice a if it wa the only choice being made, for real, and there hould be no portfolio or wealth effect making choice interdependent acro the tak. Both the independence axiom of EUT and the iolation effect of propect theory would imply thi. To ee thi for EUT, notice that the independence axiom in it unreduced compound form implie (riky with Prob = 1/300; Z with Prob = 299/300) if and only if (afe with Prob = 1/300; Z with Prob = 299/300) where Z i any other outcome or rik, including the grand lottery created by the ubject other 299 choice over the coure of thi experiment. Therefore, if ubject preference atify independence in thi unreduced compound form, random tak election hould be incentive compatible. Direct evidence doe ugget that preference generally atify the independence axiom in it unreduced compound form (Kahneman and Tverky 1979; Conlik 1989). Moreover, empirical examination of random tak election in binary lottery choice experiment find no ytematic choice difference between tak elected with relatively low or high probabilitie (Wilcox 1993) nor between tak preented ingly or under random tak election (Starmer and Sugden 1991), at leat for relatively imple tak like the pair ued here. Two competing iue urround the reolution of riky lottery outcome. On the one hand experimenter want random device to be concrete, obervable and credible. Thi i why we ue dice, card, bingo cage and o forth. We alo want ubject to have good reaon to believe thee 11

12 device are not rigged againt them: Thi i why ubject elect a die from an offered box of dice (and, if they wih, after rolling everal to tet them). However, the experimenter roll the elected die becaue ubject may believe they exercie control over the die (whether they truly can or not; ee e.g. Langer 1982). Here, the protocol compromie between the deire for credibility of randomizing device and the poibility of ubject belief in control over the die. The choice pair in Table 1 are organized into group of four tak (the row of the table) by their hared outcome context. All riky lotterie are chance q and 1-q (in ixth, generated by a ix-ided die) of receiving the high and low outcome h and l on the context, repectively: Four value of q hown in each row in Table 1 (q a, q b, q c and q d ) create four riky lotterie on each context, and each of thee i paired with afe (the middle outcome m of the context with certainty) to create four pair on the context. There are twenty-five ditinct context, all contructed from nine poitive money outcome ($40 to $120 in $10 increment). Multiple outcome context erve everal purpoe. Nonparametric identification of all utilitie and weight i impoible unle the ame event (the die roll) are matched with a multiplicity of outcome on different context. My nonparametric ambition here are high: In principle, I want to be able to etimate the utilitie and weight without functional form aumption. Multiple context improve the eparate identification of utilitie and weight. Additionally, the major difference between the context-free trong utility or SU model and the context-dependent DFT and CU model i how the latter model depend on u( h) u( l) through the D ( riky, afe) function. Therefore, the deign deliberately create a wide variety of context o that u( h) u( l) i expected to vary widely acro context: For intance, monotonicity of utilitie in outcome z implie that u( h) u( l) mut be greater on the context (40,60,110) than on 12

13 the context (50,70,100) (thee are context 19 and 20 in Table 1, repectively). Thi kind of variation i key to ditinguihing between the probabilitic model. Note alo that Table 1 divide the 25 context (and hence the 100 pair) into two dijoint et of context the in et of 16 context (64 pair) on the left, and the out et of 9 context (36 pair) on the right. For prediction comparion, I etimate pecification uing each ubject choice from the in choice pair (thi i 64 pair, for 192 obervation over three day), and ue thee etimate to predict each ubject choice from the out pair (thi i 36 pair, for 108 choice over three day). The prediction i, therefore, an out-of-context prediction ince the context of the in pair and the out pair are wholly ditinct. Finally, the choice of ixth a the probability unit for contructing rik erve everal purpoe. Firt, the ix-ided die i perhap the mot culturally familiar randomizing device: Thi reduce ome of the artificiality of laboratory rik. Second, ixth are well-uited to revealing a widely-believed hape of weighting function. Figure 4 how Prelec (1998) ingleparameter weighting function w( q ) exp( [ ln( q)] ) q (0,1), w(0)=0 and w(1)=1, at variou value of from 0.5 to 1, covering widely-held prior about the hape of the function. The linear function (heavy black line) i EU with = 1. Figure 4 how that the maximum downward deflection of the nonlinear verion (from linearity) occur very cloe to q = 5/6; and at q = 1/6 the upward deflection of nonlinear verion i about 75% of it maximum (which generally occur at a omewhat maller q). Finally, Monte Carlo imulation uggeted that relatively coare probability grid (fourth or ixth) over many different context permit relatively more precie etimation of utilitie and weight. 13

14 3. Etimation To dicu the etimation, it i helpful to define indice for pair, trial (day) and ubject, a well a ome important et of indice: i = 1,2, I, indexing I ditinct pair. Here I = 100. Pair i are then {( h i, qi, li ), mi}, or { riky i, afei }; and alo note that i = 1 to 64 are the in pair, and i = 65 to 100 are the out pair, in Table 1. t = 1,2, T, indexing T ditinct trial (day) of each pair. Here T = 3 (the three day). = 1,2, S, indexing the S ditinct ubject. Here S = 80. it: A double ubcript indicating the tth trial of choice pair i. r 1 if ubject choe riky i in her tth trial of pair i, and zero otherwie. it r et ( r it et), the oberved choice vector of ubject over thoe pair and trial in it et. The et will be in, out or all, where in = { it i 64 } (the etimation pair/trial), out = { it i 65 } (the prediction pair/trial) and all = {all it} i all 300 pair/trial. Let u (z) and w (q) denote utilitie of outcome z and weight aociated with probabilitie q, repectively, of ubject. The experiment involve nine ditinct outcome z [$40,$50,...,$110,$120] acro it 100 choice pair, o there are nine utilitie [ u (40), u (50),..., u (110), u (120)] for each ubject. Becaue of the affine tranformation invariance property of RDU and EU utilitie, we can arbitrarily chooe u ( 40) 0 and u ( 50) 1 for all ubject. So the unique utility vector u of each ubject i the utilitie of the even remaining outcome, 14

15 u [ u (60), u (70),..., u (110), u (120)]. The nonparametric treatment of utility make each of thoe even utilitie a eparate parameter to be etimated. I alo examine a 2-parameter parametric alternative, the expo-power function (Saha 1993, Holt and Laury 2002) which blend the CARA and CRRA utilitie in a flexible way, normalized o that u ( 40) 0 and u ( 50) 1. The experiment alo involve five ditinct probabilitie q [1/6,2 /6,3/ 6,4 /6,5/6], o there i a vector w of five weight to be etimated for each ubject, w [ w (1/6), w (2/6), w (3/6), w (4/6), w (5/6)]. The nonparametric approach again make each of thoe five weight a eparate parameter to be etimated. There are everal parametric weighting function in the literature uch a the Prelec (1998) one-parameter function dicued above, but none are very flexible. Intead, I ue the Beta ditribution c.d.f. (with two parameter and ) a my parametric alternative, that i, It i the right kind of function (taking the unit interval onto itelf, and monotone increaing) from the viewpoint of the general RDU repreentation theorem. It i much more flexible and (importantly) can take all major hape uggeted by the theorit who developed RDU and/or Cumulative Propect Theory. It i alo eaily called in mot nonlinear optimization oftware. and pair i, i To ummarize, the nonparametric latent index of the RDU repreentation, for ubject (10) RDU u, w ) w ( q ) u ( h ) [1 w ( q )] u ( l ) u ( m ), where i ( i i i i i w [ w (1/6), w (2/6), w (3/6), w (4/6), w (5/6)], u [ u (60), u (70),..., u (110), u (120)], and u ( 40) 0 and u ( 50)

16 Combining 10 with 5, 7 and 9, and chooing the logitic c.d.f. a F(x), we have the following choice probability pecification: rdu (11) P ( u, w, ) RDU ( u, w ) i. i (12) rdcu RDU (, ) i u w cu Pi ( u, w, ), where D cu i ( u ) u ( hi ) u ( li ). Di ( u ) (13) rddft RDU i ( u, w ) Pi ( u, w, ), where rddft Di ( u, w ) D rddft i ( u, w ) [ u ( hi ) u ( li )] w ( qi )[1 w ( qi )] Correponding EU-baed choice probabilitie with either the trong utility or contextual utility model imply omit the vector of weight w from the function argument and, in the cae of DFT, et w ( q ) q in the denominator expreion. i i Equation define the probability of the event r 1 (ubject choe riky in the tth pec trial of pair i). Letting P ( u, w, ) generically denote any of thoe probabilitie, the generic i it log likelihood of r, given ( u, w, ), i it pec pec pec (14) ( r u, w, ) r ln[ P ( u, w, )] (1 r )ln[1 P ( u, w, )]. it it i it i Therefore, the total log likelihood over any particular et (in, out or all), for ubject, i pec pec pec (15) ( ret u, w, ) r it ln[ Pi ( u, w, )] (1 rit )ln[1 Pi ( u, w, )] it et, 16

17 and etimation of ( u, w, ) by maximum likelihood, for each ubject, i traightforward. Beginning with Manki (1975), econometrician made ubtantial progre eliminating the need for chooing a pecific c.d.f. for F(x) in latent index model (e.g. Colett 1983; Klein and Spady 1993; Lewbel 2000). To my knowledge, none of thee innovative method can work with the fully nonparametric latent index of RDU (a in eq. 10). In eence, all regreor in eq. 10 are dummy variable (or interaction of dummy variable) indicating the preence or abence of particular outcome and event in any pair i. A i well-known in thi literature, dipening with knowledge of F(x) require a o-called pecial regreor with everal neceary propertie: In particular, the pecial regreor mut have an abolutely continuou ditribution. I do not believe dummy regreor and their interaction, a in eq. 10, have that property, o I don t believe I can ue thee innovative method here. Moreover, in DFT, the logitic ditribution arie a the limiting ditribution of the equential ampling proce a the time interval between ample get mall, and thi i a maintained aumption of DFT (Buemeyer and Townend 1993). In other word, the choice of the logitic c.d.f a F(x) ha a good theoretical motivation, at leat from the perpective of one of the probabilitic model. There i ome interet in etimating ( u, w, ) uing all of the data. Thi i particularly true for the fully nonparametric latent index model, ince thee provide a firt look at function-free utilitie and weight etimated imultaneouly from dicrete choice data. (I do not think thi ha been done before.) But for the purpoe of comparing the probabilitic model, I prefer to compare their out-of-context prediction quality. To do that, I etimate uing jut the in data, and ue thi etimate to predict choice on the out data. Let r in and r out denote the in and out choice vector for each of the 80 ubject. Etimate each pecification uing jut the data r in, producing pair of etimated parameter vector ˆ pec2, and ˆ for any two pec1, in in 17

18 pecification pec1 and pec2, for each. Then uing thee etimate, the out et of data and eq. 15, we can calculate log likelihood of the prediction baed on the in et etimate, r out, pec1 pec1 (16) ( ) ( ˆpec1, pec2 pec2 out rout in ), and ( ) ( ˆpec2, out rout in ). I ue Vuong (1989) tet to compare thee prediction log likelihood for pair of pecification pec1 and pec2. It allow non-neted pecification and neither need to be correct (the tet i againt the null The pecification are equally cloe to the true DGP ). Intuitively, the tet treat difference between the log likelihood of two pecification a normal variate and doe a z tet on them. Define pec1 pec2 ( pec1, pec2) ( ) ( ), out out out 80 ( pec 1, pec2) 1 out out ( pec1, pec2) /80, and SD out ( pec1, pec2) ( pec1, pec2) ( pec1, pec2) / 80 out out. Vuong tet tatitic i then (17) z out ( pec1, pec2) SD out out ( pec1, pec2) ( pec1, pec2) /, aymptotically N(0,1) under the null

19 4. Reult Table 2 how the even repreentation (and treatment of utilitie and/or weight) that I combine with the three probabilitic model to generate twenty-one pecification. All even repreentation of the latent index are neted within eq. 10, the fully nonparametric RDU latent index. By retricting eq. 10 to have either linear weight, linear utilitie or both, we get (repectively) expected utility or EU, Yaari (1987) Dual Theory or Yaari, and expected value. RDU, EU and Yaari may be further retricted by replacing the nonparametric treatment of their utilitie and/or weight by the 2-parameter function dicued earlier (the expo-power function for utilitie and/or the Beta c.d.f. for weight). Figure 5 and 6 graph the 80 weight and utility function (repectively) etimated uing all of the data, the fully nonparametric RDU latent index in eq. 10, and the eq. 12 (CU) probability model. Figure 6 how that the vat majority of utility function are uniformly concave with perhap a handful of exception. However, Figure 5 make it clear that there i great heterogeneity of weight function hape, o both Figure 5 and 6 employ a common colorcoding for ubject with alient weight function hape. Comparing Figure 4 and 5, it i curiou but true that very few of the etimated weight function appear to have the characteritic invere hape poited by many theorit. In Figure 5, thi expected hape i coded green: Although 14 of 80 weight function have thi general hape, mot of thoe 14 cro the diagonal at a relatively high q above one-half, which i not true (for intance) of the one-parameter Prelec (1998) function hown in Figure 4 (thi function alway croe the diagonal at 0.37). The plurality of etimated weight function (30 of the 80, coded blue in Figure 5) are uniformly concave and above the identity weight of EU, which Quiggin (1993) call optimim. A very mall number (4 of 80, coded red in Figure 5) are uniformly convex and below identity 19

20 weight, which Quiggin call peimim. Finally, the econd-mot-common etimated weight function (26 of 80, coded orange in Figure 5) i -haped. One interpretation i that thee people tend to round low probabilitie to zero and high probabilitie to unity, o one might call thee people approximator. Comparion of Figure 5 and 6 ugget an odd relationhip: The blue-coded optimit (concave weight function) alo tend to have the greatet concavity of their utility function. Thi ound counterintuitive ince more concave utility mean greater rik averion while more concave weight (optimim) mean le rik averion. Part of the anwer to thi puzzle i undoubtedly poor identification for relatively rik-avere ubject. A a ubject become increaingly rik-avere overall (for whatever reaon), there will be le variation in her choice and eparate identification of weight and utilitie will be relatively poor. For uch ubject, a poitive finite ample correlation between etimated utility and weight concavity will be expected. Monte Carlo imulation of information matrice of relatively rik-avere RDU/CU data-generating procee confirm thi econometric intuition. Therefore I ugget that thi apparent finding be taken with a grain of alt. Figure 7 diplay the prediction comparion between pecification in a graphical manner. The figure diplay a linearly tranformed verion of average prediction log likelihood, frontier pec out out pec pec (18) Percent prediction metric =, where ( ) / 80 frontier evu out out. out out The pecification evu i an expected value latent index with the SU probability model: Thi model ha jut one etimated parameter for each ubject (the cale parameter ) and I treat thi a an upper prediction benchmark. The pecification frontier i not a model at all: Thi i imply 20

21 the bet poible out-of-context prediction likelihood with tationary choice probabilitie. Since the experimental deign involve repeated trial, and ince there i ome inconitency of ubject choice acro repeated trial, thi bet poible likelihood i nonzero for each ubject, and o i it average acro ubject. No model with tationary probabilitie can predict better than thi. If thi metric take the value 100%, the pecification predict no better than a trong utility expected value model. If thi metric take the value 0%, the pecification predict a well a any poible pecification with tationary probabilitie can predict. Figure 7 how that pecification with 2-parameter parametric treatment of utilitie and/or weight, near the center of the figure, perform bet in out-of-context prediction. Thi i not urpriing ince thee pecification involve many fewer parameter than the fully nonparametric pecification. Yet even the fully nonparametric pecification at the left almot alway predict noticeably better than the maximally lean EV (expected value) pecification at the far right (the ole exception here being the nonparametric Yaari repreentation with DFT). There are two other important finding. Firt, notice that holding repreentation contant, the context-dependent probability model CU and DFT are almot uniformly better at prediction than the context-free trong utility or SU model. Second, examine Figure 7 comparative reult for EU and RDU repreentation with parametric utilitie and weight (near the center of Figure 7, labeled EU 3 parm para and RDU 5 parm para ). Conider the 3-parameter EU pecification with trong utility a a baeline. Notice that the improvement in prediction quality aociated with changing from trong utility to CU or DFT (about 12% in both cae) exceed the improvement in prediction quality aociated with adding two parameter and changing to the 5-parameter RDU pecification but taying with trong utility (about 9%). Thi confirm Wilcox (2008,2009) reult uing the well-known Hey and Orme (1994) data et: When we ak 21

22 what matter for prediction? it eem that the greatet marginal gain from a trong utility and EU tarting point accrue from changing the probabilitic model rather than adopting a more expanive repreentation uch a RDU. Table 3 (A, B, C and D) how the reult of the Vuong (1989) tet that compare the probabilitic model. Each of thee table hold the repreentation contant and jut compare change in probabilitic model. Table 3-A, 3-B, 3-C and 3-D how reult for expected value, expected utility, Yaari dual model and rank-dependent utility, repectively. The table all have left and right panel with in-context and out-of-context comparion of log likelihood. Additionally, except for table 3-A (expected value repreentation), the table all have top and bottom panel howing reult for the parametric and nonparametric verion of each repreentation. With the exception of Table 3-C (the Yaari dual model repreentation), the Vuong tet overwhelmingly and uniformly reject trong utility in favor of both of the contextdependent probability model. Even in Table 3-C (the Yaari repreentation), where trong utility i occaionally directionally better than the context-dependent model, it i only ignificantly better in a ingle comparion againt DFT. The contextual utility probability model i alway ignificantly better than trong utility, regardle of the repreentation or parametric expanivene of the pecification. The Vuong tet do not ditinguih DFT and contextual utility in any conitent and peruaive way. Among the even in-context comparion between them, each i ignificantly bet in three comparion (with one inignificant comparion), and among the even out-ofcontext comparion, contextual utility ignificantly beat DFT in three comparion and DFT ignificantly beat contextual utility in one comparion (with three inignificant comparion). 22

23 5. Concluion For the purpoe of predicting riky choice in new context, context-dependent model like contextual utility (Wilcox 2009) and deciion field theory (Buemeyer and Townend 1993) are overwhelmingly better than the traditional trong utility approach that dominate applied theoretical work in experimental and behavioral economic and much applied econometric work a well. Moreover, the greatet marginal gain in predictive ucce do not come from more expanive deciion-theoretic repreentation that add parameter to be etimated: Intead they come from witching from context-free to context-dependent model, with no additional parameter to be etimated. 23

24 Reference Ballinger, T. P., and N. Wilcox, 1997, Deciion, error and heterogeneity. Economic Journal 107, Becker, G. M., M. H. DeGroot and J. Marchak, 1963a, Stochatic model of choice behavior. Behavioral Science 8, Becker, G. M., M. H. DeGroot and J. Marchak, 1963b, An experimental tudy of ome tochatic model for wager. Behavioral Science 8, Blavatkyy, P. R., 2007, Stochatic expected utility theory. Journal of Rik and Uncertainty 34, Block, H. D. and J. Marchak, 1960, Random ordering and tochatic theorie of repone, in I. Olkin et al., (Ed.), Contribution to probability and tatitic: Eay in honor of Harold Hotelling. Stanford Univerity Pre, Stanford, pp Buemeyer, J. and J. Townend, 1993, Deciion field theory: A dynamic-cognitive approach to deciion making in an uncertain environment. Pychological Review 100, Camerer, C., 1989, An experimental tet of everal generalized expected utility theorie. Journal of Rik and Uncertainty 2, Camerer, C. and T-H. Ho, 1999, Experience weighted attraction learning in normal-form game. Econometrica 67: Chew, S. H., 1983, A generalization of the quailinear mean with application to the meaurement of income inequality and deciion theory reolving the Allai paradox. Econometrica 51, Conlik, J., 1989, Three variant on the Allai example. American Economic Review 79,

25 Colett, S. R., 1983, Ditribution-free maximum likelihood etimator of the binary choice model, Econometrica 51, pp Debreu, G., 1958, Stochatic choice and cardinal utility. Econometrica 26, Gul, F., and W. Peendorfer, 2006, Random expected utility. Econometrica 74, Hey, J. D., 2001, Doe repetition improve conitency? Experimental Economic 4, Hey, J. D. and C. Orme, 1994, Invetigating parimoniou generalization of expected utility theory uing experimental data. Econometrica 62, Holt, C. A. and S. K. Laury, 2002, Rik averion and incentive effect. American Economic Review 92, Kahneman, D. and A. Tverky, 1979, Propect theory: An analyi of deciion under rik. Econometrica 47, Klein, R. W. and R. H. Spady, 1993, An efficient emiparametric etimator for binary repone model. Econometrica 61, Langer, E. J., 1982, The illuion of control. In D. Kahneman, P. Slovic and A. Tverky, ed., Judgment Under Uncertainty: Heuritic and Biae. New York: Cambridge Univerity Pre. Lewbel, A., 2000, Semiparametric qualitative repone model etimation with unknown heterocedaticity or intrumental variable. Journal of Econometric 97, Loome, G., 2005, Modeling the tochatic component of behaviour in experiment: Some iue for the interpretation of data. Experimental Economic 8, Loome, G. and R. Sugden, 1998, Teting different tochatic pecification of riky choice. Economica 65, Luce, R. D., Individual Choice Behavior: A Theoretical Analyi. New York: Wiley. 25

26 Luce, R. D. and P. Suppe, 1965, Preference, utility and ubjective probability, in R. D. Luce, R. R. Buh and E. Galanter, (Ed.), Handbook of mathematical pychology Vol. III. Wiley, New York, pp Machina, M., 1987, Choice under uncertainty: Problem olved and unolved. Journal of Economic Perpective 1, Manki, C., 1975, Maximum core etimation of the tochatic utility model of choice, Journal of Econometric 3, McKelvey, R. and T. Palfrey, 1995, Quantal repone equilibria for normal form game. Game and Economic Behavior 10, Moteller, F. and P. Nogee, 1951, An experimental meaurement of utility. Journal of Political Economy 59, Pratt, J. W., 1964, Rik averion in the mall and in the large. Econometrica 32, Prelec, D., 1998, The probability weighting function. Econometrica 66, Quiggin, J, 1982, A theory of anticipated utility. Journal of Economic Behavior and Organization 3, Quiggin, J, Generalized Expected Utility Theory: The Rank-Dependent Model. Norwell, MS: Kluwer. Saha, A, 1993, Expo-Power Utility: A Flexible Form for Abolute and Relative Rik Averion. American Journal of Agricultural Economic 75, Savage, L. J., The Foundation of Statitic. New York: Wiley. Starmer, C. and R. Sugden, 1989, Probability and juxtapoition effect: An experimental invetigation of the common ratio effect. Journal of Rik and Uncertainty 2,

27 Starmer, C. and R. Sugden, 1991, Doe the random-lottery incentive ytem elicit true preference? An experimental invetigation. American Economic Review 81, Tverky, A., 1969, Intranitivity of preference. Pychological Review 76, Tverky, A. and D. Kahneman, 1992, Advance in propect theory: Cumulative repreentation of uncertainty. Journal of Rik and Uncertainty 5, Vuong, Q., 1989, Likelihood ratio tet for model election and non-neted hypothee. Econometrica 57, Wald, A., Sequential Analyi. New York: Wiley. Wilcox, N., 1993, Lottery choice: Incentive, complexity and deciion time. Economic Journal 103, Wilcox, N., 2008, Stochatic model for binary dicrete choice under rik: A critical tochatic modeling primer and econometric comparion. In J. C. Cox and G. W. Harrion, ed., Reearch in Experimental Economic Vol. 12: Rik Averion in Experiment pp Bingley, UK: Emerald. Wilcox, N., 2009, Stochatically more rik avere: A contextual theory of tochatic dicrete choice under rik. Journal of Econometric doi: /j.jeconom Yaari, M. E., 1987, The dual theory of choice under rik. Econometrica 55,

28 Table 1: The 100 Choice Pair the in et for etimation, and the out et for prediction. The in et of 64 pair 4 pair on each of 16 context (for etimation) The out et of 36 pair 4 pair on each of 9 context (ue etimate to predict here) the in context four pair the out context four pair # (l,m,h) q a q b q c q d # (l,m,h) q a q b q c q d 1 (40,50,80) 5/6 4/6 3/6 2/6 17 (40,50,60) 4/6 3/6 2/6 1/6 2 (40,50,90) 5/6 4/6 3/6 2/6 18 (40,50,70) 4/6 3/6 2/6 1/6 3 (40,60,100) 5/6 4/6 3/6 2/6 19 (40,60,110) 5/6 4/6 3/6 2/6 4 (40,60,120) 5/6 4/6 3/6 2/6 20 (50,70,100) 4/6 3/6 2/6 1/6 5 (50,60,90) 5/6 4/6 3/6 2/6 21 (60,80,110) 4/6 3/6 2/6 1/6 6 (50,70,110) 5/6 4/6 3/6 2/6 22 (70,90,110) 4/6 3/6 2/6 1/6 7 (50,70,120) 5/6 4/6 3/6 2/6 23 (80,90,120) 5/6 4/6 3/6 2/6 8 (60,70,90) 4/6 3/6 2/6 1/6 24 (80,100,120) 4/6 3/6 2/6 1/6 9 (60,80,120) 5/6 4/6 3/6 2/6 25 (90,100,120) 4/6 3/6 2/6 1/6 10 (70,80,100) 4/6 3/6 2/6 1/6 11 (70,80,110) 5/6 4/6 3/6 2/6 12 (70,80,120) 5/6 4/6 3/6 2/6 13 (80,90,100) 4/6 3/6 2/6 1/6 14 (80,90,110) 4/6 3/6 2/6 1/6 15 (90,100,110) 4/6 3/6 2/6 1/6 16 (100,110,120) 4/6 3/6 2/6 1/6 28

29 Table 2. The even repreentation etimated, including the treatment of utility and/or weigh function and the reulting number of parameter etimated. Each of thee i combined with one of the three probability model to produce twenty-one pecification in all, and each pecification involve the etimation of one extra parameter, the cale parameter. Structure Utility function treatment Weight function treatment Expected Value (EV) Expected Utility (EU) nonparametric (7 parm.) Expected Utility (EU) expo-power (2 parm.) Yaari Dual Theory (Yaari) nonparametric (5 parm.) Yaari Dual Theory (Yaari) beta c.d.f. (2 parm.) Rank-Dependent Utility (RDU) nonparametric (7 parm.) nonparametric (5 parm.) Rank-Dependent Utility (RDU) expo-power (2 parm.) beta c.d.f. (2 parm.) 29

30 Table 3-A. Voung comparion between the pecification: Expected value repreentation. Repreentation: Expected Value (1 parameter) Explanatory performance (in-context fit) Log Likelihood on the in data et, uing the in parameter etimate Deciion Strong Field Theory Utility Contextual z = 4.18 z = 5.86 Utility p = 4.2x10-5 p = 2.7x10-9 Deciion Field z = 6.14 Theory p = 4.2x10-10 Predictive performance (out-of-context fit) Log Likelihood on the out data et, uing the in parameter etimate Deciion Field Strong Utility Theory Contextual z = 2.51 z = 7.43 Utility p = x10-14 Deciion Field z = 6.82 Theory p = 4.5x10-12 Note: All comparion are baed on parameter etimated from the 192 obervation of the in data. The in-context fit comparion compare the maximized log likelihood of thoe 192 obervation. The out-of-context fit comparion ue the parameter etimate to calculate log likelihood in the out data, and compare thoe log likelihood. In each cell, Vuong z tatitic i baed on the difference between the row and column probability model log likelihood, o negative z indicate that the column model fit bet while poitive z indicate that the row model fit bet. 30

31 Table 3-B. Vuong comparion between the pecification: Expected utility repreentation. Repreentation: Expected Utility (expo-power utilitie, 3 parameter) Explanatory performance (in-context fit) Log Likelihood on the in data et, uing the in parameter etimate Deciion Strong Field Theory Utility Contextual z = 2.00 z = 6.02 Utility p = p = 8.8x10-10 Deciion z = 5.86 Field Theory p = 2.3x10-9 Predictive performance (out-of-context fit) Log Likelihood on the out data et, uing the in parameter etimate Deciion Field Strong Utility Theory Contextual z = 0.20 z = 5.18 Utility p = 0.42 p = 1.1x10 7 Deciion Field z = 5.72 Theory p = 5.1x10 9 Repreentation: Expected Utility (nonparametric utilitie, 8 parameter) Explanatory performance (in-context fit) Log Likelihood on the in data et, uing the in parameter etimate Deciion Strong Field Theory Utility Contextual z = 1.99 z = 6.02 Utility p = p = 8.8x10 10 Deciion z = 5.86 Field Theory p = 2.3x10 9 Predictive performance (out-of-context fit) Log Likelihood on the out data et, uing the in parameter etimate Deciion Field Strong Utility Theory Contextual z = 0.20 z = 5.18 Utility p = 0.42 p = 1.1x10-7 Deciion Field z = 5.73 Theory p = 5.1x10-9 Note: All comparion are baed on parameter etimated from the 192 obervation of the in data. The in-context fit comparion compare the maximized log likelihood of thoe 192 obervation. The out-of-context fit comparion ue the parameter etimate to calculate log likelihood in the out data, and compare thoe log likelihood. In each cell, Vuong z tatitic i baed on the difference between the row and column probability model log likelihood, o negative z indicate that the column model fit bet while poitive z indicate that the row model fit bet. 31

32 Table 3-C. Vuong comparion between the pecification: Yaari dual theory repreentation. Repreentation: Yaari (beta c.d.f weight, 3 parameter) Explanatory performance (in-context fit) Log Likelihood on the in data et, uing the in parameter etimate Deciion Strong Field Theory Utility Contextual z = 0.07 z = 0.65 Utility p = 0.47 p = 0.26 Deciion z = 0.66 Field Theory p = 0.25 Predictive performance (out-of-context fit) Log Likelihood on the out data et, uing the in parameter etimate Deciion Field Strong Utility Theory Contextual z = 2.22 z = 1.76 Utility p = p = Deciion Field z = 0.77 Theory p = 0.22 Repreentation: Yaari (nonparametric weight, 6 parameter) Explanatory performance (in-context fit) Log Likelihood on the in data et, uing the in parameter etimate Deciion Strong Field Theory Utility Contextual z = 7.84 z = 1.26 Utility p = 2.3x10 15 p = 0.10 Deciion z = 4.95 Field Theory p = 3.7x10 7 Predictive performance (out-of-context fit) Log Likelihood on the out data et, uing the in parameter etimate Deciion Field Strong Utility Theory Contextual z = 4.98 z = 4.20 Utility p = 3.2x10-7 p = 1.3x10-5 Deciion Field z = 4.16 Theory p = 1.6x10-5 Note: All comparion are baed on parameter etimated from the 192 obervation of the in data. The in-context fit comparion compare the maximized log likelihood of thoe 192 obervation. The out-of-context fit comparion ue the parameter etimate to calculate log likelihood in the out data, and compare thoe log likelihood. In each cell, Vuong z tatitic i baed on the difference between the row and column probability model log likelihood, o negative z indicate that the column model fit bet while poitive z indicate that the row model fit bet. 32

33 Table 3-D. Vuong comparion between pecification: Rank-dependent utility repreentation. Repreentation: Rank-Dependent Utility (expo-power utility, beta c.d.f. weight, 5 parameter) Explanatory performance (in-context fit) Log Likelihood on the in data et, uing the in parameter etimate Deciion Strong Field Theory Utility Contextual z = 4.63 z = 6.63 Utility p = 1.8x10-6 p = 1.6x10-11 Deciion z = 4.17 Field Theory p = 1.5x10-5 Predictive performance (out-of-context fit) Log Likelihood on the out data et, uing the in parameter etimate Deciion Field Strong Utility Theory Contextual z = 0.13 z = 6.35 Utility p = 0.45 p = 1.0x10-10 Deciion Field z = 5.54 Theory p = 1.5x10-8 Repreentation: Rank-Dependent Utility (nonparametric utilitie and weight, 13 parameter) Explanatory performance (in-context fit) Log Likelihood on the in data et, uing the in parameter etimate Deciion Strong Field Theory Utility Contextual z = 2.85 z = 5.82 Utility p = p = 3.0x10-9 Deciion z = 6.37 Field Theory p = 9.6x10 11 Predictive performance (out-of-context fit) Log Likelihood on the out data et, uing the in parameter etimate Deciion Field Strong Utility Theory Contextual z = 1.78 z = 4.32 Utility p = p = 7.9x10-6 Deciion Field z = 3.05 Theory p = Note: All comparion are baed on parameter etimated from the 192 obervation of the in data. The in-context fit comparion compare the maximized log likelihood of thoe 192 obervation. The out-of-context fit comparion ue the parameter etimate to calculate log likelihood in the out data, and compare thoe log likelihood. In each cell, Vuong z tatitic i baed on the difference between the row and column probability model log likelihood, o negative z indicate that the column model fit bet while poitive z indicate that the row model fit bet. 33

34 Figure 1. An example pair, howing it diplay to ubject. The context in thi example i ( 40,50,90) (in U.S. dollar). Left option [ riky ] Right option [ afe ] Generally, riky i ( h, q, l), where h > l, q = Pr(h) and 1 q = Pr(l) Here, h = $90, q = 1/6 and l = $40. Generally, afe i m with Prob 1, where h > m > l Here m = $50. 34

35 Figure 2. The example pair a a pair of coordinate in a Machina/Marchak triangle. Example Pair in the Machina Marchak triangle repreenting all pair on it context p h = probability of receiving high outcome 90 5/6 2/3 1/2 1/3 1/6 0 afe 1 p h p l = probability of receiving middle outcome 50. example pair riky 0 1/6 2/6 3/6 4/6 5/6 1 p l = probability of receiving low outcome 40 35

36 Figure 3: Cumulative Ditribution of Riky Choice Proportion Acro Subject, by Day. 100 Cumulative Ditribution of Riky Choice Proportion by Day Percentage of 80 Subject day 1 day 2 day Proportion of Choice of Riky from the 100 Pair 36

37 Figure 4. The Prelec one-parameter weighting function with value of from 0.5 to 1. Prelec-type one-parameter weighting function probability weight on highet outcome in pair /6 5/6 probability of highet outcome in pair 37

38 Figure 5: 80 individually etimated probability weighting function. Weight aociated with probability q Probability q of receiving high outcome in riky Blue = 30 Optimit (above identity line). Red = 4 Peimit (below identity line). Green = 14 Propect Theorit (initially above, then below identity line). Orange = 26 Approximator (initially below, then above identity line). Yellow = 6 Other (croe 38

39 Figure 6: 80 individually etimated utility function, color-coded to denote weighting function type in Figure 5. Utility of outcome z with u(40)=0 and u(120)= Outcome z in U.S. Dollar Blue = 30 Optimit (above identity line). Red = 4 Peimit (below identity line). Green = 14 Propect Theorit (initially above, then below identity line). Orange = 26 Approximator (initially below, then above identity line). Yellow = 6 Other (croe identity line more than once). 39

40 Figure 7: Percent prediction metric for twenty-one pecification, by repreentation and probability model. Comparion of prediction (out of ample) negative log likelihood of variou pecification: Average of individual etimation Percent Prediction Metric (lower i better) contextual dft trong trong dft contextual Structure (utility function treatment, weighting function treatment) 40