Optimal casino betting: why lucky coin and good memory are important

Transcription

1 Optimal casino betting: why lucky coin and good memory are important Xue Dong He Sang Hu Jan Ob lój Xun Yu Zhou First version: May 24, 2013, This version: December 7, 2014 Abstract We consider the dynamic casino gambling model initially proposed by Barberis (2012) and study the optimal stopping strategy of a pre-committing behavioral gambler endowed with the cumulative prospect theory preferences. We show that it is optimal to build decisions based on the whole past betting history and not only on the current wealth. In addition, the optimal strategy has to further rely on an independent randomization. We show explicitly how the strategies computed in Barberis (2012) can be strictly improved using this procedure. Then, we develop a systematic and analytical approach to solve the underlying gambling model based on a change of decision variable and Skorokhod embedding techniques. It follows that in most cases only randomized strategies may be optimal. Key words: casino gambling, cumulative prospect theory, path-dependence, randomized strategies, optimal stopping The main results in this paper are contained in the PhD thesis of Sang Hu which was submitted to the Chinese University of Hong Kong (CUHK) in September The authors thank Nick Barberis for his helpful comments on an earlier version of the paper. The results were also announced and presented at the Stochastic Dynamics in Economics and Finance Trimester Seminar held in Hausdorff Research Institute for Mathematics in August 2013, as well as at the Second NUS Workshop on Risk & Regulation held in National University of Singapore in January Comments from the participants of these events are gratefully acknowledged. Department of Industrial Engineering and Operations Research, Columbia University, S. W. Mudd Building, 500 W. 120th Street, New York, NY 10027, US. xh2140@columbia.edu. Risk Management Institute, National University of Singapore, 21 Heng Mui Keng Terrace, Singapre rmihsa@nus.edu.sg. Mathematical Institute, the Oxford-Man Institute of Quantitative Finance and St John s College, University of Oxford, Oxford, UK. Jan.Obloj@maths.ox.ac.uk. Part of this research was completed while this author was visiting CUHK in March 2013 and he is grateful for the support from the host. He also gratefully acknowledges support from ERC Starting Grant RobustFinMath Mathematical Institute, The University of Oxford, Woodstock Road, OX2 6GG Oxford, UK, and Oxford Man Institute of Quantitative Finance, University of Oxford. zhouxy@maths.ox.ac.uk. This author acknowledges financial support from research funds at the University of Oxford and the Oxford Man Institute of Quantitative Finance. 1

2 1 Introduction Gambling is the wagering of money on an event with uncertain outcomes. Casino gambling is an important gambling type that enjoys huge popularity. By its very nature, casino gambling is an act of risk seeking since a typical bet at any casino has strictly negative expected value. Such risk-loving behaviors cannot be accommodated within the classical economics models which use expected utility framework with a concave utility function. A person endowed with a concave utility function shall never enter into a casino 1. In contrast, behavioral economic models offer a promising alternative to study and explain risk-seeking strategies. Barberis (2012) was the first to employ the cumulative prospect theory (CPT) of Kahneman and Tversky (1992) to model and study a dynamic casino gambling problem. In his model, a gambler comes to a casino at time 0 and is offered a bet with an equal chance to win or lose $1. If the gambler accepts the bet, it is then played out and the gambler either gains $1 or loses $1 at time 1, when he is offered the same bet again. At this point of time he can choose to exit the casino or to continue gambling. If he continues, the bet is offered again at time 2, and so on. Whether the gambler will enter the casino to play the game at time 0 depends on his preferences. In Barberis (2012), the gambler s risk preferences are represented by CPT. This theory endows the agent with an S-shaped utility function and two inverse-s shaped probability weighting functions. The latter typically overweight tails of a distribution and consequently the gambler overestimates very large profit. If the gambler initially decides to enter the casino, he needs to plan the optimal strategy, i.e. the optimal time to stop gambling, as seen at time 0. However, as discussed at great length by Barberis (2012), due to the probability weighting, the underlying dynamic optimization problem is time inconsistent in the sense that the initial optimal stopping strategy may no longer be optimal at t = 1 or afterwards. If the gambler is able to uphold his initial plan then he is called a pre-committed gambler and the strategy a pre-commitment one. Barberis (2012) assumes the gambler can only stay in the casino for no more than five periods and then finds the optimal exit time from the casino by enumerating all possible path-independent (Markovian) stopping strategies, i.e. decisions based on state but not 1 We note that some models, such as in Conlisk (1993), try to explain gambling by introducing an additional utility of gambling and appending it to the expected utility model. This however appears more an ad hoc approach rather than a systematic way to explain general risk-seeking phenomena exhibited in gambling. 2

3 on the path which has led to it. One of the main conclusions therein is that with some parameter values the gambler chooses to enter the casino and designs the strategy so that the payoff of his cumulative gains and losses at the time when he leaves the casino resembles that of a lottery ticket. Therefore, the effect of overweighting tails of a lotterylike probability distribution with high skewness becomes significant. If the gambler is able to commit to his initial exit strategy and does not revisit the problem in the future, e.g. by using a commitment device (see Barberis (2012)), then this strategy can be implemented as planned. If the gambler does not follow the pre-committed strategy in the future and reoptimizes at each time, he is called a naive gambler. Barberis (2012) shows that the strategy realized by a naive gambler can be entirely different from the pre-commitment strategy. An important restriction of Barberis model is that only Markovian strategies are being considered, whereas there is a continuum of more complex decision rules available to the gambler. Indeed, in practice the whole betting history may be relevant, and sometimes critical, for deciding whether a person will continue gambling. In other words, a person can base his decision to stop or continue not only on the current state (wealth) but also on the whole past of his winnings which have led to it. We call such strategies path-dependent. Moreover, the agent may want to toss a few coins and make decisions based on the outcomes. Strategies based not only on the betting history but also on some independent random outcomes are called randomized. From an economics point of view, the necessity to use randomized decision rules is twofold. Firstly, it is analogous to the way randomized strategies are used in (discrete-time) game theory to achieve Nash equilibria. On any finite time horizon, there are only finitely many strategies based solely on the winnings history due to the discrete nature of both time and space of the underlying model. Such strategies can only generate certain types of winnings distributions and are a constraint for a typical agent facing a binary decision at each discrete time step. 2 Randomization will then help reach all the possible final wealth distributions and, as a consequence, a behavioral gambler will find it optimal to toss (possibly different) coins every time and then to help with decision making. Secondly, randomization is consistent with the risk-seeking component in the gambler s CPT preferences, namely 2 Mathematically speaking, only relatively few distributions, a fractal subset of all centered distributions, may be realized by stopping a random walk at a stopping time relative to its natural filtration, provided doubling strategies are excluded or that equivalently the expectation of the length of the time in casino is finite, see Cox and Ob lój (2008). 3

4 both the convex part of the value function and the exaggeration of the small probabilities of big wins. It is therefore necessary to include randomization to improve the overall CPT value. The main contributions of our work are the following. First, we show via simple numerical examples that complex strategies strictly outperform the simpler ones. Pathdependent strategies outperform Markovian ones and randomized strategies outperform path-dependent ones. Second, we develop a systematic approach (together with a companion paper He et al. (2014)) to solving the underlying optimal exit problem. In Barberis (2012), an exhaustive search was used to obtain optimal strategies. While enumerating simple strategies in a five period model is feasible, the effort grows exponentially as the time horizon increases. It is therefore necessary to devise an analytical machinery to solve the model. Formally, we consider the problem of finding the optimal pre-commitment strategy on an unrestricted time horizon and formulate it as an optimal stopping problem. Because the objective function of the problem, i.e. the preference value of the agent s cumulative gain or loss, involves two nonlinear probability weighting functions, the classical Snell envelope approach to solving optimal stopping problems is not applicable. Instead we change the decision variable to the distribution of the terminal wealth. The corresponding abstract theoretical problem is an infinite dimensional program and is solved analytically in He et al. (2014). We show in Theorem 1 below, using techniques of Skorokhod embedding (see e.g. Ob lój (2004), that the resulting optimal distribution may be achieved using a feasible, typically randomized, gambling strategy. Such strategies may involve coin tossing at each step, see Appendix A. They can however be taken in much simpler forms. In He et al. (2014) a strategy is devised by which the gambler continues to play as long as he is winning and when he starts to lose too much, relative to his past high winnings, he uses his lucky coins to decide whether he should stop or continue. The coins are biased to reflect his preferences. A version of this strategy is implemented below, see Figures 10 and 11. The remaining of this paper is organized as follows: In Section 2, we briefly review the CPT and then formulate the model of casino gambling in the CPT framework. In Section 3, we offer numerical examples to show that path-dependent and randomized strategies strictly outperform, respectively, path-independent and non-randomized ones. In particular, we demonstrate that the optimal CPT value of the model considered in Barberis (2012) can be strictly improved by adding randomization. In Section 4, we derive a sys- 4

5 tematic approach to solving the problem and carry out a numerical study. Finally, Section 5 concludes. A technical construction proving existence of the optimal strategy among randomized strategies is given in the Appendix. 2 The Model 2.1 Cumulative Prospect Theory In classical economics theories, one evaluates uncertain payoffs according to the expected utility framework. However, expected utility theory (EUT) has been challenged by numerous experimental evidences as well as theoretical paradoxes and puzzles. In response to this, non-expected utility theories such as behavioral ones have been proposed. In this paper we adopt the cumulative prospect theory (CPT) proposed by Kahneman and Tversky (1992), which we now recall. There are important features fundamentally differentiating CPT from EUT. First, in CPT there exist a reference point, denoted by k, that distinguishes gains from losses. The excess values of wealth over k are called gains and the shortfall values are losses. The carriers of value function are gains and losses, rather than the total wealth level. The agent is risk averse on gains but risk-seeking on losses unless these payoffs occur very with small probabilities. Moreover, for the same magnitude of a gain and a loss, the gambler is more sensitive to the loss (termed as loss aversion ). Hence, the value function is of the following form u + (x k) for x k, u(x) = λu (k x) for x < k, where u + ( ), u ( ) are concave functions and λ 1. Overall u( ) is an S-shaped function; see Figure 1 for an illustration. Kahneman and Tversky (1992) proposed a possible form of u( ): x α + for x 0, u(x) = (1) λ( x) α for x < 0. 5

6 u(x) x Figure 1: S-shaped utility function (1) with α + = α = 0.5, and λ = 2.5. where α + = α = 0.88, λ = Finally, there are cumulative probability weighting (distortion) functions w + ( ), w ( ) applied to gains and losses respectively. An effect of the probability weighting is that both extremely good and bad events are overweighted. An inverse-s shaped weighting function is concave in the lower part for small probabilities and convex in the upper part for large probabilities. Note that such a weighting function overweights tails of a distribution, leading to the exaggeration of small probabilities. Kahneman and Tversky (1992) suggested an analytical form of w( ): w(p) = p δ, (2) (p δ + (1 p) δ ) 1/δ where δ = 0.61 and 0.69 respectively for the weighting functions applied to gains and losses; see Figure 2 for an illustration. Note that δ = 1 means that no distortion is applied. We assume the gambler s reference point is his initial wealth, so his gain and loss is the same as the actual gain and loss of playing in the casino. Denote a discrete gain/loss random variable by..., ( n, p n ),..., ( 1, p 1 ), (0, p 0 ),..., (n, p n ),... 6

7 delta = 0.65 delta = w(p) p Figure 2: Inverse-S shaped probability weighting function (2). The dotted line corresponds to δ = 0.65, the dashed line to δ = 0.4, and the solid line to δ = 1. where p n is the probability of gaining $n and p n is the probability of losing $n, n N, + n= p n = 1. The CPT value of this random gain/loss is n=1 ( u + (n) w + ( p j ) w + ( j=n j=n+1 ) p j ) λ n=1 ( u (n) w ( p j ) w ( j=n j=n+1 ) p j ). (3) A CPT gambler is risk-averse on gains and risk-seeking on losses with moderate and large probabilities, so u + (x) and u (x) are both concave. The gain part is also differentiated from the loss part by applying possibly different weighting functions. Moreover, he overweights the tails of a distribution, so w + (p) and w (p) are concave for small values of p. 2.2 The Casino Gambling Problem We now formulate a simple model of casino gambling following Barberis (2012). At each step, until he decides to leave the casino, the gambler takes a bet that wins or loses $1 with equal probability. The gain/loss process can be represented as a binomial tree. Each node is marked by a pair (n, x), where n stands for the time and x for the amount of (cumulative) gain/loss. For example, the node (2, 2) signifies a total loss of $2 at time 2. See Figure 3 7

8 (2,2) (1,1) (0,0) (2,0) (1, 1) (2, 2) Figure 3: The gain/loss process can be represented as a binomial tree. Each node is marked by a pair (n, x), where n stands for the time and x the amount of (cumulative) gain/loss. For example, the node (2, 2) signifies a total loss of $2 at time 2. for an illustration of this process. The gambler has the CPT preferences as described in 2.1 and wants to determine the best time (an exit or stopping strategy) to quit the casino. At time 0, the gambler determines whether or not he should enter the casino based on the optimal CPT value of gambling at that time. If the value is larger than that of not gambling, i.e. is larger than zero, then the gambler enters the casino to gamble; otherwise he leaves. Moreover, when the optimal CPT value is positive, he will need to determine a future optimal stopping strategy. Note, however, that an optimal strategy with a positive CPT value is only optimal at time 0 and may no longer be optimal after the game starts due to the presence of the 8

9 probability weighting. Such a strategy, possibly enforceable by some commitment device, is called a pre-commitment strategy. The gain/loss process of the gambler, denoted by (S t : t N), is a simple symmetric random walk. We denote by τ a strategy representing the time when the gambler stops and exits the casino. If τ is restricted to be bounded by a prescribed date T, we say the problem is on a finite time horizon; otherwise it is on an infinite time horizon. Let p j (τ) = P(S τ = j), j Z be the probability distribution of the terminal gain or loss associated with τ. In view of (3), denote by v(τ) the CPT value of a stopping strategy τ: v(τ) = n=1 λ ( u + (n) w + ( p j (τ)) w + ( j=n j=n+1 ( u (n) w ( p j (τ)) w ( n=1 j=n ) p j (τ)) j=n+1 ) p j (τ)). (4) For simplicity, in what follows whenever there is no ambiguity we use p j and p j to stand for p j (τ) and p j (τ) respectively. A behavioral gambler attempts to maximize the CPT value of gambling, v(τ), over all the allowed exit strategies τ A at time 0. The choice of a specific set of admissible strategies is critical for forming the optimal strategy, and we will specify a few important types below. 2.3 Types of Stopping Strategies A simple class of strategies, considered in Barberis (2012), is given by rules τ which assign a binary decision to each node in the winnings process, i.e. τ can be seen as a map from the nodes of the binomial tree to set {1, 0}. We call such strategies Markovian or pathindependent and denote their set A M. Extending the above, we can consider path-dependent strategies which base the decision not only on the current state but also on the whole betting history. Naturally, τ can not peak into the future. Mathematically speaking this corresponds to τ being a stopping time relative to F t = σ(s u : u t) the natural filtration of the S. However the set of such stopping times is too large since it includes strategies which use unlimited credit line strategies like wait till you win $1M or the so-called doubling strategies. There are many way to exclude such strategies. We could impose a finite credit line and say that the 9

10 gambler is thrown out when he losses $M. More generally, we require (S t τ : t N) to be uniformly integrable, i.e. that 3 lim sup E[ S τ t 1 M Sτ t M] = 0. t The set of such stopping times is denoted by A ST. Finally, we consider strategies that can make use of independent coin tosses. To this end, let (ξ t,x : t N, x { t, t + 2,..., t 2, t}) be a sequence of independent random variables that are independent of (S t : t N), taking values in {0, 1}, i.e. P(ξ t,x = 0)+P(ξ t,x = 1) = 1. We think of ξ t,x as outcome of a biased coin toss when the gain/loss is x at time t. We do not specify here the distribution of each ξ t,x these are endogenous and will be determined by the gambler as part of the optimal strategy. We let Ft R := σ(ξ u,x : u [0, t] N, x { u, u + 2,..., u 2, u}) be the information set generated by those random variables up to time t. The joint information flow of the gambling proceeds and coin tossing is given by G t = σ(f t Ft R ), t N. A randomized strategy τ is a stopping time relative to (G t ) such that (S t τ : t N) is uniformly integrable. The set of such strategies is denoted by A R. We stress that the random variables (ξ t,x : t N, x { t, t + 2,..., t 2, t}) are a part of the optimal solution that needs to be derived. 4 3 Comparison of Strategies In this section we compare the three types of strategies defined above, and demonstrate through three examples that path-dependent and randomized strategies respectively can strictly increase the optimal value achieved using the Markovian ones considered in Barberis (2012). Here we assume the utility function and the probability distortion function follow the analytical form (1) and (2) of Kahneman and Tversky (1992) and α + = α = α, 3 If E[S 2 τ ] < then this is equivalent to requiring E[τ] <. 4 When considering a finite horizon problem, t T, for all the strategies above we have to further restrict them to take values in [0, T ]. Note that the uniform integrability requirement is then trivially satisfied and elements in A ST, respectively A R, correspond to all stopping times relative to (F t ), respectively (G t ), which are bounded by T. 10

11 δ + = δ = δ. Example 1: Path-dependence versus Path-independence We show in this example that the optimal CPT value over path-independent strategies is strictly improved by path-dependent strategies. Let (α, δ, λ) = (0.91, 0.81, 1) and consider a 4-period horizon. First, consider the case in which only path-independent strategies are allowed. The optimal strategy can be found through exhaustive search. The optimal CPT value is v = Figure 4 shows the optimal path-independent strategy, where black nodes stand for stop and white nodes stand for continue. Observe that this strategy is to stop at any loss state and to continue at any gain state except for node (3, 1). Now, we allow path-dependent strategies. Again, the optimal strategy can be found through exhaustive search. The optimal CPT value is strictly improved: v = Figure 5 shows this optimal path-dependent strategy. Node (3, 1) is half-white-half-black, meaning if the previous node the path has gone through is node (2, 2), then stop; if the previous node is (2, 0), then continue. Notice that even though (3, 1) is the only different node compared with the optimal path-independent strategy, the optimal CPT value has already been strictly increased. Example 2: Randomization versus Non-randomization Next we show that randomization can strictly increase the optimal CPT value. First, let (α, δ, λ) = (0.95, 0.2, 1) and consider a 5-period horizon. If only path-independent strategies are allowed, the optimal CPT value is v = Figure 6 shows this pathindependent strategy. Now we slightly modify this strategy by introducing randomization into node (4, 4) which is grey in Figure 7. This means if node (4, 4) is reached, then the gambler tosses a biased coin with (head : 1/3, tail : 2/3), i.e. the probability of getting a head is 1/3, while the probability of getting a tail is 2/3. If the result is a head, then the gambler continues; otherwise he stops. The CPT value of such a strategy is strictly larger than the previous one: v = Example 3: Randomization versus Non-randomization in Barberis (2012) Finally, we consider the same parameter values as in Barberis (2012) with a 5-period horizon: (α, δ, λ) = (0.95, 0.5, 1.5). In this particular case, it happens that path-dependent strategies do not improve path-independent ones; the optimal strategy is shown in Figure 8 and the optimal CPT value is v = Now we introduce randomization into node (4, 2) which is grey in Figure 9. Once this node (4, 2) is reached, the gambler tosses a biased coin with (head : 31/32, tail : 1/32) and accordingly continues or stops. The CPT 11

12 (3,3) (2,2) (1,1) (3,1) (0,0) (2,0) Figure 4: Optimal path-independent strategy in Example 1. The optimal CPT value is v = Black nodes stand for stop and white nodes stand for continue. (3,3) (2,2) (1,1) (3,1) (0,0) (2,0) Figure 5: Optimal path-dependent strategy in Example 1. The optimal CPT value is v = Black nodes stand for stop and white nodes stand for continue. Half-white-halfblack node means if the previous node the path has gone through is node (2, 2), then stop; if the previous node is (2, 0), then continue. 12

13 (4,4) (3,3) (2,2) (1,1) (0,0) Figure 6: Optimal path-independent strategy in Example 2. The optimal CPT value is v = Black nodes stand for stop and white nodes stand for continue. (4,4) (3,3) (2,2) (1,1) (0,0) Figure 7: randomized strategy in Example 2. The CPT value is v = Black nodes stand for stop and white nodes stand for continue. Grey node means if node (4, 4) is reached, then the gambler tosses a biased coin with (head : 1/3, tail : 2/3), i.e. the probability of getting a head is 1/3, while the probability of getting a tail is 2/3. If the result is a head, then the gambler continues; otherwise he stops. 13

14 (4,4) (3,3) (2,2) (4,2) (1,1) (0,0) Figure 8: Optimal path-dependent strategy in Example 3. The optimal CPT value is v = Black nodes stand for stop and white nodes stand for continue. (3,3) (4,4) (2,2) (4,2) (1,1) (0,0) Figure 9: randomized strategy in Example 3. The CPT value is v = Black nodes stand for stop and white nodes stand for continue. Grey node means if node (4, 2) is reached, then the gambler tosses a biased coin with (head : 31/32, tail : 1/32). If the result is a head, then the gambler continues; otherwise he stops. 14

15 value of such a strategy is increased to v = In general, the following chain on inequalities is obvious: sup τ A M v(τ) sup τ A ST v(τ) sup τ A R v(τ), (5) because each time we allow for more and more sophisticated strategies. Clearly, a pathindependent strategy is easier than a path-dependent one since less information is needed to make a decision at each node. Both path-independent and path-dependent strategies use information from the gambling process itself, while a randomized strategy makes use of additional independent information. 4 Solving the Optimal Gambling Problem Recall that we converted the gambling problem into an optimal stopping problem sup v(τ) = τ A R ( n=1 u +(n) w + ( j=n p j(τ)) w + ( ) j=n+1 p j(τ)) λ n=1 u (n) ( w ( j=n p j(τ)) w ( j=n+1 p j(τ)) ), (6) where the randomization (ξ t,x ) is a part of the optimal solution to (6). The classical approaches in solving optimal stopping problems include martingale theory and dynamic programming, which respectively depend on the linearity of mathematical expectation and time consistency. These both fail due to the presence of probability weighting functions. Observe that p j (τ), the probability distribution of S τ, appears in the objective function, which motivates us to take the probability distribution of S τ as the decision variable. This idea of changing decision variable from τ to the probability distribution function or quantile function of S τ was first applied by Xu and Zhou (2012) to solve a continuous-time optimal stopping problem with probability weighting. Here, due to our discrete-time setting, we use only distribution functions. The resulting problem after this change of variable is an infinite-dimensional program, which has been studied extensively in literature. 5 The difference is not due to the numerical error. 15

16 To summarize, the process for solving (6) can be divided into three steps. First, change the decision variable from stopping time τ to the probability distribution of S τ. Second, solve an infinite-dimensional program to obtain the optimal distribution of S τ. Third, recover the optimal stopping strategy from the optimal probability distribution that has been obtained in the previous step via Skorokhod embedding techniques. Consider the problem on an infinite time horizon. To carry out the first step, we consider the set of probability measures on integers with zero expected value: { M 0 (Z) = µ : probability measure on Z s.t. n µ({n}) < and } n µ({n}) = 0. n Z n Z For any µ M 0 (Z), let x n = j=n p i, and y n (x; y) (x 1, x 2,..x n,..; y 1, y 2,..y n,..) and = j=n p j, where p j = µ({j}). Let V (x; y) = = n=1 n=1 ( ) u + (n) w + (x n ) w + (x n+1 ) λ ( ) u + (n) u + (n 1) w + (x n ) λ n=1 n=1 ( ) u (n) w (y n ) w (y n+1 ) ( ) u (n) u (n 1) w (y n ), where the second sum is assumed to be finite and the second equality is due to Fubini s theorem. When (x, y) are defined from a measure µ M 0 as above, we sometimes write (x µ, y µ ). Since x and y are decumulative probabilities in gains and accumulative probabilities in losses respectively, we have the following natural constraints: 1 x 1 x 2... x n... 0, 1 y 1 y 2... y n... 0, x 1 + y 1 = 1 µ({0}) 1. 16

17 In summary, we have the following candidate infinite-dimensional programming problem sup (x;y) V (x; y) subject to 1 x 1 x 2... x n... 0, 1 y 1 y 2... y n... 0, x 1 + y 1 1, n=1 x n = n=1 y n <. (7) The following result asserts that solving the original gambling problem indeed boils down to solving the above problem. Theorem 1 The optimal gambling problem sup τ AR v(τ) is equivalent to problem (7). More precisely, if τ A R and µ is the distribution of S τ then µ M 0 (Z) and v(τ) = V (x µ, y µ ). Conversely, for any µ M 0 (Z) there exists τ A R such that S τ is distributed according to µ and v(τ) = V (x µ, y µ ). The only part of the statement which requires a proof is that any µ M 0 (Z) may be represented as the distribution of S τ for some τ A R. We do this by providing an explicit construction of τ; see Appendix. 6 Theorem 1 also suggests the mathematical reason why the second inequality in (5) is typically strict. Our reasoning shows that we can transform the original gambling problem into a maximization problem over probability distributions. Theorem 1 links sup τ AR v(τ) with (7). In contrast, using τ A ST essentially corresponds to taking µ from a fractal subset of M 0 (Z), see Theorem 7 in Cox and Ob lój (2008), which corresponds to only a fraction of all measures in M 0 (Z). To complete our program it remains to solve (7). Observe that V ( ; ) is generally non-convex in (x; y) and thus is hard to solve. The whole analysis leading to a thorough solution is complex and lengthy; so we fully develop it in a companion paper, He et al. (2014). Briefly, the main idea of He et al. (2014) is to decompose (7) into two optimization subproblems, one being to maximize the gain part and the other to minimize the loss part, and introduce a new parameter s into the two subproblems to link the two subproblems 6 The above result also has a finite time horizon analogue but its statement is more involved and will be provided in a companion paper. 17

18 and to satisfy the last constraint in (7) by fixing the value of x 1, y 1 as parameters first. The authors also study the well-posedness of the gambling problem, i.e., when the optimal CPT value is finite, and fully solve the infinite-dimensional program when the utility function is piece-wise power and the two probability weighting functions are power. Finally, He et al. (2014) also present simple constructions of stopping times τ A R which embed a given µ M 0 (Z) and which appear more intuitive and appealing than the fully-randomized solution we present in Appendix A. One variant can be seen as a randomized discrete-time version of Azéma and Yor (1979) solution to the Skorokhod embedding problem and it has a natural interpretation: the gambler follows a randomized stop-loss strategy, i.e. he sets a stop-loss level which is dynamically updated based on the past maximum, and exits as soon as that level is hit and otherwise continues only that at times he tosses a (biased) coin to determine whether he should stop instead. We close this section with a numerical example which illustrates the procedure set out above. Following a numerical example in He et al. (2014) where α + = 0.6, α = 0.8, δ + = δ = 0.7, and λ = 1.05, the optimal desired of S τ is obtained as, ) ((n 0.6 (n 1) 0.6 ) 1/0.3 ((n + 1) 0.6 n 0.6 ) 1/0.3 for n 2, p for n = 1, n = for n = 1, 0 for n = 0 or n 2. We now derive an optimal strategy which achieves this distribution. In fact, we build two such strategies. First, we could consider the strategy which is described in Appendix A: each time his wealth reaches state n the gambler tosses a biased coin and stops betting with probability r n. The numerical algorithm presented in Appendix A allows one to compute 18

19 r n explicitly: 0 for n = 0 or n 2, 1 for n = 1, for n = 1, r n = for n = 2, for n = 3,... This strategy is simple but it does not use the winnings history and relies solely on randomization, namely, the gambler depends only on the outcome of the coin toss at every node. Our second strategy only uses randomization when it is required. It is an AY-like stopping time, derived analytically and described in more detail in He et al. (2014). We use two random paths to illustrate how the optimal stopping time works. Figure 10 shows one possible path. Initially, the gambler starts at node (0, 0). According to the solution he first sets an initial stop-loss level S n = 1, which means if he loses $1 at time 1 he will quit. If he wins $1 at time 1, he tosses a coin to decide whether he will continue or stop. Hence, node (1, 1) is grey. The probability of stopping (tails) is and the probability of continuing (heads) is If it shows a head the gambler continues. The stop-loss level is kept unchanged since this path does not achieve a new maximum larger than 1. When a black node (7, 1) is hit, the gambler stops with cumulative loss equal to $1. Figure 11 shows another possible path. Initially, the gambler starts at node (0, 0) and his initial stop-loss is S n = 1. If he wins $1 at time 1, he tosses the same coin as described above to decide whether he will continue or stop. Here the coin comes heads up and he continues with the path being the same as the first one up to time 3. However now the gambler wins at time 4 and his cumulative gain is then $2 with the path arriving at node (4, 2). A new maximum 2 is achieved so that the stop-loss level is updated to S n = 1. If the gambler then wins at time 5, loses at time 6 and time 7, the lower bound is unchanged so he finally stops at time 7 with cumulative gain hitting $1. 19

20 (1,1) (0,0) (7,-1) Figure 10: A possible paths illustrating the optimal strategy achieving (p n). The gambler starts at node (0, 0) and he first sets an initial stop-loss level S n = 1, which means if he loses $1 at time 1 he will quit. If he wins $1 at time 1, he tosses a coin to decide whether he will continue or stop, represented by grey node (1, 1). The probability of a tail of the coin is and the probability of a head is If it shows a head the gambler continues. The stop-loss level is kept unchanged and when a black node (7, 1) is hit, the gambler stops with cumulative loss equal to $1. (4,2) (1,1) (3,1) (7,1) (0,0) Figure 11: A second possible paths illustrating the optimal strategy achieving (p n). The path is the same as the first one before time 3. If the gambler wins at time 4 then his cumulative gain is $2 and the path arrives at node (4, 2). A new maximum 2 is achieved so that the stop-loss level is updated to S n = 1. If the gambler then wins at time 5, loses at time 6 and time 7, the lower bound is unchanged so he finally stops at time 7 with cumulative gain equal to $1. 20

21 5 Conclusion This paper considers the dynamic casino gambling model under CPT preference that was initially proposed by Barberis (2012). We have shown, via simple examples, that one should consider path-dependent as well as randomized stopping strategies to achieve optimality. We have then developed a general approach to solve the model systematically, which involves a change of decision variable, an infinite dimensional program, and the Skorokhod embedding technique. The main thrust of the paper is to argue about the necessity of considering pathdependence and randomization; the whole technical development for solving the model is highly involved, technical and lengthy. So it will be carried out in several forthcoming companion papers. In particular, Theorem 1 is under the assumption of an infinite time horizon, an assumption that will greatly simplify the Skorokhod embedding procedure. The case of a finite time horizon will be studied in Hu et al. (2014). However, from a practical point of view, infinite horizon can reasonably model situations in which the gambler has no definite time horizon or the frequency of bets is sufficiently high. From a theoretical point of view, a model with a finite horizon may be approximated by a sequence of infinite horizon problems with Poisson shocks. 7 Barberis (2012) took a special note of the time-inconsistency issue inherent in his model, and accordingly discussed about the behaviors of three types of gamblers: naive ones who are unaware of time-inconsistency so they will keep changing their plans ( type 1 ), sophisticated ones who are aware of time-inconsistency but unable to commit and hence will never enter a casino ( type 2 ), and sophisticated ones who are aware of time-inconsistency and able to commit ( type 3 ). Clearly, the solution procedure developed here applies readily to type 3 gamblers as pre-commitment strategies. It can also be used to explain the behaviors of type 1 gamblers since at each node they solve an optimal stopping problem which is the same as the problem solved here. Note. Recently, and independently from us, Henderson et al. (2014) observed that randomized strategies may be necessary for optimal gambling strategies. This has emerged through a private conversation between two of the authors at the SIAM Financial Math- 7 We thank Nick Barberis for suggesting this idea in a private communication, although in Hu et al. (2014) we solve the problem by directly employing Skorokhod embedding in a finite time horizon. 21

22 ematics meeting in November 2014 in Chicago. The other paper has been subsequently posted on SSRN. A Proof of Theorem 1 To finish the proof of Theorem 1 we need to show that for any µ M 0 (Z) we can find a τ A R such that S τ has distribution µ: S τ µ. To achieve this we consider a family of strategies which work in a straightforward way. At each node, a (biased) coin is tossed. If it shows heads, then continue; if it shows tails, then stop. In particular, the gambling history is not used for decision making. Consider a family of independent random variables (ξi k : i Z, k N) such that P(ξi k = 0) = r i = 1 P(ξi k = 1), where i stands for the level of gain/loss state and k is a counter to determine how many times state i has been previously visited. Let τ(r) := inf{t 0 : ξ ρ(st) S t = 0, t N}, where ρ(i) is the number of times i has been visited up to, and including, time t. other words, τ(r) is the first time when a tail appears. Note that here we are using the independent randomization somewhat more efficiently than in Section 2.3 and τ(r) is a stopping time with respect to the filtration G t := σ(s u, ξ ρ(su) S u : u t) and S remains a simple symmetric random walk w.r.t. (G t ). Finally, recall that M 0 (Z) is the set of probability distributions on integers with zero mean. Theorem 1. We now give a more specific formulation of Theorem 2 Given any µ M 0 (Z), there exists r = (r i ) [0, 1] Z such that the stopping time τ(r) embeds µ into the random walk, i.e. S τ(r) µ and (S τ(r) n : n 0) is uniformly integrable. In the rest of this Appendix we prove Theorem 2. We first consider µ M 0 (Z) with bounded support: µ([ B, A]) = 1 and µ({a}) > 0, µ({ B}) > 0. Let H [ B,A] = inf{t N : S t A or S t B}. Note that τ H [ B,A] is a necessary condition of the uniform integrability of (S τ t : t N) (Cox and Ob lój, 2008, Proposition 1). Consider the set of r In 22

23 which embed less mass than in µ on ( B, A): R µ = {r [0, 1] Z : for i Z, r i = 1 if i / ( B, A) and P(S τ(r) = i) µ({i}) if i ( B, A)}. Note that we can choose r i = 0 for i Z ( B, A) and r i = 1 for i Z ( B, A) c, so R µ is non-empty. Further, by definition, τ(r) H [ B,A] for any r R µ. Proposition 3 If r, r R µ, then so does their maximum, i.e. r R µ, where r i = r i r i, i Z. Proof Fix r, r R µ and r = ( r i : i Z) with r i = r i r i. To obtain a family of random variables generating τ( r), consider (ξi k : i Z, k N) we used to construct τ(r). Define ξ i k := ξi k ε k i, where (ε k i : i Z, k N) are all mutually independent, and independent of all ξi k and the random walk (S t : t N) such that P(ε k i = 0) = (r i r i) + 1 r i = 1 P(ε k i = 1). We have P( ξ k i = 0) = P(ξ k i = 0) + P(ξ k i = 1)P(ε k i = 0) = r i + (1 r i ) (r i r i ) + 1 r i = r i r i, as required. By construction, for any i Z, k N, we have {ξ k i = 0} { ξ k i = 0} so τ( r) τ(r). Now pick i 0 Z ( B, A) with r i0 r i 0 so that r i0 = r i0 and ξ i 0 = ξ i 0. It follows that {S τ( r) = i 0 } {S τ(r) = i 0 } and hence P(S τ( r) = i 0 ) P(S τ(r) = i 0 ) µ({i 0 }), since r R µ. A symmetric argument shows that also P(S τ( r) = i) µ({i}) also for i with r i = r i and we conclude that r R µ. By Proposition 3 we conclude that R µ has a maximal element r max = (ri max fact, this element provides a solution to the Skorokhod embedding problem. Proposition 4 S τ(r max ) µ and (S τ(r max ) t : t N) is uniformly integrable. Q.E.D : i Z). In Proof Uniform integrability follows instantly from τ(r max ) H [ B,A]. The embedding property is shown by contradiction. Suppose there exists a point x Z ( B, A) such that P(S τ(r max ) = x) < µ({x}). Consider r ɛ with ri ɛ = ri max for i x and rx ɛ = rx max + ɛ. It follows that P(S τ(r ɛ ) = i) P(S τ(r max ) = i) µ({i}) for i x. Further, P(S τ(r ɛ ) = x) is a continuous increasing function for ɛ 0 and hence, for epsilon small enough, P(S τ(r ɛ ) = x) µ({x}). But then r ɛ R µ contradicting the fact that r max was the maximal element of R µ. 23 Q.E.D

24 Proposition 4 established Theorem 2 for µ with bounded support. We now extend it to general µ M 0 through a limiting and coupling procedure. Let us fix µ M 0 without bounded support and, without loss of generality, assume the support is unbounded on the right, i.e. µ((, n]) < 1 for all n 0. For n N, let n be defined by { } n 1 k 1 n = min k N : iµ({i}) < iµ({ i}) + k(1 µ(( k, n))) i=1 i=1 where it follows from µ M 0 that n is well defined, non-decreasing in n and converges to inf{i : µ((, i]) > 0} as n. Further, there exists a unique centered probability measure µ n on Z [ n, n] satisfying µ n ({i}) = µ({i}) for all i Z ( n, n). By Propositions 3 and 4 we know that R µn admits a maximal element r n such that S τ(r n ) µ n. Moreover, ri n = 1 for i Z ( n, n) c. In particular τ(r n ) H [ n,n] and hence τ(r n ) stops upon hitting n or n while τ(r n+1 ) could hit n and then come back to arbitrary i Z (n, n). It follows that ri n r n+1 i for any i Z (n, n). In particular, the for a fixed i the sequence (ri n ) converges and we let r i := lim n ri n, i Z. We now build a coupling of stopping times τ(r n ). Let ξ k i (n) = 1 ε k i (1) ε k i (2)... ε k i (n), where ε k i (j) are all mutually independent, and independent of (S k ), and P(ε k i (1) = 1) = r 1 i = 1 P(ε k i (1) = 0), P(ε k i (2) = 1) = r2 i r 1 i... = 1 P(ε k i (2) = 0), P(ε k i (n) = 1) = rn i r n 1 i = 1 P(ε k i (n) = 0). Then P(ξi k (n) = 0) = P(ε k i (1) = ε k i (2) =... = ε k i (n) = 1) = ri n = 1 P(ξi k (n) = 1) and we can use (ξi k (n)) to define the stopping time τ(r n ). However with this representation, ξ k i (n) ξ k i (n + 1) so if τ n does not stop at the k-th visit to state i, then none of τ m does, for m n. It follows that τ n τ n+1... τ m... and hence, let τ := lim n τ n, which is well defined. We now argue that τ < a.s. To this end, let F t := S t t 1 j=0 1 S j =0, t N. One easily checks that (F t ) is a martingale. By the optional sampling theorem, using τ(r n ) H [ n,n], 24

25 we have E[F τn ] = 0. This yields the first equality in the following computation: 1 Sj =0] = E[ S τn ] = i µ n ({i}) i µ({i}) <. (8) j=0 i Z i Z τ n 1 E[ By the monotone convergence theorem, the LHS of the above converges to E[ τ 1 j=0 1 S j =0] and it follows that τ < a.s. since otherwise the value would be infinite by the recurrence of symmetric random walk. Therefore, we have τ n τ and S τn S τ a.s. Note that P(S τn = i) = µ n ({i}) = µ({i}) for n large enough, except possibly for the left end of the support of µ, if finite. Hence, S τn S τ in distribution and it follows that S τ µ. Furthermore, E[ S τn ] = i Z i µ n({i}) i Z i µ n+1({i}) = E[ S τn+1 ]. We have lim n E[ S τn ] = E[ S τ ] by the uniform bound in (8). It follows by Scheffe s lemma (see e.g. Williams (1991)) that S τn S τ in L 1 and hence E[S τ F τn ] = S τn a.s. By martingale convergence theorem and tower property of conditional expectation, E[S τ F τ t ] = lim n E[S τ F τn t] = lim n E[E[S τ F τn ] F τn t] = lim n E[S τn F τn t] = lim n S τn t = S τ t. We conclude that (S τ t : t N) is uniformly integrable as required. This completes the proof of Theorem 2. We end with a description of an algorithm which can be used to compute (r i ) numerically. Note that the above procedure also gives a straightforward way to numerically estimate r = (r i : i Z) from r n = (ri n : i Z) corresponding to the bounded support case. So it suffices to describe how to obtain the latter. Consider a discrete probability measure µ with bounded support such that µ([ m, n]) = 1. Denote by r = (r i [0, 1] : i Z) the set of probabilities generating this µ via S τ(r) µ. Note that r i = 1 for i n and i m. Let p i = µ({i}), i Z [ m, n] and note that if p i = 0 then clearly r i = 0. Suppose p n 1 > 0 and we derive r n 1 first. Note that a path stops at the state n if and only if it does not stop at the state n 1 in the previous step and goes up. The weight of paths which get to n 1 but do not stop is p n 1 /r n 1 which lead to 1 p n 1 (1 r n 1 ) = p n. 2 r n 1 25

26 For the same reason, we can derive r n 2. If p n 1 > 0, then r n 2 is such that If p n 1 = r n 1 = 0, then we replace p n 1 r n 1 1 p n 2 (1 r n 2 ) = p n 1. 2 r n 2 r n 1 by 2p n and r n 2 is such that 1 p n 2 (1 r n 2 ) = 2p n. 2 r n 2 Suppose now we have derived the value of r i, i = k, k + 1,...n 1, where 1 k n 2. If p k > 0 and p k+1 > 0, then r k 1 is such that 1 p k+1 (1 r k+1 ) + 1 p k 1 (1 r k 1 ) = p k. 2 r k+1 2 r k 1 r k If p k = 0, p k+1 > 0 and p k+2 > 0, then we replace p k r k by 2( p k+1 r k+1 1 p k+2 2 r k+2 (1 r k+2 )) and r k 1 is such that 1 p k+2 (1 r k+2 ) p k+1 (1 r k+1 ) + 2 r k+2 2( 1 p ) k 1 (1 r k 1 ) = p k+1, 2 r k+1 2 r k 1 r k+1 so on so forth. In other words, r k is derived from p k, p k+1,...p n. Likewise, the probabilities at a loss node, r i for i Z [ m + 1, 1], can be derived in the same way as above. References Azéma, J. and Yor, M. (1979). Une solution simple au problème de Skorokhod, in C. Dellacherie, P.-A. Meyer and M. Weil (eds), Séminaire de Probabilités, XIII (Univ. Strasbourg, Strasbourg, 1977/78), Vol. 721 of Lecture Notes in Math., Springer, Berlin, pp Barberis, N. C. (2012). A model of casino gambling, Management Science 58(1): Conlisk, J. (1993). The utility of gambling, Journal of Risk and Uncertainty 6(3): Cox, A. and Ob lój, J. (2008). Classes of measures which can be embedded in the Simple Symmetric Random Walk, Electron. J. Probab. 13:

27 He, X. D., Hu, S., Ob lój, J. and Zhou, X. Y. (2014). Stopping strategies of behavioral gamblers in infinite horizon, working paper. Henderson, V., Hobson, D. and Tse, A. (2014). Randomized strategies and prospect theory in a dynamic context. SSRN: Hu, S., Ob lój, J. and Zhou, X. Y. (2014). Stopping strategies of behavioral gamblers in finite horizon, working paper. Kahneman, D. and Tversky, A. (1992). Advances in prospect theory: Cumulative representation of uncertainty, Journal of Risk and Uncertainty 5(4): Ob lój, J. (2004). The skorokhod embedding problem and its offspring, Probability Surveys 1: Williams, D. (1991). Probability with Martingales, Cambridge University Press. Xu, Z. Q. and Zhou, X. Y. (2012). Optimal stopping under probability distortion, The Annuals of Applied Probability 23(1):