1 Gambler s Ruin Problem

Transcription

1 Coyright c 2009 by Karl Sigman 1 Gambler s Ruin Problem Let N 2 be an integer and let 1 i N 1. Consider a gambler who starts with an initial fortune of $i and then on each successive gamble either wins $1 or loses $1 indeendent of the ast with robabilities and q = 1 resectively. Let X n denote the total fortune after the n th gamble. The gambler s objective is to reach a total fortune of $N, without first getting ruined (running out of money). If the gambler succeeds, then the gambler is said to win the game. In any case, the gambler stos laying after winning or getting ruined, whichever haens first. {X n } yields a Markov chain (MC) on the state sace S = {0, 1,..., N}. The transition robabilities are given by P i,i+1 =, P i,i 1 = q, 0 < i < N, and both 0 and N are absorbing states, P 00 = P NN = 1. 1 For examle, when N = 4 the transition matrix is given by q P = 0 q q While the game roceeds, this MC forms a simle random walk X n = i n, n 1, X 0 = i, where { n } forms an i.i.d. sequence of r.v.s. distributed as P ( = 1) =, P ( = 1) = q = 1, and reresents the earnings on the successive gambles. Since the game stos when either X n = 0 or X n = N, let τ i = min{n 0 : X n {0, N} X 0 = i}, denote the time at which the game stos when X 0 = i. If X τi = N, then the gambler wins, if X τi = 0, then the gambler is ruined. Let P i (N) = P (X τi = N) denote the robability that the gambler wins when X 0 = i. P i (N) denotes the robability that the gambler, starting initially with $i, reaches a total fortune of N before ruin; 1 P i (N) is thus the corresonding robably of ruin Clearly P 0 (N) = 0 and P N (N) = 1 by definition, and we next roceed to comute P i (N), 1 i N 1. Proosition 1.1 (Gambler s Ruin Problem) 1 ( q )i P i (N) = 1 ( q, if q; )N (1) i N, if = q = There are three communication classes: C 1 = {0}, C 2 = {1,..., N 1}, C 3 = {N}. C 1 and C 3 are recurrent whereas C 2 is transient. 1

2 Proof : For our derivation, we let P i = P i (N), that is, we suress the deendence on N for ease of notation. The key idea is to condition on the outcome of the first gamble, 1 = 1 or 1 = 1, yielding P i = P i+1 + qp i 1. (2) The derivation of this recursion is as follows: If 1 = 1, then the gambler s total fortune increases to X 1 = i+1 and so by the Markov roerty the gambler will now win with robability P i+1. Similarly, if 1 = 1, then the gambler s fortune decreases to X 1 = i 1 and so by the Markov roerty the gambler will now win with robability P i 1. The robabilities corresonding to the two outcomes are and q yielding (2). Since + q = 1, (2) can be re-written as P i + qp i = P i+1 + qp i 1, yielding P i+1 P i = q (P i P i 1 ). In articular, P 2 P 1 = (q/)(p 1 P 0 ) = (q/)p 1 (since P 0 = 0), so that P 3 P 2 = (q/)(p 2 P 1 ) = (q/) 2 P 1, and more generally Thus P i+1 P i = ( q )i P 1, 0 < i < N. P i+1 P 1 = = i (P k+1 P k ) k=1 i k=1 ( q )k P 1, yielding i P i+1 = P 1 + P 1 ( q i )k = P 1 ( q )k = k=1 k=0 1 ( q P )i ( q ), if q; P 1 (i + 1), if = q = 0.5. (3) (Here we are using the geometric series equation i n=0 a i = 1 ai+1 1 a, for any number a and any integer i 1.) Choosing i = N 1 and using the fact that P N = 1 yields 1 ( q 1 = P N = P )N 1 1 ( q, if q; ) P 1 N, if = q = 0.5, from which we conclude that P 1 = 1 q 1 ( q )N, if q; 1 N, if = q = 0.5, 2

3 thus obtaining from (3) (after algebra) the solution 1 ( q )i P i = 1 ( q, if q; )N (4) i N, if = q = Becoming infinitely rich or getting ruined In the formula (1), it is of interest to see what haens as N ; denote this by P i ( ) = lim N P i (N). This limiting quantity denotes the robability that the gambler, if allowed to lay forever unless ruined, will in fact never get ruined and instead will obtain an infinitely large fortune. Proosition 1.2 Define P i ( ) = lim N P i (N). If > 0.5, then P i ( ) = 1 ( q )i > 0. (5) If 0.50, then P i ( ) = 0. (6) Thus, unless the gambles are strictly better than fair ( > 0.5), ruin is certain. Proof : If > 0.5, then q < 1; hence in the denominator of (1), ( q )N 0 yielding the result. If < 0.50, then q > 1; hence in the the denominator of (1), ( q )N yielding the result. Finally, if = 0.5, then i (N) = i/n 0. Examles 1. John starts with $2, and = 0.6: What is the robability that John obtains a fortune of N = 4 without going broke? i = 2, N = 4 and q = 1 = 0.4, so q/ = 2/3, and we want P 2 (4) = 1 (2/3)2 1 (2/3) 4 = What is the robability that John will become infinitely rich? P 2 ( ) = 1 (2/3) 2 = 5/9 = If John instead started with i = $1, what is the robability that he would go broke? The robability he becomes infinitely rich is P 1 ( ) = 1 (q/) = 1/3, so the robability of ruin is 1 P 1 ( ) = 2/3. 3

4 1.2 Alications Risk insurance business Consider an insurance comany that earns $1 er day (from interest), but on each day, indeendent of the ast, might suffer a claim against it for the amount $2 with robability q = 1. Whenever such a claim is suffered, $2 is removed from the reserve of money. Thus on the n th day, the net income for that day is exactly n as in the gamblers ruin roblem: 1 with robability, 1 with robability q. If the insurance comany starts off initially with a reserve of $i 1, then what is the robability it will eventually get ruined (run out of money)? The answer is given by (5) and (??): If > 0.5 then the robability is given by ( q )i > 0, whereas if 0.5 ruin will always ocurr. This makes intuitive sense because if > 0.5, then the average net income er day is E( ) = q > 0, whereas if 0.5, then the average net income er day is E( ) = q 0. So the comany can not exect to stay in business unless earning (on average) more than is taken away by claims. 1.3 Random walk hitting robabilities Let a > 0 and b > 0 be integers, and let R n = n, n 1, R 0 = 0 denote a simle random walk initially at the origin. Let (a) = P ({R n } hits level a before hitting level b). By letting i = b, and N = a + b, we can equivalently imagine a gambler who starts with i = b and wishes to reach N = a + b before going broke. So we can comute (a) by casting the roblem into the framework of the gambler s ruin roblem: (a) = P i (N) where N = a + b, i = b. Thus Examles 1 ( q )b (a) = 1 ( q, if q; )a+b (7) b a+b, if = q = Ellen bought a share of stock for $10, and it is believed that the stock rice moves (day by day) as a simle random walk with = What is the robability that Ellen s stock reaches the high value of $15 before the low value of $5? We want the robability that the stock goes u by 5 before going down by 5. This is equivalent to starting the random walk at 0 with a = 5 and b = 5, and comuting (a). (a) = 1 ( q )b 1 (0.82)5 1 ( q = = 0.73 )a+b 1 (0.82) What is the robability that Ellen will become infinitely rich? 4

5 Here we equivalently want to know the robability that a gambler starting with i = 10 becomes infinitely rich before going broke. Just like Examle 2 on Page 3: 1 (q/) i = 1 (0.82) = Maximums and minimums of the simle random walk Formula (7) can immediately be used for comuting the robability that the simle random walk {R n }, starting initially at R 0 = 0, will ever hit level a, for any given ositive integer a 1: Kee a fixed while taking the limit as b in (7). The result deends on wether < 0.50 or A little thought reveals that we can state this roblem as comuting the tail P (M a), a 0, where M def = max{r n : n 0} is the all-time maximum of the random walk; a non-negative random variable, because {M a} = {R n = a, for some n 1}. Proosition 1.3 Let M def = max{r n : n 0} for the simle random walk starting initially at the origin (R 0 = 0). 1. When < 0.50, P (M a) = (/q) a, a 0; M has a geometric distribution with success robability 1 (/q): P (M = k) = (/q) k (1 (/q)), k 0. In this case, the random walk drifts down to, w1, but before doing so reaches the finite maximum M. 2. If 0.50, then P (M a) = 1, a 0: P (M = ) = 1; the random walk will, with robability 1, reach any ositive integer a no matter how large. Proof : Taking the limit in (7) as b yields the result by considering the two cases < 0.5 or 0.5: If < 0.5, then (q/) > 1 and so both (q/) b and (q/) a+b tend to as b. But before taking the limit, multily both numerator and denominator by (q/) b = (/q) b, yielding (a) = (/q)b 1 (/q) b (q/) a. Since (/q) b 0 as b, the result follows. If > 0.5, then (q/) < 1 and so both (q/) b and (q/) a+b tend to 0 as b yielding the limit in (7) as 1. If = 0.5, then (a) = b/(b + a) 1 as b. If < 0.5, then E( ) < 0, and if > 0.5, then E( ) > 0; so Proosition 1.3 is consistent with the fact that any random walk with E( ) < 0 (called the negative drift case) satisfies lim n R n =, w1, and any random walk with E( ) > 0 ( called the ositive drift case) satisfies lim n R n = +, w1. 2 But furthermore we learn that when < 0.5, although w1 the chain drifts off to, it first reaches a finite maximum M before doing so, and this rv M has a geometric distribution. 2 R From the strong law of large numbers, lim n n n + if E( ) > 0. = E( ), w1, so R n ne( ) if E( ) < 0 and 5

6 Finally Proosition 1.3 also offers us a roof that when = 0.5, the symmetric case, the random walk will w1 hit any ositive value, P (M a) = 1. By symmetry, we also obtain analogous results for the minimum, : Corollary 1.1 Let m def = min{r n : n 0} for the simle random walk starting initially at the origin (R 0 = 0). 1. If > 0.5, then P (m b) = (q/) b, b 0. In this case, the random walk drifts u to +, but before doing so dros down to a finite minimum m 0. Taking absolute values of m makes it non-negative and so we can exress this result as P ( m b) = (q/) b, b 0; m has a geometric distribution with success robability 1 (q/): P ( m = k) = (q/) k (1 (q/)), k If 0.50, then P (m b) = 1, b 0: P (m = ) = 1; the random walk will, with robability 1, reach any negative integer a no matter how small. Note that when < 0.5, P (M = 0) = 1 (/q) > 0. This is because it is ossible that the random walk will never enter the ositive axis before drifting off to ; with ositive robability R n 0, n 0. Similarly, if > 0.5, then P (m = 0) = 1 (q/) > 0; with ositive robability R n 0, n 0. Recurrence of the simle symmetric random walk Combining the results for both M and m in the revious section when = 0.5, we have Proosition 1.4 The simle symmetric ( = 0.50) random walk, starting at the origin, will w1 eventually hit any integer a, ositive or negative. In fact it will hit any given integer a infinitely often, always returning yet again after leaving; it is a recurrent Markov chain. Proof : The first statement follows directly from Proosition 1.3 and Corollary 1.1; P (M = ) = 1 = P (m = ) = 1. For the second statement we argue as follows: Using the first statement, we know that the simle symmetric random walk starting at 0 will hit 1 eventually. But when it does, we can use the same result to conclude that it will go back and hit 0 eventually after that, because that is stochastically equivalent to starting at R 0 = 0 and waiting for the chain to hit 1, which also will haen eventually. But then yet again it must hit 1 again and so on (back and forth infinitely often), all by the same logic. We conclude that the chain will, over and over again, return to state 0 w1; it will do so infinitely often; 0 is a recurrent state for the simle symmetric random walk. Thus (since the chain is irreducible) all states are recurrent. Let τ = min{n 1 : R n = 0 R 0 = 0}, the so-called return time to state 0. We just argued that τ is a roer random variable, that is, P (τ < ) = 1. This means that if the chain starts in state 0, then, if we wait long enough, we will (w1) see it return to state 0. What we will rove later is that E(τ) = ; meaning that on average our wait is infinite. This imlies that the simle symmetric random walk forms a null recurrent Markov chain. 6