6.042/18.062J Mathematics for Computer Science December 12, 2006 Tom Leighton and Ronitt Rubinfeld. Random Walks

Transcription

1 6.042/8.062J Mathematics for Comuter Science December 2, 2006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Random Walks Gambler s Ruin Today we re going to talk about one-dimensional random walks. In articular, we re going to cover a classic henomenon known as gambler s ruin. The gambler s ruin roblem is a articularly good way to end the term since its solution requires several of the techniques that we learned during the term. Those of you who like to gamble are sure to find it interesting. Suose we start with n dollars, and make a sequence of bets. For each bet, we win dollar with robability, and lose dollar with robability. We quit if either we go broke, in which case we lose, or when we reach T n + m dollars, that is, when we win m dollars. For examle, in Roulette, If n 00 dollars, and m dollars, then T 200 dollars. What are the odds we win 00 dollars before losing 00 dollars? Most folks would think that since.473.5, the odds are not so bad. In fact, as we will see, we win before we lose with robability at most This is an amazing result! The classic strategy is to continue betting until you are a little bit ahead, and then quit. We re going to see in just a moment that this strategy is not very good. Even if you quit when you re u 20%, say by making 20 dollars in the above exmale, we will show the chance of winning before losing is at most /8. This roblem is a classic examle of a roblem that involves a one-dimensional random walk. In such a random walk, there is some value - say the number of dollars we have - that can go u or down or stay the same at each ste with some robabilities. In this examle, we have a random walk in which the value can go u or down by at each ste. We can diagram it as follows. n+m n

2 2 Random Walks The robability of making an u move at any ste is, no matter what has haened in the ast. The robability of making a down move is. This random walk is a secial tye of random walk where moves are indeendent of the ast, and is called a martingale. If /2, the random walk is unbiased, whereas if /2, the random walk is biased. We also have boundaries at 0 and n + m. If the walk hits a boundary, then we sto laying, i.e., we quit when broke lose n) or when we get to n + m win m). So we care about the robability of winning or the robability of going broke. Note that these do not necessarily sum u to since there is some chance that we never hit a boundary and walk forever. However, we will show later that this chance is essentially 0. So let s figure out the robability that we gain m before losing n. To set things u formally, let W be the event we hit T before we hit 0, where T n + m. Let D t be a random variable that denotes the number of dollars we have at time ste t. Let P n Pr W D 0 n) be the robability we get T before we go broke, given that we start with n dollars. Our question then, is what is P n? We re going to use a recursive aroach. 0 if n 0 Claim. P n if n T P n+ + )P n, if 0 < n < T The intuition here is clear if n 0 or n T, since in this case we have either already lost or already won. Otherwise, since our moves are indeendent of the ast, with robability we obtain n + dollars, and with robability we obtain n dollars, which gives the recurrence. Proof. P 0 Pr W D 0 0) 0 since we ve already lost, and P T Pr W D 0 T ) since we ve already won. Now, assume 0 < n < T. Let E be the event that the first bet is a win, and Ē the event that the first bet is a loss. Then, P n Pr W D 0 n) Pr W E D 0 n) + Pr W Ē D 0 n ) Pr E D 0 n) Pr W E D 0 n) + Pr Ē D 0 n ) Pr W Ē D 0 n ) Pr W D n + ) + ) Pr W D n ) Pr W D 0 n + ) + ) Pr W D 0 n ) P n+ + )P n. The first equation is by definition. The second is by the theorem of total robability. The third is by the definition of conditional robability. The fourth and fifth follow from the fact that E is indeendent of the current dollar amount, and the time ste we are at. Now we have a recurrence to solve. Rewriting it, we obtain P n+ P n + )P n 0, where P 0 0 and P T. This is a linear homogeneous recurrence. We start by solving

3 Random Walks 3 the characteristic equation r 2 r + ) 0. Using the quadratic formula, r ± 4 ) 2 ± ± 2) , 2 2,. Thus, we get two distinct roots iff /2. So, in this case we know ) n ) n P n A + B) n A + B. Using the boundary conditions, 0 P 0 A + B, so B A. Also, ) T P T A + B A ), and so Thus, A P n, B ) n ) n ) n ) m. n.

4 4 Random Walks Note that the last exression is even indeendent of n. It is also exonentially small in m. If 9/9 in our earlier examle, then / ) 9/0, and for any n, if m 00 dollars, then ) 00 9 Pr Win) Thus, even if we start with n 0 9 dollars, we have the same very small uer bound on the robability that we win. Note that if m 000 dollars, then Pr Win) ) , 0 so there is really no chance. On the other hand, suose n 0 dollars and T 20 dollars. In this case the robability of winning is ) 0 ) 20.26, so the odds are better, though not good. So if you re going to gamble, learn to count cards and lay blackjack, or bet it all at once! What s the intuition for this? Normally we would think that the robability of winning 00 dollars before losing 200 dollars is better than winning 0 before losing 0, i.e., that the ratio is what matters. In fact, the ratio is what matters if the game is fair, i.e., if /2. Let s look at that case. When /2, we have the characteristic equation r 2 r+ ) 0, or 2 r2 r+ 2 0, or r 2 2r + 0, or r ) 2 0. Thus, there is a double root at r. Thus, P n An) n + B) n An + B. Using the boundary conditions, 0 P 0 B imlies B 0, and P T AT + B AT imlies A. Thus, for, we obtain T 2 P n n T n n + m. So the robability of getting to T before hitting 0 is n. This is closer to what our intuition T exects. So the trouble comes when the game is not fair. Indeed, in this case, if n 200 and m 00 so T 300), we have Pr Win) On the other hand, if n 0 and m 0 so T 20), then Pr Win). Thus, actually, now we are more likely to win in 2 the first case! So the trouble is that our intuition tells us that if the game is almost fair, then we exect the results to be almost the same as if the game were fair. It turns out this is not the case! For the unbiased case, the intuition here is that it takes a long time to hit the boundary, but we will do so eventually we ll talk about time later), and the walk is symmetric about the starting amount.

5 Random Walks 5 n+m n Swing On the other hand, for the biased case, by the time you are likely to swing high, your baseline has already droed too low for the swing to hit T. The downward drift seems small, but it really does dominate the effect of the swings. So the difference between the unbiased and biased games is the drift. The swings are similar but the drift takes over in biased games. Drift 2 The time to comletion So we ve figured out the robability of winning some number of dollars before going broke. As we saw, this robability is very low for biased games. It s logical to conclude that this means that we re very likely to end the game without any money. Before we can conclude this, though, we need to rule out one other ossibility, namely, the ossibility that we lay forever. Indeed, it is ossible that you can lay forever without winning the required amount and without going broke. In fact, it turns out that the robability that you never end is 0 even for unbiased games! Theorem 2., the robability that the game never ends is 0. This seems aradoxical since there are lots of walks that go forever, yet the robability of going forever is still zero. To really understand this, we need to get into measure theory, which will say that the set of non-terminating walks is an infinite set of measure 0. In any case, we can rove the theorem now. Proof. We can assume 0 < <, since if 0 we go broke and if we win, so the game ends. Now define Q n Pr Play forever D 0 n). We first claim that Q n 0 if n 0 0 if n T Q n+ + )Q n, if 0 < n < T This is the same as our constraint on P n, excet now we have Q n 0 if n T rather than the P T we had before). Hence, Q n has the same recurrence as P n, excet the boundary condition is different. Thus, ) n Q n A + B if 2

6 6 Random Walks We can solve for A and B with the new boundary conditions, obtaining 0 A + B, so B A, and 0 A + B, so A A 0. But for, this imlies A 0. 2 Thus, A B 0, so Q n 0, so the robability we lay forever is 0. Now we check the case when 2. In this case Q n An + B. Using the boundary conditions, 0 Q 0 B, so B 0. Also, 0 Q T AT + B, so A 0. Thus, Q n 0, as desired. The receding argument tells us that the robability of laying forever is 0, but it doesn t give a very good idea of how long you should exect to lay. It turns out that you can also comute the exected time to win or go broke by using recurrences. Definition. Let S be the number of stes until we hit a boundary, that is, until we win or lose. Let E n Ex S D 0 n) be the exected time to win or lose given that we start with n dollars. 0 if n 0 Claim 3. E n 0 if n T + E n+ + )E n, if 0 < n < T Proof. The case when n 0 or T is immediate, since we have already lost or won. The case when 0 < n < T is similar to before. We have, E n Ex S D 0 n) Ex S D 0 n win st bet) Pr win st bet) + Ex S D 0 n lose st bet) Pr lose st bet) Ex S D n + ) + Ex S D n n ) ) + Ex S D 0 n + )) + + Ex S D 0 n )) ) + E n+ + + )E n + E n+ + )E n. So now we just need to solve the recurrence for E n. Rewriting, we have E n+ E n + )E n, with E 0 0 and E T 0. The difference here is just that the recurrence is inhomogeneous. First, we take a homogeneous solution, ) n E n A + B if 2,

7 Random Walks 7 which is the same as before. Next, we guess a articular solution E n an + b. Plugging in the articular solution, an + ) + b) an + b) + )an ) + b) an + a + b an b + an a + b an + a b and b is unconstrained. So we can set b 0. Thus, ) n E n A + B + n 2. 2a a a2 ) a 2, We now solve for A and B using that E 0 E T 0. We have 0 E 0 A + B, so B A. T We have 0 E T A + B + T 2), and thus A. Plugging in A and B, 2 and simlifying, we get E n n 2 T 2) ) n if 2 This exression looks messy, but we can conclude a few nice things. First, for < /2, E n n, and as T, E 2 n n. This is not surrising since we exect to lose 2 2 dollars each bet. Indeed, the exected loss er bet is ) 2. If we actually were to lose 2 dollars each bet, then we would go broke in exactly n stes. 2 We must also consider the case when /2. In that case we have the homogeneous solution E n An + B. If we were to guess E n an + b as a articular solution, we d have E n+ E n + )E n an + a + b) an b + )an a + b) an + a + b an b + an a + b an + a b 2a a 2 2 a a 0, which is clearly a contradiction. Due to the reeated root, we need to guess a higher-order olynomial. We guess E n an 2 + bn + c. Plugging this into the recurrence, some algebra shows that it works with a and b c 0. Thus, E n An + B n 2.

8 8 Random Walks Now for the boundary conditions. We have E 0 0 B, so B 0. Also, E T 0 AT T 2, so A T. Thus, E n T n n 2. But since T n+m, we have E n n+m)n n 2, or E n nm if 2. This is a very clean result, which states that in an unbiased game, you exect to lay for the roduct of the amount you re willing to lose times the amount you want to win. As a corollary, if you never quit when you re ahead, then you can exect to lay forever in a fair game. That is, as m, E n. So if you lay until you go broke, you can exect to lay forever. This is very good news if you like to gamble. We can rove this by observing that the exected time to go broke is at least the exected time to go broke or hit m for all m. Let E n be the exected time to go broke starting at n. Then for all m, E n nm, which means that E n for n > 0 and 2. This holds even for n dollar. Thus, starting with just dollar, you still exect to lay forever! Well, not exactly, more recisely the exected time to go broke is. Note that you could easily go broke, and 50% of the time this haens on the first bet. In fact, even though you exect to lay an infinite number of bets, the robability you will eventually go broke is, as we will now show. This seems imossible, but things like this can haen when the exectation is infinite. Theorem 4. If you start with n dollars and and you lay until you go broke, then for all n, 2 Pr go broke). This says that even if you start with million dollars and are laying an unbiased i.e., fair) game, with robability you will eventually lose it all. Proof. For all T, Pr go broke) Pr go broke before reach T ) Pr reach T before go broke) since Pr lay forever ) 0) n T as T In other words, the robability you go broke cannot be less than, so it must be. So even if you lay a fair game and choose not to quit, you will eventually go broke. n Crash!

9 Random Walks 9 The result is not true if > /2. In this case there is a non-zero chance that you can lay forever and never go broke. We ll cover this next year. 3 Aendix In lecture the following question concerning a generalization of the gambler s ruin roblem was asked. Suose there is a finite sequence z, z 2,..., z s of distinct integers, such that each z i. Let, 2,..., s be a sequence of real numbers with 0 < i < for all i, and s i i. Consider a variation of gambler s ruin in which at each ste, the amount of money the gambler has changes by z i which may be negative) with robability i. Define the downward drift d to be d s i z i. i Suose you start with n > dollars. The question is what the exected time E n is until you end u with 0 dollars, assuming you never quit otherwise. If no z i is equal to, then since the z i are integers, they are all non-negative. Clearly in this case E n since you will never end u with less than n dollars. Assume, then, w.l.o.g. that z. By our earlier arguments, we know E + i> i E +zi. Consider E n for any integer n >. Let R i be the number of stes in between the first time you have i dollars and the first time you have i dollars. By linearity of exectations, E n n i Ex R i). By symmetry, Ex R i ) E for all i. Thus, E n ne. Thus, E + i> i + z i )E, and thus either E is infinite or E d. i> i + z i ) i> i i> iz i ) d )) + + d

10 0 Random Walks Thus, either E n is infinite, or is n. Let P d i be the robability we don t go broke until at least i stes. Now, E n is just P i. i One can show using Chernoff bounds that for a sufficiently large integer i 0, for all i > i 0 P i c i, for some constant c >. Thus, we can uer bound the above sum as, i 0 i0 P i + c i. The first sum is over a finite number of real numbers, and is thus finite, and the second sum is a geometric series, and is thus finite. Thus, E n is finite. The result, however, does not hold if you allow some of the z i to be less than. For instance, if z, z 2 2,, and 2, then E, E 2 +, and E In this case the above would say that E n n/2 ), but this is imossible unless. i 0 +