Probability Models - Notes2 Conditional robability Definition If A and B are events and PB) > 0 then we define the conditional robability of A given B to be PA B) = PA B) PB) Definition Events B 1,...,B n are said to artition the samle sace S if n i=1 B i = S and B i B j = ϕ for all i j. So the events are mutually exclusive and exhaustive) The law of total robability Let E be an event in S and let B 1,...,B n artition S. Then PE) = n PE B j )PB j ) j=1 You derived this result and looked at simle examles using this law in Probability 1 Examle: A fair dice is thrown twice. Find PE) where E is the event that the roduct of the numbers on the die from the first and second throw is even. Let B 1 be the event that the number on the first throw is odd and B 2 be the event that the number on the first throw is even. Then B 1,B 2 form a artition of S. PB 1 ) = PB 2 ) = 1 2. If the number on the first throw is even then the roduct is certain to be even. So PE B 1 ) = 1 2. If the number on the first throw is odd, then the roduct will be even if the number on the second throw is even, so PE B 2 ) = 1. Hence PE) = PE B 1 )PB 1 ) + PE B 2 )PB 2 ) = 1 1 2 + 1 2 1 2 = 3 4 Use of the law of total robability for seuences of indeendent trials or games Consider an indeendent seuence of throws of a die. We throw the die until the number on the die is either a 6 or is less than or eual to 2, when we sto. Let E be the event that we sto with a throw of 6. Find PE). A useful aroach is to look back to the first throw of the die. Let B 1, B 2 and B 3 corresond to the event that the first throw gives resectively 6, less than or eual to 2 and neither 6 nor less than or eual to 2. Then B 1,B 2,B 3 is a artition. PE) = PE B 1 )PB 1 ) + PE B 2 )PB 2 ) + PE B 3 )PB 3 ) Now PB 1 ) = 1 6, PB 2) = 1 3 and PE 3) = 1 2. Also PE B 1) = 1 and PE B 2 ) = 0. If B 3 occurs then after the first throw we are essentially in the same situation statistically as we were at the outset. We have a seuence of indeendent throws and will continue until the number thrown euals 6 or is less than 2. So PE B 3 ) = PE). Hence 1
PE) = 1 6 + PE)1 2 Therefore solving gives PE) = 1 3. An alication of this method is the gambler s ruin roblem and the random walk on a line. The Random Walk on a Line and the Gambler s Ruin Problem. Definition A random walk RW) on a line is the following rocess. A article moves along a line visiting only integer oints. The jums occur at integer time moments t = 0, 1, 2,... If at time t it is at oint n then at time t + 1 it will jum either to n + 1 or to n 1. The robability of juming from n to n + 1 is and the robability of juming from n to n 1 is, where + = 1. The random direction of the jum does not deend on revious jums. Remark In mathematical literature such RWs are often called simle random walks. In these notes we shall use the abbreviated term random walk. Denote by X t the coordinate of the RW at time t. The above definition means that X t changes randomly as the time rogresses according to the following rule: P{X t+1 = n + 1 X t = n} = P{X t+1 = n 1 X t = n} = and these robabilities do not deend on the history of the walk, that is to say on the values X 0,X 1,...X t1. Euivalently one can say that X t+1 = X t +Y t+1, where Y 1,Y 2,... are indeendent random variables each taking values 1 with robability and 1 with robability. Thus if X 0 is the starting osition of the walk then X 1 = X 0 +Y 1, X 2 = X 0 +Y 1 +Y 2 and it is easy to see that Exercise. Prove this relation. X t = X 0 +Y 1 +Y 2 + +Y t, We shall consider the following uestions concerned with the behaviour of the RW. 1. Consider M n N. What is the robability that the RW starting from n would reach N before visiting M? Would reach M before N? 2. Consider M n. What is the robability that the RW starting from n would reach + before visiting M? 3. Suose that X 0 = 0. What is the robability that the RW would ever return to 0? 4. Let M n N. If X 0 = n denote by T n the time at which the RW reaches either N or M. What is the exectation of T n? In other words what, is the mean duration of the walk?) Before solving these roblems we discuss a model which is euivalent to that of a RW. 2
The Gambler s Ruin Problem. A gambler starts with a caital k. He lays a seuence of games. At each game he has a robability of winning and robability = 1 of losing the game. If he wins then he receives 1 and if he loses then he ays 1. He decides that he will sto if his caital either grows to N or declines to M in the original roblem M = 0 so he goes broke, i.e. is ruined). What is the robability that at the end of the series of games his caital would be M? Would be N? If he lays against a casino, what is the robability that he wins an infinite sum of money? To understand the relation between these two models note that if X t is the gambler s caital after t games, then X t+1 = X t +Y t+1, where Y t is a random variable which reresents the amount he gains in the t + 1)st game, which is 1 with robability and 1) with robability. Thus X t is a random variable which behaves exactly like the coordinate of the random walk in the model considered above. In articular the robability that at the end of the series of games his caital would be N is the same as the robability that a article starting from k would reach N before M. Exercise. State the analogue of uestions 2 and 4 in terms of the gambler s Ruin Problem. Solutions. QUESTION 1. Let A n be the event that the RW starting from n reaches N before visiting M. Set r n = PA n ). Theorem The robabilities r n, M n M satisfy the following system of euations: r n = r n+1 + r n1, if M < n < N 1) r M = 0, r N = 1 2) Proof We condition on the outcome of the first jum, so B 1 and B 2 are the events the first jum is to the right and the first jum is to the left resectively. Then PB 1 ) =, PB 2 ) =. Next, B 1, B 2 form a artition and we therefore can use the Total Probability Law: PA n ) = PB 1 )PA n B 1 ) + PB 2 )PA n B 2 )PB 2 ) 3) If B 1 occurs then the walk jums to n +1 and the rocess is in the same situation as initially but is now starting from n + 1. Hence PA n B 1 ) = PA n+1 ) = r n+1. Similarly, if B 2 occurs then the walk jums to n 1 and therefor PA n B 2 ) = PA n1 ) = r n1. Therefore euation 3) can be rewritten as r n = r n+1 + r n1 which roves that the main euation 1) in the statement of the Theorem holds. To rove 2) remember the walk stos immediately when it reaches M or N. Hence the robability to reach N starting from N is one: r N = 1. And the robability to reach N starting from M is zero: r M = 0. The Theorem is roved. SOLVING EQUATIONS 1), 2). Euation 1) is just a simle second order difference euation. Its artial solutions can be found in the form r n = θ n, where θ satisfies the associated uadratic euation θ 2 θ + = 0. If the latter euation has two distinct roots θ 1 θ 2 then the general solution to 1) is given by r n = aθ n 1 + bθn 2. 3
If θ 1 = θ 2 then the general solution to 1) is given by r n = a + bn)θ n 1. We note that in our case the roots are θ 1 = 1 and θ 2 =. The roots will be eual if = = 1 2. We thus have to consider two cases. Case when 1 2. The solution to the difference euation is ) n r n = a1 + b = a + b Since 0 = r M = a + b ) M and 1 = rn = a + b, we obtain the solution r n = ) M ) M 4) Case = 1 2. The solution to the difference euation is r n = a + bn)1 = a + bn. Since 0 = r M = a + bm and 1 = r N = a + bn, we obtain the solution r n = n M N M Similarly, let F n be the event that the walk reaches M before N and set l n = PF n ). Then l n satisfies the same difference euation as r n, namely l n = l n+1 + l n1 if M < n < N. However, the boundary conditions are different: l M = 1 and l N = 0. 5) Case when 1 2. The solution is l n = ) M 6) Remark. Note that r n + l n = 1 so the series of games are certain to finish. Case = 1 2. The solution is l Remark. Again r n + l n = 1. n = N n N M 7) If we indicate in the notation the boundaries M and N then we relace r n by r n M,N) and l n by l n M,N) in the results above. QUESTION 2. To answer this uestion we observe first that r n M, ) def = P{reach + before M} = r nm,n). We write r n M,N) to emhasize the deendence in 4) on n, M, and N. However, n, M will be fixed while N +. The answer deends on the relation between and. Case when >. Then < 1 and ) N = 0. 4
Therefore r nm,n) = Case when <. Then > 1 and ) M ) M = ) N =. ) M 0 ) M = 1 ) nm. Therefore ) M ) M r nm,n) = Case when =. Then one obtains from 5) that ) M = = 0. r n M nm,n) = N M = 0. The three cases can be summarized as follows: nm 1 r n M, ) = ) if < 0 othewise. Exercise. Prove that l n,n) def = P{reach before N} = M l nm,n) = ) Nn 1 if < 0 othewise and ) Nn r n,n) def = P{reach N before } = r nm,n) = if < M 1 othewise. QUESTION 3. Let B 1, B 2 be the events that a RW starting from 0 makes its first ste to the right or left resectively. Let R denote the event that the RW returns to 0. Then by TPF PR) = PB 1 )PR B 1 ) + PB 2 )PR B 2 ) = l 1 0,+ ) + r 1,0). The above formulae allow us to solve the roblem immediately. However, we need one more ingredient, namely l 1 0,+ ). One can either comute it directly as above) or to use the relation l 1 0,+ ) = 1 r 1 0,+ ). As before, the answer will deend on the relation between and. E.g. suose that <. Then l 1 0,+ ) = 1 r 1 0,+ ) = 1 0 = 1. Next, r 1,0) = n ) 01) = =. Thus PR) = 1 + = 2. Similarly, if >, then PR) = 2. Note that PR) = 1 if and only if = = 0.5. Summarizing, we can state that { 2 if <, PR) = 2 othewise. 5