Be sure this exam has 1 pages including the cover The University of British Columbia Sessional Exams 11 Term Mathematics 33 Introduction to Stochastic Processes Dr. D. Brydges Last Name: First Name: Student Number: This exam consists of 8 questions worth 1 marks each so that the total is 8. No aids are permitted. Please show all work and calculations. On some parts no explanation will mean no marks. Numerical answers need not be simplified. Problem total possible score 1. 1. 1 3. 1 4. 1 5. 1 6. 1 7. 1 8. 1 total 8 1. Each candidate should be prepared to produce his library/ams card upon request.. Read and observe the following rules: No candidate shall be permitted to enter the examination room after the expiration of one half hour, or to leave during the first half hour of the examination. Candidates are not permitted to ask questions of the invigilators, except in cases of supposed errors or ambiguities in examination questions. CAUTION - Candidates guilty of any of the following or similar practices shall be immediately dismissed from the examination and shall be liable to disciplinary action. a Making use of any books, papers or memoranda, other than those authorized by the examiners. b Speaking or communicating with other candidates. c Purposely exposing written papers to the view of other candidates. The plea of accident or forgetfulness shall not be received. 3. Smoking is not permitted during examinations.
April 4 1 Math 33 Final Exam Page of 1 1. Consider the Markov chain X n, n =, 1,.. whose seven states and transitions of positive probability are as indicated in the diagram. 1 7 4 3 6 5 3 points a What are the communicating classes? {1,, 3, 4}, {5, 6, }, {7} 1 points b Does this Markov chain have a stationary distribution? Briefly explain. π i = { 1 i = 7 else is a stationary distribution. 1 points c Suppose X = 1. Does lim n Pi,j n exist when i = 1? Briefly explain. No. If we start the chain in state 1 then it stays in states {1,, 3, 4} and with only these states it is irreducible but in the next part you find it is not aperiodic. 3 points d Define the period of a state and use your definition to determine the periods of the recurrent states. The period of state i is the greatest common divisor of the times for which return to i is possible, in other words d i = gcd{n 1 : P i X n = i > }. The states {1,, 3, 4} have period and 7 has period one. The remaining states {5, 6} are transient. points e Suppose PX = i = ρ i for every state i where ρ i, i = 1,,..., 7 are given probabilities. Give a formula for PX 1 = 7 in terms of ρ i and the transition probability matrix P ij. i ρ ip i,7 or ρ 6 P 6,7 + ρ 7 P 7,7
April 4 1 Math 33 Final Exam Page 3 of 1. A soap opera has N characters. In each episode some are evil and the others are good. Before making each episode the director tosses a coin which has probability p of showing heads. If the result is heads then a character is chosen at random and, if evil, becomes good, if good, remains good; if the result is tails then a character is chosen at random and, if good, becomes evil, if evil, remains evil. For n = 1,,..., let X n denote the number of good characters in episode n. For n = set X n =. points a Draw the transition diagram of the Markov chain X n n when N = 4, showing the transitions of positive probability. You need not insert the probabilities. 1 3 4 3 points b Find the nonzero transition probabilities P ij. For N = P i,i+1 = p N i, i =, 1,..., N 1, N P i,i 1 = 1 p i, i = 1,..., N, N P i,i = p i i + 1 pn, i =,..., N. N N P,1 = p, P 1, = p, P 1, = 1 1 p, P,1 = 1 p, P, = 1 p, P 1,1 = 1, P, = 1 p 5 points c This soap opera has run for a very long time. Find, as a formula involving the number of characters N, the proportion of episodes in which there are exactly two good characters. What is the answer for N =? As we saw in MT problem an MC with states that are connected in a line is reversible. In this case we want π where π i solves equations of detailed balance,
April 4 1 Math 33 Final Exam Page 4 of 1 π i P ij = π j P ji. π p = π 1 1 p 1 N p N, π 1 = π 1 1 p π 1 p N 1 N = π 1 p N p N, π = π 1 p π p N N = π 31 p 3 N p N, π 3 = π 3 1 p πi = 1 and binomial theorem leads to π = 1 p N. 3 π i = N p i 1 p N i, π = i N p 1 p N = p for N=.
April 4 1 Math 33 Final Exam Page 5 of 1 3. An urn contains four balls, which can be black or white. At each time n =, 1,,... a ball is chosen at random from the urn and replaced by a ball of the opposite colour. 3 points a What does it mean to say that a state is positively recurrent? Are all the states in this Markov chain positively recurrent? A state is positively recurrent if it is recurrent and the expected time to return is finite. In this case there is one class of recurrent states. Theorem: recurrent states are positively recurrent when there are finitely many states. Therefore answer to second part is YES. points b For this Markov chain, running forever, all four balls are white for 1 16 of the times. If we start the sytem in the state where all the balls are white, what is the expected time before they are again all white? 16 because π all white = 1 ET all white. 5 points c Initially there is one white ball and three black balls. What is the probability that they all become white before they are all black? Hint. Gamblers ruin. For i =, 1,, 3, 4, starting with i white balls, let p i be the probability that all the balls become white before they all become black. As in gamblers ruin, there are some boundary conditions and playing one game leads to equations, p 4 = 1, p =, p 1 = 1 4 p + 3 4 p, p = 4 p 1 + 4 p 3, p 3 = 3 4 p + 1 4 p 4, These can be solved. The fast way is that by symmetry p = 1, so p 1 = 3 1 4 = 3 8.
April 4 1 Math 33 Final Exam Page 6 of 1 4. Let S n be the position at time n of symmetric simple random walk on Z with S =. 4 points a Thinking of steps to the left and the right what is the formula for PS n = in terms of n? n PS n = = n }{{} choose which steps 1 n }{{} prob with steps specified. 1 points b Let N = #{n : S n = } be the number of visits to the origin. State and prove the formula that relates EN to PS n =. Hint. Indicator functions. N = n I Sn= EN = n EI Sn= = n PS n = 3 points c Define what it means for state to be recurrent. What is the relation to EN? Probability of return to the origin is one. Theorem: recurrent iff EN =. points d Prove that the origin is recurrent. n! πnn n e n. is divergent because n n 1 EN = n n = n! n! EI Sn= = n 1 n πn πn and n> 1 n is a p series with p 1. PS n = = n n n e n n n e n n n 1 n 4 1 n πn = = 1 πn πn 5
April 4 1 Math 33 Final Exam Page 7 of 1 5. Bacteria reproduce by cell division. In a unit of time, a bacterium will either die with probability 1 4, stay the same with probability 1 4, or split into bacteria with probability 4. Interpret this as a branching process. Let Z n be the number of bacteria at time n =, 1,,... and let G n s be the generating function for Z n. points a Suppose initially there is only one bacterium. Find G 1 s as a function of s. G 1 s = 1 4 + 1 4 s + 4 s 4 points b Suppose initially there is only one bacterium. Find G s as a function of s. Since Z = 1, Gs = G 1 s and G s = GG 1 s = 1 4 + 1 4 1 4 + 1 4 s + 4 s + 1 4 4 + 1 4 s + 4 s. I am happy with this answer but it can be multiplied out to get 11 3 + 1 8 s+ 9 3 s + 1 8 s3 + 1 8 s4. There is less good way to get this by finding PZ = i for all i by hand. points c Suppose that initially there are 1 bacteria. What is the expected number in generation n. 1µ n where µ = Eξ = 1 4 + 1 1 4 + 4 = 5 4. points d Suppose that initially there are 1 bacteria. What is the probability that the population goes extinct. The solutions of s = 1 4 + 1 4 s + 4 s are 1 and 1. Therefore extinction probability starting with one bacterium is 1. Starting with 1 is 1 1.
April 4 1 Math 33 Final Exam Page 8 of 1 5 points 6. a One of the standard definitions of a Poisson process Nt of rate r, involves a statement about Ns + h Ns when h is small. Write out this definition 1 explaining the meaning of terms in it such as increment and oh. The Poisson process with rate r is a non-negative integer valued process Nt such that 1. N =. Nt Ns and Nt 1 Ns 1 are independent random variables when the intervals s 1, t 1 and s, t are disjoint. 3. 1 rh oh n =, PNs + h Ns = n = rh + oh n = 1, oh n > 1. oh denotes a function fh that does not depend on s such that fh/h as h. 6 points b Suppose that crimes in a large city occur according to a rate r Poisson process Nt. For each crime, the criminal responsible for the crime is sent to prison with probability 1/5. Ignoring the delay between the crime and the beginning of the prison sentence, what process P t describes the arrivals of new prisoners? What assumption not yet mentioned are you making? Assume that each criminal is independently sent to prison with probability. Then P t is a rate r/5 Poisson process. 1 5 3 points c Partly justify your answer in part b by showing that P t satisfies a Ns + h Ns axiom as in part a. With p = 1/5, P P s + h P s = 1 = 7 pp N 1 s + h N 1 s = 1 + p1 pp N 1 s + h N 1 s = + 8 = prh + h 9 Similarly P P s + h P s = oh which verifies that P t satisfies axiom 3 for a rate r/5 Poisson process. 1 This term in class I presented this as a theorem to reduce confusion the book gives two definitions.
April 4 1 Math 33 Final Exam Page 9 of 1 7. Blackbirds come to a bird feeder according to a Poisson process Bt with rate b per hour; robins come to the same bird feeder according to an independent Poisson process Rt with rate r per hour. 4 points a What is the probability that exactly four birds arrive between 1 and 1:3pm? The combined process Nt is a rate b + r process. PN1/ = 4 = b+r 4 e b+r. 4! points b Exactly one bird arrived during the first hour. What is the probability that it actually arrived during the first quarter of an hour. Conditional on Nt = 1 the arrival time is uniform in, t so the answer is 1 4. Or P N 1 4 = 1 N1 = 1 = P N 1 4 = 1, N1 = 1 P N1 = 1 = r/4e r/4 e 3r/4 re r = 1 4 = P N 1 4 = 1 P N 3 4 = P N1 = 1 points c Suppose that the first robin arrives at time S 1 and the second robin arrives at time S. What is the probability that S > 4S 1? Write S = S 1 + T where T is the time between the arrival of the first and the second robins. S 1 and S are independent Exp r. P S 1 + T > 4S 1 = P T > 3S 1 = I t>3s re rt re rs dt ds = re rt dt re rs ds = e r3s re rs ds = re 4rs ds = 1 4 3s points d What is the probability that exactly one blackbird arrives before the first robin? t n e at dt = n!a n 1. Let T R be the time to first robin. Condition on T R = t P BT R = 1 = = P BT R = 1 T R = t re rt dt = bt 1! e bt re rt dt = br te b+rt dt = P Bt = 1 re rt dt br b + r
April 4 1 Math 33 Final Exam Page 1 of 1 Or use memoryless property, restarting the Poisson process after the first robin arrives, P S B,1 < S R,1, S B, > S R,1 = P SB, > S R,1 S B,1 < S R,1 P SB,1 < S R,1 = P r b S B,1 > S R,1 P SB,1 < S R,1 = r + b r + b
April 4 1 Math 33 Final Exam Page 11 of 1 8. Students arrive at a help centre according to a rate r Poisson process. When there are n 1 students in the centre, the first one to leave does so at a random Exp r time. points a Let X t be the number of students in the centre at continuous time t. Regarding this as a birth/death process, what are the parameters λ n and µ n for n = and for n = 1,,...? λ n = r, n =, 1,..., { r n = 1,,... µ n = n = points b For n =, 1,,..., if there are presently n students in the centre, what is the expected time to the next transition? The minimum of an Exp r and an Exp r is an Exp r + r so { 1 r+r n = 1,,... 1 r n = points c For n =, 1,,..., if there are presently n students in the centre, what is the probability that the next transition will be to n + 1 students? { r r+r n = 1,,... 1 n = 4 points d Suppose that there are presently no students in the centre. What is the expected time until there are two students? Let T be the random time until there are two students, starting with none, let t = 1 r, t = 1 r be the expected times in states, 1 before next transition. Write an equation for ET by thinking about the additional time to reach state after reaching state 1: if the next transition is to then it is the time in state 1 plus nothing; if the next transition is to it is the time in state 1 plus the time to reach if you start in so ET = t + P 1, t 1 + P 1, t 1 + ET = t + t 1 + P 1, ET
April 4 1 Math 33 Final Exam Page 1 of 1 where we know t 1 from part b and P 1, = 1 P 1, from part c. Solve for ET, ET = 1 1 t + t 1 = 3 1 P 1, r + 1 = 4 3r r