Markov Chains in the Game of Monopoly

Transcription

1 April 1, 2013

2 Markov Chains Markov Chain: random process containing a sequence of variables X 1, X 2, X 3,..., X r such that given the present state, the future state is conditionally independent of past states. p(x t+1 = j X t = i t )

3 Markov Chains Examples: Games of chance Drunkard s walk Google PageRank Asset pricing models Baseball analysis

4 State of Economy Example Figure: Directed Graph Figure: Transition Matrix

5 Long Term Markov Chain Behavior Transition Matrix: To n From 1 a 1,1 a 1,2 a 1,n 2 a 2,1 a 2,2 a 2,n n a n,1 a n,2 a n,n

6 Long Term Markov Chain Behavior Define p as the probability state distribution of ith row vector, with transition matrix, A. Then at time t = 1, pa = p 1

7 Long Term Markov Chain Behavior Define p as the probability state distribution of ith row vector, with transition matrix, A. Then at time t = 1, pa = p 1 Taking subsequent iterations, the Markov chain over time develops to the following (pa)a = pa 2, pa 3, pa 4

8 State of Economy Example For example if at time t we are in a bear market, then 3 time periods later at time t + 3 the distribution is, pa 3 = p 3 [ ] = [ ]

9 Long Term Markov Chain Behavior To determine stationary state distributions, we must find a probability distribution p which satisfies the condition pa = p a 1,1 a 1,2 a 1,n [ ] a 2,1 a 2,2 a 2,n p(1) p(2) p(n) = [ p(1) p(2) p(n) ] a n,1 a n,2 a n,n

10 Long Term Markov Chain Behavior However, there is an easier way to determine stationary probability distributions. Let s reverse our thinking and consider the probability of being in a certain state at t + 1. p(1) =.9p(1) +.15p(2) +.25p(3), p(2) =.075p(1) +.8p(2) +.25p(3), p(3) =.025p(1) +.05p(2) +.5p(3), with the condition, p(1) + p(2) + p(3) = 1

11 Four Square Circuit To From 1 1/6 3/6 2/ /6 3/6 2/ /6 3/6 1/ /6 3/6 1/6 0

12 Four Square Circuit After the first throw, the probabilities of landing on each square are: p 1 (1) = 1 6 p 1 (2) = 1 2 p 1 (3) = 1 3 p 1 (4) = 0 After two throws, the probabilities of landing on each square are: p 2 (1) = 2 9 p 2 (2) = 1 2 p 2 (3) = 5 18 p 2 (4) = 0

13 Four Square Circuit Let p t(n) represent the probability of landing on square n after t die rolls. p 0 (1) = 1, p 0 (2) = p 0 (3) = p 0 (4) = 0. p t+1 (1) = 1 6 pt(1) pt(2) pt(3) pt(4), p t+1 (2) = 3 6 pt(1) pt(2) pt(3) pt(4), p t+1 (3) = 2 6 pt(1) pt(2) pt(3) pt(4), p t+1 (4) = 0.

14 Four Square Circuit p(1) = 1 6 p(1) p(2) p(3) p(2) = 3 6 p(1) p(2) p(3) p(3) = 2 6 p(1) p(2) p(3) p(4) = 0 with the condition, p(1) + p(2) + p(3) + p(4) = 1

15 Four Square Circuit row reduce echelon form p(1) = 3 14 p(2) = 1 2 p(3) = 2 7 p(4) = 0

16 Application to Monopoly Modifications 40 squares Doubles Rule Community Chest and Chance Cards

17 Application to Monopoly Modifications 40 squares Doubles Rule Community Chest and Chance Cards Markov Chain with 3 40 = 120 states

18 Stable Probabilties

19 Monopoly Strategy Considerations Rent Earnings Probability of Landing on Property Development Costs

20 Monopoly Strategy Analyze by probability of landing on a square for a single turn, not a roll. p(1) = 30 36, p(2) = 6 ( ) 30, p(3) = 6 ( ) 6 (1) ( ) ( 30 6 E[X ] = ) ( ) = = 1.194

21 Monopoly Strategy Consider the following inequality. Revenue Cost

22 Monopoly Strategy Consider the following inequality. Revenue Cost p(n) R E[X ] Turn Cost

23 Monopoly Strategy Consider the following inequality. Revenue Cost p(n) R E[X ] Turn Cost ( ) Cost Turn = p(n) R E[X ]

24 Monopoly Strategy

25 Monopoly Strategy

26 Monopoly Strategy Color Investment Turn Orange Hotel 20 Light Blue Hotel 25 Dark Blue 3 House 29 Maroon 3 House 29 Red 3 House 29 Yellow 3 House 30 Railroad All 4 32 Green 3 Houses 34 Purple Hotel 37

27 Jorg Bewersdorff, Luck, Logic and White Lies: The Mathematics of Games, A K Peters (2005), J. Laurie Snell Finite Markov Chains and their Applications, The American Mathematical Monthly (1959), 66 (2),