CHAPTER 3 - GAME THEORY

Transcription

1 CHAPTER 3 - GAME THEORY Game theory studies the actions of rational individuals who can take various actions, when their payoffs depend on the actions of other rational individuals. For example, roulette is not a game under this definition, since the payoff of an individual depends only on his/her action and the result of an experiment (the roll of the wheel). Such situations are often called games against nature. However, the lottery is a game. The probability of winning the grand prize is fixed. However, if I pick numbers that other people do not pick, when I choose the correct numbers I can win more. 1 / 50

2 Game Theory - Introduction Games can either be deterministic (e.g. chess, naughts and crosses) or stochastic (e.g. backgammon, bridge). In games against nature, we can use standard optimisation theory to maximise the expected payoff (or utility) of an individual. 2 / 50

3 Game Theory - Concepts of Solution Many concepts of solution have been proposed for games. 1. Minimax solutions. 2. Nash equilibria. 3. Evolutionary stable strategies. 4. Correlated equilibria (requires communication). 5. Cooperative solutions (require communication and transfer of payments) 3 / 50

4 Game Theory - 2-Person Bimatrix Games Suppose Player i can choose from a finite set of actions S i. The actions available to Player 1 are labelled A 1, A 2,..., A m. The actions available to Player 2 are labelled B 1, B 2,..., B n. The reward obtained by Player i when Player 1 takes action a and Player 2 takes action b is denoted R i (a, b). 4 / 50

5 Game Theory - 2-Person Bimatrix Games The payoffs can be illustrated in matrix form. The actions taken by Player 1 correspond to the rows. The actions taken by Player 2 correspond to the columns. For example, if both players can take actions H and D (the Hawk-Dove game) ( H D ) H ( 3, 3) (2, 0) D (0, 2) (1, 1) i.e. when Player 1 plays H and Player 2 plays D, then Player 1 obtains a payoff of 2 and Player 2 obtains a payoff of 0. 5 / 50

6 Game Theory - 2-Person Bimatrix Games A game is symmetric if S 1 = S 2 (i.e. the players have the same set of strategies available to them) and R 1 (a, b) = R 2 (b, a). This definition can be easily extended to other types of game. The Hawk-Dove game given above is symmetric. 6 / 50

7 Game Theory - Fixed Sum Games If R 1 (a, b) + R 2 (a, b) = k, for all a S 1 and b S 2, then a game is fixed sum game. In such a game, there is a fixed amount of pie available, so by maximising one s portion of the pie, one is also minimising the amount of pie the other player receives. Adding a constant to the payoffs will not change the solution of a game (whatever concept in the list above is used). Hence, by subtracting k 2 from the payoffs of an individual in a fixed-sum game, we obtain a zero-sum game. For a zero-sum game R 2 (a, b) = R 1 (a, b). 7 / 50

8 Game Theory - Concepts of Strategies A person using a pure strategy always chooses the same action. A person using a mixed strategy chooses the action to be taken from a specified distribution. For example, in the rock-scissors-paper game it is clear that a player does not want to give his/her opponent any information about the action that he/she is going to take. Hence, he/she chooses each of the actions with probability / 50

9 Game Theory - Expected Payoffs Under Mixed Strategies When one or both players use a mixed strategy, this leads to a distribution over the pairs of actions taken. When there is no communication, it is assumed that the actions are independent. For example, if Player 1 and Player 2 play H with probability p and q, respectively, the joint distribution over the action pairs is given below: ( H D ) H pq p(1 q) D (1 p)q (1 p)(1 q) 9 / 50

10 Game Theory - Expected Payoffs Under Mixed Strategies The expected payoffs of the players are calculated according to this joint distribution. Denoting the mixed strategies used by p and q, we have: R 1 (p, q)= 3pq + 2p(1 q) + (1 p)(1 q) = 1 + p q 4pq R 2 (p, q)= 3pq + 2q(1 p) + (1 p)(1 q) = 1 + q p 4pq 10 / 50

11 Concepts of Solution - Minimax Strategies Both players attempt to maximise the minimum gain they can make. We can calculate the strategy of Player 2 that minimises Player 1 s payoff for any particular value of p by differentiating R 1 (p, q) with respect to q. This gives Player 1 s minimum possible payoff as a function of p. By maximising this function, we find Player 1 s minimax strategy. 11 / 50

12 Concepts of Solution - Minimax Strategies We have R 1 q R 1 (p, q)=1 + p q 4pq = 1 4p Since p [0, 1], it follows that R 1 q < 0. Hence, in order to minimise the payoff of Player 1, Player 2 should always choose q to be as large as possible (i.e. q = 1, always play Hawk). 12 / 50

13 Concepts of Solution - Minimax Strategies Given Player 2 always plays H, R 1 (p, 1) = 3p. It follows that Player 1 maximises his minimum possible reward by choosing p = 0. Hence, the minimax strategy of Player 1 is to always play D. The maximum payoff Player 1 can ensure is / 50

14 Concepts of Solution - Minimax Strategies Since the game is symmetric, it follows that the minimax strategy of Player 2 is to always play D. The maximum payoff Player 1 can ensure is 0. It should be noted that when both players play their minimax strategy, they both obtain a payoff of 1 (a greater payoff than either player can ensure themselves). 14 / 50

15 Concepts of Solution - Minimax Strategies - Advantages 1. There is a unique minimax solution to a game. 2. The solution is relatively simple (in general the problem can be presented as a linear programming problem). 3. This concept seems very logical when we consider fixed-sum games. By maximising one s payoff, one is simultaneously minimising the payoff of the other player. Hence, it is logical to assume that the other player will attempt to minimise one s payoff. 4. Such an approach can be used when a player has no information on the payoffs of the other player. 15 / 50

16 Concepts of Solution - Minimax Strategies - Disadvantages 1. The minimax payoff is not a clear concept in variable sum games. For example, by playing their minimax strategies in the Hawk-Dove game, players obtain a larger payoff than they can ensure themselves. What should we accept as the minimax payoff? 2. In many variable sum games players can gain by coordinating their actions. In such cases, it does not seem logical to assume that the other player aims to minimise one s payoff. 16 / 50

17 Concepts of Solution - Pure Nash equilibria A pair of pure strategies (a, b ) is a Nash equilibrium iff R 1 (a, b ) R 1 (a, b ) R 2 (a, b ) R 2 (a, b) for all a S 1 and b S 2. That is to say, it does not pay either of the individuals to unilaterally change their action. It should be noted that a is the best response to b and vice versa. To look for a pure Nash equilibrium we use the following fact: Player 1 must obtain the maximum reward possible in the column corresponding to b and Player 2 must obtain the maximum reward possible in the row corresponding to a. 17 / 50

18 Concepts of Solution - Mixed Nash equilibria Let M 1 and M 2 be the sets of mixed strategies of Players 1 and 2, respectively, i.e. the set of probability distributions over the available actions. A pair of mixed strategies (a, b ) is a Nash equilibrium iff for all a M 1 and b M 2. R 1 (a, b ) R 1 (a, b ) R 2 (a, b ) R 2 (a, b) 18 / 50

19 Concepts of Solution - Nash equilibria Suppose at a pure Nash equilibria Player 2 plays b 1 and Player 1 plays a 1. Consider what happens when Player 2 plays b 1 and Player 1 uses a mixed strategy: play a i with probability p i, i = 1, 2,..., n. The pair of actions taken is (a i, b 1 ) with probability p i. It follows that the payoff of Player 1 is p 1 R 1 (a 1, b 1 ) + p 2 R 1 (a 2, b 1 ) p n R 1 (a n, b 1 ). By assumption a 1 is the best response to b 1. Hence, to maximise his payoff Player 1 should choose a 1 with probability / 50

20 Concepts of Solution - Nash equilibria It follows that a 1 is Player 1 s best response out of the set of possible mixed strategies. Arguing similarly, b 1 is Player 2 s best response out of the set of possible mixed strategies. Hence, if (a 1, b 1 ) is a pure Nash equilibrium, it is a Nash equilibrium in the set of all mixed strategies. If a strategy involves randomisation, it will be referred to as essentially mixed. 20 / 50

21 The Bishop-Cannings Theorem We say that the action a i is in the support of strategy π, if the probability of playing a i is greater than 0. Suppose (π 1, π 2 ) is an essentially mixed equilibrium. Suppose a i and a j are in the support of π 1, but a k is not. We have R 1 (a i, π 2) = R 1 (a j, π 2) = R 1 (π 1, π 2) R 1 (a k, π 2) < R 1 (π 1, π 2) 21 / 50

22 The Bishop-Cannings Theorem The intuition behind this is as follows: Suppose both a i and a j may be chosen by Player 1 at an essentially mixed ESS. We cannot have R 1 (a i, π2 ) > R 1(a j, π2 ), since otherwise Player 1 would choose a i rather than a j. The Bishop-Cannings theorem is very useful in deriving essentially mixed equilibria and the expected payoffs gained (the Nash values). 22 / 50

23 Example 1: The Hawk-Dove Game - Pure Nash equilibria ( H D ) H ( 3, 3) (2, 0) D (0, 2) (1, 1) Player 1 s greatest payoff in column 1 (0) is obtained at (D, H). This is also Player 2 s greatest payoff in row 2. Hence, (D, H) is a pure Nash equilibrium. Player 1 s greatest payoff in column 2 (2) is obtained at (H, D). This is also Player 2 s greatest payoff in row 1. Hence, (H, D) is a pure Nash equilibrium (note that this follows from the symmetry of the game). Thus (0, 2) and (2, 0) are Nash values of the game. 23 / 50

24 Example: The Hawk-Dove Game - Mixed Nash Equilibria We now look for an essentially mixed Nash equilibrium. By symmetry, both players will play the same strategy: play Hawk with probability p, otherwise play Dove. This strategy is denoted ph + (1 p)d. Suppose Player 2 plays such an equilibrium strategy. Player 1 is indifferent between playing H and playing D. Hence, R 1 (H, ph + (1 p)d)=r 1 (D, ph + (1 p)d) 3p + 2(1 p)=1 p p = / 50

25 Example: The Hawk-Dove Game - Mixed Nash Equilibria Hence, at the mixed Nash equilibrium of this both players play Hawk with probability 1 4 and Dove with probability 3 4. The Nash value of this equilibrium is given by R 1 (ph+(1 p)d, ph+(1 p)d) = R 1 (D, ph+(1 p)d) = 1 p = / 50

26 Nash Equilibria - Advantages 1. The concept of Nash equilibrium can be sensibly applied to fixed-sum games. It can be shown that the unique Nash equilibrium is the same as the minimax solution. 2. The concept of Nash equilibrium seems much more natural than the concept of a minimax solution in general games (for example, it may enable coordination). All the players attempt to maximise their payoff given what the others are doing. 3. It can be shown that a Nash equilibrium always exists in a bimatrix game. 26 / 50

27 Nash Equilibria - Disadvantages 1. As shown above, there may be multiple equilibria. In the symmetric game considered above we may want the equilibrium to be symmetric. The only symmetric Nash equilibrium is the mixed equilibrium, but this gives an inefficient payoff vector (i.e. the sum of the payoffs is lower than at the two asymmetric, pure equilibria). Since there is no difference between the players, how can one of the Nash equilibria be attained? 2. In order to choose an equilibrium appropriately, information is needed on the payoffs of the other player. 3. Finding all the Nash equilibria in a complex game is a more difficult problem than finding the minimax strategies. 27 / 50

28 Refinements of Nash equilibria - Pareto Efficiency (Optimality) Although the concept of Nash equilibrium is intuitively more pleasing than the concept of a minimax solution, due to the problems outlined above many refinements of the concept of Nash equilibrium have been made. For example, if two Nash equilibria exist and neither Player obtains less and at least one obtains more at Equilibrium 1 than at Equilibrium 2, then Equilibrium 1 Pareto dominates Equilibrium 2. An equilibrium is Pareto optimal among the set of Nash equilibria if no other Nash equilibrium Pareto dominates it. It seems reasonable that the Players choose a Nash equilibrium with a value that is Pareto optimal among the set of Nash values. 28 / 50

29 Refinements of Nash equilibria - Pareto Efficiency (Optimality) However, it is possible that there is no unique Nash equilibrium which has a Pareto optimal value in the set of Nash values. For example, in the Hawk-Dove game all 3 Nash equilibria are Pareto optimal among the set of Nash values. Nash Equilibrium Nash Value (H, D) (2, 0) (D, H) (0, 2) (0.25H D, 0.25H D) (0.75, 0.75) Note: In the section on correlated equilibria, we will consider solutions that are Pareto optimal within the set of all attainable payoffs. 29 / 50

30 Refinements of Nash equilibria - Pareto Optimality Consider the coordination game ( U D ) U (r1, r 2 ) (0, 0), D (0, 0) (r 3, r 4 ) where all the r i are greater than 0. The pure Nash equilibria for this game are (U, U), (D, D). 30 / 50

31 Refinements of Nash equilibria - Pareto Optimality There is one mixed equilibrium at which Player 1 plays U with probability and Player 2 plays U with probability r 2 r 2 +r 4 r 1 r 1 +r 3. It is relatively easy to show that both the pure Nash equilibria dominate the mixed equilibrium in the Pareto sense. Suppose r 1 > r 3 and r 2 > r 4. In this case, both players prefer (U, U) to (D, D). (U, U) is the Pareto dominant equilibrium. When both players know the payoff function of the other (it suffices to know the opponent s ranking of the possible outcomes), they know that (U, U) is preferred by both players and hence should play U. 31 / 50

32 Refinements of Nash equilibria - Pareto Optimality Even if (U, U) is Pareto dominant, this argument is not valid when a player does not know the preferences of the other player. However, (U, U) may be achieved if both attempt to obtain their maximum possible gain. Things are more complex when e.g. r 1 > r 3 and r 2 < r 4. In this case, Player 1 plays (U, U) and Player 2 prefers (D, D). There is no Pareto dominant equilibrium. The outcome may well depend on the interaction between the two players (one may be dominant and force the other to play his/her preferred equilibrium). 32 / 50

33 Strong and Weak Nash equilibria A Nash equilibrium (a, b ) is called strong when both players obtain a strictly lower payoff when they unilaterally change their strategy. If either player can unilaterally change their strategy and obtain the same payoff, then such a Nash equilibrium is called a weak Nash equilibrium. By definition an essentially mixed Nash equilibrium is a weak Nash equilibrium. Suppose (π1, π 2 ) is a mixed equilibrium and Player 1 plays π1. Player 2 can obtain the Nash value of the game by choosing any action (pure strategy) from the support of π2. If a weak and a strong equilibrium give the same Nash value, then it is assumed that the strong equilibrium is chosen. 33 / 50

34 Definition of Dominated Strategies The pure strategy a dominates the pure strategy b for Player 1, iff R 1 (a, c) R 1 (b, c), for all c S 2 and this inequality is strict for at least one c S 2. This inequality states that whatever Player 2 does, Player 1 does at least as well by playing a as by playing b. In such a case Player 1 should never play b and when looking for Nash equilibria (or minimax solutions), we can eliminate the row corresponding to b. 34 / 50

35 Definition of Dominated Strategies Similarly, the pure strategy a dominates the pure strategy b for Player 2, iff R 2 (c, a) R 2 (c, b), for all c S 1 and this inequality is strict for at least one c S 1. The mixed strategy m dominates the pure strategy b for Player 1, iff R 1 (m, c) R 1 (b, c), for all c S 2 and this inequality is strict for at least one c S / 50

36 Reduction of a Matrix Game If a pure strategy a of Player 1 is dominated by a pure or mixed strategy, we can eliminate the row corresponding to a in the payoff matrix. If a pure strategy b of Player 2 is dominated by a pure or mixed strategy, we can eliminate the column corresponding to b in the payoff matrix. When looking for a minimax solution or Nash equilibrium, dominated strategies can be successively eliminated, until no dominated strategies remain. 36 / 50

37 Reduction of a Matrix Game If at a given stage a pure strategy of a player is a strictly best response to any pure strategy of the other player, it cannot be eliminated at that stage. Weak Nash equilibria may be eliminated using this procedure, but no strong equilibrium will be eliminated (see example on the matrix form of an asymmetric Hawk-Dove game in the next section). 37 / 50

38 Example: Reduction of Matrix Games Show that the following bimatrix game can be reduced to a game in which both players have 2 possible actions and hence derive the minimax solution and Nash equilibria of this game. D E F G A (3,2) (5,0) (2,5) (4,1) B (2,2) (4,4) (1,1) (3,3) C (1,3) (3,3) (4,2) (2,5) 38 / 50

39 Example: Reduction of Matrix Games First note that B is dominated by A. Hence, we can eliminate the second row of the payoff matrix. We obtain D E F G A (3,2) (5,0) (2,5) (4,1) C (1,3) (3,3) (4,2) (2,5) 39 / 50

40 Example: Reduction of Matrix Games After eliminating B, E is dominated by D (and G). Hence, we eliminate the second column of the payoff matrix. We obtain D F G A (3,2) (2,5) (4,1) C (1,3) (4,2) (2,5) At this stage A is the best response to D and C is the best response to F. Similarly, F is the best response to A and G is the best response to C. 40 / 50

41 Example: Reduction of Matrix Games The only strategy that might be dominated at this stage is D. It is simple to check that it is not dominated by either of the pure strategies F or G. We attempt to find a mixed strategy M = pf + (1 p)g that dominates D. In order to this, we must consider the expected payoffs of such a strategy against A and against C and compare them to the payoff that D obtains. R 2 (A, M) = 5p + (1 p); R 2 (A, D) = 2 R 2 (C, M) = 2p + 5(1 p); R 2 (C, D) = 3 41 / 50

42 Example: Reduction of Matrix Games It follows that M dominates A iff the following two inequalities hold: 4p + 1 2; 5 3p 3. These inequalities are satisfied for 1 4 p 2 3. It follows that M dominates D for 1 4 p 2 3 eliminate the first column. and hence we can 42 / 50

43 Example: Reduction of Matrix Games The reduced game is as follows: F G A (2,5) (4,1) C (4,2) (2,5) Now, A is the best response to G, C is the best response to F, F is the best response to A and G is the best response to C. It follows that the game cannot be further reduced. 43 / 50

44 Example: The Minimax Solution Suppose Player 1 plays the mixed strategy M 1 = pa + (1 p)c and Player 2 plays the mixed strategy M 2 = qf + (1 q)g. The expected payoff of Player 1 under these strategies is R 1 (M 1, M 2 )=2pq + 4p(1 q) + 4(1 p)q + 2(1 p)(1 q) =2 + 2p + 2q 4pq 44 / 50

45 Example: The Minimax solution To investigate how Player 2 can minimise Player 1 s payoff, we differentiate with respect to q. If p < 1 2, then R 1 q R 1 q = 2 4p. > 0. Player 2 should choose q = 0. Similarly, when p > 1 2, then Player 1 should choose q = 1. When p = 1 2, the payoff of Player 1 is unaffected by the strategy of Player / 50

46 Example: The Minimax solution We now consider the expected payoff of Player 1 given that Player 2 minimises his/her payoff. When p = 1 2, the payoff of Player 1 is 3. When p < 1 2, by assumption Player 2 chooses q = 0. The payoff of Player 1 is 2 + 2p < 3. When p > 1 2, by assumption Player 2 chooses q = 1. The payoff of Player 1 is 4 2p < 3. It follows that Player 1 can ensure a payoff of 3 by playing A with probability / 50

47 Example: The Minimax solution The expected payoff of Player 2 under these strategies is R 2 (M 1, M 2 )=5pq + p(1 q) + 2(1 p)q + 5(1 p)(1 q) =5 4p 3q + 7pq To investigate how Player 1 can minimise Player 2 s payoff, we differentiate with respect to p. R 1 p = 7q 4. If q < 4 7, then R 1 p < 0. Player 1 should choose p = 1. Similarly, when q > 4 7, then Player 1 should choose p = 0. When q = 4 7, the payoff of Player 2 is unaffected by the strategy of Player / 50

48 Example: The Minimax solution We now consider the expected payoff of Player 2 given that Player 1 minimises his/her payoff. When q = 4 7, the payoff of Player 2 is When q < 4 7, by assumption Player 2 chooses p = 1. The payoff of Player 1 is 1 + 4q < When q > 4 7, by assumption Player 2 chooses p = 0. The payoff of Player 1 is 5 3q < It follows that Player 2 can ensure a payoff of 23 7 probability 4 7. by playing F with 48 / 50

49 Example: Nash Equilibria It is relatively easy to check that there are no pure Nash equilibria. It follows that there must be a mixed equilibrium. To find Player 2 s equilibrium strategy qf + (1 q)g, we use the fact that when he/she plays this strategy, Player 1 is indifferent between his/her two actions. Hence, 2q + 4(1 q) = 4q + 2(1 q) q = 1 2. The equilibrium payoff of Player 1 can be calculated from substituting q = 1 2 into either side of the original equation. The equilibrium payoff of Player 1 is / 50

50 Example: Nash Equilibria To find Player 1 s equilibrium strategy pa + (1 p)c, we use the fact that when he/she plays this strategy, Player 2 is indifferent between his two actions. Hence, 5p + 2(1 p) = p + 5(1 p) p = 3 7. The equilibrium payoff of Player 2 can be calculated from substituting p = 3 7 into either side of the original equation. The equilibrium payoff of Player 2 is / 50