Nash Equilibria and Related Observations in One-Stage Poker Zach Puller MMSS Thesis Advisor: Todd Sarver Northwestern University June 4, 2013
Contents 1 Abstract 2 2 Acknowledgements 3 3 Literature Review 4 4 Model 6 5 Lemmas 7 6 Examples 10 7 Results 13 8 Works Cited 15 1
1 Abstract Poker is a classic example of a dynamic game with incomplete information. However, depending on how the most simplified forms of the game our modeled, analysis can be conducted in other directions. This paper looks at single betting round poker variants, typically consisting of two stages (ie player 1 bets or folds, player 2 calls or folds), and models them as static games where each player chooses a strategy over all possible hands simultaneously, followed by the random distributing of hands and subsequent execution of respective strategies. The component of bluffing is observed when players choose strategies that bet in weak hands, in particular when these strategies are part of mixed strategies that bet on such hands only a fraction of the time, so as to limit the information given by a particular action. While player 2 in such games is thought of as receiving information from player 1s move, the information associated with betting can be incorporated into each possible hand prior to any move, in particular because player 2 only makes a call/fold decision if player 1 has already bet (if player 1 folds, the game ends). Thus the static model sacrifices no aspect of the simplified game. The simplified game discussed builds off of the discrete, 3-card Borel model in which both players are dealt from a single deck consisting of 3 cards, numbered from 1 to 3. Player 1 chooses to bet or fold first. If he folds, the game ends, and if he bets, player 2 chooses to call or fold. The game is discussed in greater detail below. This paper aims to focus on variations of this model in particular, looking at versions that play out the same, with the exception that the deck contains any amount of cards greater than 3. In particular, we wish to see how the dominant strategy develops as the number of cards increases, if certain rules of thumb characterize all strategies, and if the strategy converges to that of the continuous model as the number of cards approaches infinity. 2
2 Acknowledgements I would like to take this opportunity to thank my advisor, Professor Todd Sarver for his rigorous guidance, heavy time commitment, and encouragement throughout the past year. His support enabled me to overcome the various challenges which popped up throughout the process. I would also like to thank Professor Joseph P. Ferrie for his direction and understanding throughout the writing of the thesis, and finally Professor William Rogerson and Sarah M. Ferrer for their tremendous guidance and assistance throughout my years at Northwestern University. 3
3 Literature Review Considerable amounts of research have been done on simplified versions of poker. The draw to this area is two-fold. One factor leading researchers in this direction is the rising popularity of game theory in general since the 40s and 50s, along with the rising popularity of poker itself in more recent years. The other is the economic relevance that can be tied to game theoretic analysis of poker. While common variants of poker such as Texas Holdem are considerably too complex for finding dominant strategies (at this point), simplifying the game removes that obstacle and allows us to investigate what many consider to be the central element of poker, bluffing. Specifically, we can come up with a more mathematical grasp on when and why an opponent would bluff, or, conversely, why we, as a player, would want to bluff more often in certain situations than in others. Certainly this line of thinking can be extended into various business and economic scenarios in which competing firms pursue different actions which give away information to consumers and competitors. The motives of each of such moves, with respect to their immediate consequences as well as the implications of any information revealed in such moves, is of great interest to economists. The Borel Poker model was one of the first simplified two-person poker models to be studies with rigorous game theoretic modeling. Borel presents a continuous version of his simplified game. In the such a case, first both players give an ante of 1 to the pot. Player 1 is dealt a random hand denoted by x, where 0 < x < 1, and x comes from a uniform probability distribution. Player 2 is similarly dealt a random hand Y from the same distribution. Player 1 acts first in the game, deciding whether or not to bet. If he does not bet, player 2 wins both antes. If he does bet (a value denoted by B), then player 2 chooses to call or fold. If he folds, player 1 wins both antes. If he calls, the hands are compared and the winning player (higher numbers win over lower ones) wins B + 1 from the loser. One shortcoming of this model is that if player 1 does not bet, the game ends, thus trivializing a component of the decision space for player 2. Von Neumann corrects for this in his own model, by allowing player 1 to check rather than fold, thus putting player 2 in player 1 s shoes (with some information). If player 2 checks, a showdown happens, and if he bets, player 1 must call or fold. For simplicity, we will work with the discrete version of the Borel model, because, as well will see, adding cards to the deck adds new layers of complexity, and it will be of great help to simplify all other components of the game as much as possible in order to analyze the effect of the deck 4
itself. The sorts of questions that come up with respect to deck size are: given a larger deck, does a player bluff more? Less? Does he bet more or less for value? Does he bet with lower cards in a bigger deck (ie no bet on 2/3 but bet on 2/4)? Does the betting behavior begin to evolve in a predictable way as size increases? The analysis of these games will parallel that of Karremans Three card poker with a limited deck. Karremans shows that in this game, there is a mixed strategy Nash equilibrium in which player 1 (player A) bets with the highest two cards, and bets with the lowest card with probability 1/3. Likewise, player 2 (player B) calls with the highest card, folds with the lowest card, and calls with the middle card with probability 1/3. In variants of the game with 4, 5, 6, etc. we would expect to see similar equilibria in which player 1 sometimes bets with losing cards, and player 2 sometimes calls with mid-range cards, however there is a non-trivial chance that the math itself with prove deceptive and nullify this hypothesis. A point of interest of the infinite Von Neumann model lies in the equilibrium solution. Both players are dealt a value from the uniform 0 1 distribution, and, depending on the value of the ante relative to the size of the bet, values 0 < a < b < 1 and 0 < c < 1 exist such that player 1 should bet with card x unless a < x < b and player 2 should call with card y such that y > c. This is significant because optimal play is not characterized by a mixed strategy, whereas in the limited deck games, mixed strategy is present and significant in that it captures one important facet of bluffing, that a player could bet sometimes with low hands and fold other times, thus keeping more information private from the other player. The discrepancy of equilibrium style brings up the question of whether the mixed strategy component of optimal play will become a smaller and smaller part of the decision space as more and more cards are added, eventually becoming trivial. 5
4 Model In actual play of this sort of game, the dealer would deal both cards before player 1 chooses whether or not to bet. However, the order of this move is trivial on a theoretical level because player 1 gains no information from player 2 knowing what his card is. More importantly, sequencing the moves to have the dealer wait to deal the second card makes the analysis visually simpler, so we will keep that assumption. This is an example of a dynamic game since players take actions in sequence, with information being revealed along the way. However, since each player only takes one move (ie there is only one stage of betting), it is easy to model the game as a static game, and extract the same results. In the static version of the game, each player chooses whether he would bet (call) or fold for every possible card he could be dealt. Further, player 2 is deciding whether to call or fold only given a bet from player 1, and does not need a strategy to capture the situations where player 1 folds, since the game ends and there is no decision to be made at that point. Thus, we simply have a game where both players choose their strategies at the same time (the information is of course kept private), and then the remainder of the game (cards being dealt, subsequent betting) happens immediately, with the cards being dealt at random (ie the dealer does not need to be treated as a player with payoffs). Thus, the payoffs of any pair of strategies can be equated to the expected value to each player given an equal probability of being dealt any card in the deck. 6
5 Lemmas Lemma 5.1. If player 1 bets with card x given some strategy s of player 2, than he will bet with x + 1. Proof. If P1 has card x, he will bet if, for some strategy s of P2 (with card y), U bet,s,x = p(y < x)π s,x (y < x) + p(y > x + 1)π s,x (y > x + 1) + p(y = x + 1)π s,x (y = x + 1) > 1 Assuming this inequality holds, let s now look at P1 s betting utility if he has card x + 1 U bet,s,x+1 = p(y < x)π s,x+1 (y < x) + p(y > x + 1)π s,x+1 (y > x + 1) + p(y = x)π s,x+1 (y = x) This value will be greater than U bet,s,x if p(y = x)π s,x+1 (y = x) > p(y = x + 1)π s,x (y = x + 1) p(y = x)π s,x+1 (y = x) means player 1 has the higher card, so it equals either 2 or 1. p(y = x + 1)π s,x (y = x + 1) means player 1 has the lower card, so it equals either 1 or -2. so, p(y = x)π s,x+1 (y = x) p(y = x + 1)π s,x (y = x + 1) So, P1 is weakly better off betting with card x + 1 than with card x, meaning if he bets with x for P2 strategy s, he should certainly also bet with x + 1. Lemma 5.2. If player 1 folds with card x given some strategy s of player 2, than he will fold with x 1. Proof. If P1 has card x, he will fold if, for some strategy s of P2 (with card y), U bet,s,x = p(y < x 1)π s,x (y < x 1) + p(y > x)π s,x (y > x) + p(y = x 1)π s,x (y = x 1) < 1 Assuming this inequality holds, let s now look at P1 s betting utility if he has card x 1 U bet,s,x 1 = p(y < x 1)π s,x 1 (y < x 1) + p(y > x)π s,x 1 (y > x) + p(y = x)π s,x 1 (y = x 1) This value will be less than U bet,s,x if p(y = x 1)π s,x, x(y = x 1) > p(y = x)π s,x 1 (y = x) p(y = x 1)π s,x (y = x 1) means player 1 has the higher card, so it equals either 2 or 1. p(y = x)π s,x 1 (y = x) means player 1 has the lower card, so it equals either 1 or -2. so, p(y = x 1)π s,x (y = x 1) p(y = x)π s,x 1 (y = x) So, P1 is weakly worse off betting with card x 1 than with card x, meaning if he folds with x for P2 strategy s, he should certainly also fold with x 1. Theorem 5.3. For any strategy of player 2, and for any two adjacent cards m and m + 1, a strategy that bets with m and folds with m + 1 is weakly dominated by one that bets with m + 1 and folds with m Proof. Suppose player 1 is playing some strategy...bf... in which he bets with m and folds with m + 1. Then, according to lemma 5.1, he is weakly better off playing...bb..., and according to lemma 5.2, he is weakly better off playing...f F..., therefore he must be weakly better off playing...f B... 7
Though these lemmas only show weak dominance, we will restrict our results to equiibria of cutoff strategies for simplicity. We can also show that these types equilibria must always exist. Lemma 5.4. In a version of this game where players can choose only from cutoff strategies, there exists either a pure strategy Nash equilibrium or a mixed strategy Nash equilibrium. Proof. See Nash s Existence Theorem. Theorem 5.5. There must always exist a Nash equilibrium using cutoff strategies in the one-stage poker game. Proof. We know from 5.4 that when only cutoff strategies are allowed, equilibria will exist. Now, if we take the restricted game and allow players to deviate to non-cutoff strategies, we know from 5.4 that neither player will do this because they are already playing best response strategies. Therefore, these strategy pairs are also Nash equilibria in the full version of the game. Lemma 5.6. In an n-card deck, player 1 will always bet with card x if x > (n/3) + (2/3) Proof. We will denote the worst case utility of betting with card x by U(B x ) wc. For example, in the 4-card game, the worst case utility of betting with the 3 is the expected utility if player 2 is playing the strategy FFFB, ie he will call if he has you beat, or fold otherwise. U(B x ) wc = P (y < x)(1) + P (y > x)( 2) = (y 1)/(n 1) 2(n x)/(n 1) U(F ) = 1 Player 1 should bet if U(B x ) wc > U(F ) (x 1)/(n 1) 2(n x)/(n 1) > 1 (x 1) 2(n x) > (n 1) 3x 1 2n > 1 n 3x > n + 2 x > n/3 + 2/3 Lemma 5.7. In an n-card deck, player 1 will always fold with card y if y < (n/4) + (3/4) Proof. We know from 5.1 that P1 bets if x > n/3 + 2/3 We will denote the best case utility of betting with card y by U(B y ) bc. 1. If y > n/3 + 2/3 best case, p1 always bets, so P2 folds if 8
U(B y ) bc = p(x < y)(2) + p(x > y)( 2) = 2(y 1)/(n 1) 2(n y)/(n 1) < 1 2y n 1 < (1 n)/2 y < n/4 + 3/4 So P2 has no card for which he always folds, since y > n/3 + 2/3 is assumed, meaning y < n/4 + 3/4 is never satisfied. 2. If y < n/3 + 2/3 P2 s best case is that p1 folds if y < x < n/3+2/3. But if we re only looking at cutoff strategies, there are two potential best cases. (a) p1 folds if x < n/3 + 2/3 U(B y ) bc = p(x < n/3 + 2/3)(1) + p(x > n/3 + 2/3)( 2) < 1 n/3 + 2/3 2(2n/3 5/3) < 1 15 < 3n n > 5 (b) p1 always bets U(B y ) bc = p(x < y)(2) + p(x > y)( 2) < 1 2(y 1)/(n 1) 2(n y)/(n 1) < 1 2y n 1 < (1 n)/2 y < n/4 + 3/4 so fold if y < n/4 + 3/4 In all subcases, P2 necessarily is folding if his card is less than n/4 + 3/4. 6 Examples Note: Table entries are given as payoff to player 1. Since the games are zero-sum, the payoff to player 2 is the opposite of that to player 1. Thus, player 2 is, given a choice of values in the following tables, trying to minimize. Bold entries denote best responses to player 1, while italicized entries denote best responses to player 2. In tables with one entry, the entry is the pure-strategy 9
Nash equilibrium and payoff to player 1. In tables with more than one entry, there is no purestrategy Nash equilibrium, so there is a mixed-strategy Nash equilibrium using the non-dominated strategies, which are those shown in the tables. 10
Table 1: 3-Card Game FBB FFB BBB -0.3333333 0 FBB 0-0.1666667 Table 2: 4-Card Game FBBB FFBB BBBB -0.25-0.1666667 FBBB 0-0.1666667 Table 3: 5-Card Game FBBBB FFBBB FFFBB BBBBB -0.2-0.2 0 FBBBB 0-0.15-0.1 FFBBB 0.05-0.1-0.2 11
Table 4: 6-Card Game FFBBBB FFFBBB BBBBBB -0.2-0.1 FBBBBB -0.1333333-0.1333333 FFBBBB -0.0666667-0.1666667 Table 5: 7-Card Game FFBBBBB FFFBBBB FFFFBBB BBBBBBB -0.1904762-0.1428571 0 FBBBBBB -0.1190476-0.1428571-0.0714286 FFBBBBB -0.047619-0.1428571-0.1428571 Table 6: 10-Card Game FFFBBBBBBB FFFFBBBBBB FFFFFBBBBB BBBBBBBBBB -0.16666667-0.13333333-0.05555556 FBBBBBBBBB -0.13333333-0.13333333-0.08888889 FFBBBBBBBB -0.1-0.13333333-0.12222222 FFFBBBBBBB -0.06666667-0.13333333-0.15555556 12
7 Results While we have to speculate to an extent the formal nature of the game as we approach the infinite deck, there are some important patterns to be observed in the games we have solved for. The structure to the equilibria to each game is as follows: Both players have a best response strategy to each possible strategy of the other players (these are shown in bold and italics in the tables). So, if any pair of strategies is simultaneously a best response to player 1 s strategy and a best response to player 2 s strategy, then there exists a pure strategy Nash equilibrium in the game. If this does not occur, then some mixed strategy Nash equilibrium exists instead, in which each player players a combination of non-dominated strategies some fraction of the time (we assume the players can mix between strategies with perfect randomness). Going through each game solved, we have: n Table 7: Equilibria Number of Strategies in MSNE 3 2 4 1 5 3 6 3 7 1 10 1 In this table, the number of strategies in the MSNE of a game represents the greater number between player 1 and player 2. For example, if player 1 alternates between 2 strategies and player 2 alternates between 3 strategies, the number will be 3. If the number is 1, we have a PSNE. This table is of great interest to us in answer the question of whether or not the way to play one-stage poker grows increasing complex as you add cards or rather increasingly simple. As it turns out, the number of strategies comprising an equilibrium varies rather unpredictably, and ranges from 1 to 3 (in this small sample of games). If we assume the development continues a similar manner as we keep adding cards, then we can assume this number of strategies remains relatively small and constant. However, the size of the deck itself grows at a constant rate, meaning if we look at the proportion of cards in the deck with variable plays (this is the same as the number of strategies in a MSNE), this number will be proportional to 1/n, meaning as n grows, the relative size of the MSNE converges to zero. 13
A more formal way to think of this is to think of each player playing a set of cutoff strategies where he chooses c to always fold if his card x is less than c ɛ and always bet if x > c + ɛ. If his card falls between these two values, he will bet sometimes and fold sometimes. If we consider the cards in the deck to be n values distributed uniformly between 0 and 1 (ex. n = 3, the three cards are 0,.5, 1), then, as discussed above, ɛ will be proportional to 1/n since the number of adjacent cards included in the cutoff strategies remains relatively constant. This is depicted in the graph below. So, ɛ will approach 0 as n approaches infinity, meaning we will be approaching the cutoff strategies of Borel and Von Neumann exactly as expected. 14
8 Works Cited References [1] Ferguson, Chris, and Thomas Ferguson. On the Borel and Von Neumann Poker Models. UCLA Math (n.d.) n. pag. Web. http://www.math.ucla.edu/ tom/papers/poker1.pdf. [2] Hoehn, Bret, Finnegan Southey, Robert C. Holte, and Valeriy Bulitko. Effective Short-Term Opponent Exploitation in Simpli. University of Alberta, Dept. of Computing Science (n.d.): n. pag. Web. http://poker.cs.ualberta.ca/publications/aaai05.pdf. [3] Karremans, Vincent. The Nash Equilibrium In Simplified Games of Poker. Erasmus University Rotterdam (n.d.): n. pag. Print. [4] Ferguson, Chris, Tom Ferguson, and Cephas Gawargy. UNIFORM(0,1) TWO- PERSON POKER MODELS. N.p., n.d. Web. UCLA Math (n.d.) n. pag. Web. http://www.math.ucla.edu/ tom/papers/poker1.pdf. 15