Computational Game Theory and Clustering

Computational Game Theory and Clustering Martin Hoefer mhoefer@mpi-inf.mpg.de

1 Computational Game Theory? 2 Complexity and Computation of Equilibrium 3 Bounding Inefficiencies 4 Conclusion

Computational Game Theory (Micro-)Economics Operations Research Computational Game Theory Applied Mathematics Computer Science

Nash equilibrium S G S G 5 0 4 0 1 4 1 5 n players, strategy space S i for each i [n] State space S = S 1... S n, utility u i : S R s S is pure Nash equilibrium (PNE) if for every player i u i(s) max j S i u i(j, s i).

Nash equilibrium S G S G 5 0 4 0 1 4 1 5 (S,S) and (G,G) are both PNE. n players, strategy space S i for each i [n] State space S = S 1... S n, utility u i : S R s S is pure Nash equilibrium (PNE) if for every player i u i(s) max j S i u i(j, s i).

Nash equilibrium H T H T 50-50 -50 50-50 50 50-50 No PNE. n players, strategy space S i for each i [n] State space S = S 1... S n, utility u i : S R s S is pure Nash equilibrium (PNE) if for every player i u i(s) max j S i u i(j, s i).

Main Questions Existence: Does a given game have an equilibrium? Recognition: Is a given state an equilibrium? Classic Decision Problems

Main Questions Existence: Does a given game have an equilibrium? Recognition: Is a given state an equilibrium? Classic Decision Problems Computation: How can we compute an equilibrium in polynomial time? Dynamics: How can players reach/learn equilibria (quickly)? Fixed-Point Computation, Local Search, etc.

Main Questions Existence: Does a given game have an equilibrium? Recognition: Is a given state an equilibrium? Classic Decision Problems Computation: How can we compute an equilibrium in polynomial time? Dynamics: How can players reach/learn equilibria (quickly)? Fixed-Point Computation, Local Search, etc. Inefficiency: How good are equilibrium states in terms of social and system objectives? Price of... -Results, Approximation

Main Questions Existence: Does a given game have an equilibrium? Recognition: Is a given state an equilibrium? Classic Decision Problems Computation: How can we compute an equilibrium in polynomial time? Dynamics: How can players reach/learn equilibria (quickly)? Fixed-Point Computation, Local Search, etc. Inefficiency: How good are equilibrium states in terms of social and system objectives? Price of... -Results, Approximation Mechanism Design: Design rules of a game to obtain (only) desirable equilibria Auctions, Social Choice, Voting, etc.

Equilibrium Existence and Computation Concepts in Games: Pure Nash equilibrium Mixed Nash equilibrium Correlated equilibrium Coarse-correlated equilibrium Perfect equilibrium Pairwise/(k-)Strong equilibrium Core Shapley Value Nucleolus Market equilibrium Evolutionary Stable Strategies etc. Methods of Computation: Centralized Algorithms Oracle Models Tâtonnement Best-Response Dynamics Sequential Improvements Concurrent Improvements No-Regret Learning Logit-Response Dynamics etc.

Correlation Clustering Games n players, w ij = w ji {0, 1,..., W } Strategies are clusters, S i = [n] i Utility for i of partition s: u i(s) = w ij + (W w ij). j:s i =s j j:s i s j Maximize weight within clusters, minimize weight between clusters.

Correlation Clustering Games 2 1 1 n players, w ij = w ji {0, 1,..., W } Strategies are clusters, S i = [n] i Utility for i of partition s: u i(s) = w ij + (W w ij). j:s i =s j j:s i s j Maximize weight within clusters, minimize weight between clusters. 2 4

Correlation Clustering Games 2 1 They are potential games: u i(s i, s i) u i(s) = Φ(s i, s i) Φ(s) 1 with potential function Φ(s) = 1 u i(s). 2 i [n] Every profitable unilateral deviation of any player increases Φ. Every local optimum of Φ is PNE. [Monderer, Shapley 1996] 2 4

Correlation Clustering Games 2 2 They are potential games: u i(s i, s i) u i(s) = Φ(s i, s i) Φ(s) 2 with potential function Φ(s) = 1 u i(s). 2 i [n] Every profitable unilateral deviation of any player increases Φ. Every local optimum of Φ is PNE. [Monderer, Shapley 1996] 3 3

In a Nutshell: Complexity and Equilibria We represent every number in at most log W bits. Compact Representation: Correlation Clustering Game: O(n 2 log W ) In contrast, normal-form representation has size O(n n n log(nw )). Polynomial vs. Exponential All games studied here have a natural compact representation. It is exponentially smaller than the standard normal-form representation. We favor quick convergence and efficient algorithms. They should terminate in a polynomial number of steps in the size of the natural compact representation.

PLS What is the complexity of computing a pure Nash equilibrium? Non-Constructive Existence Argument: Potential Function Idea: Single players iteratively deviate until local maximum of Φ is reached. Potential function bounded by 0 Φ(s) < n 2 W Convergence in a number of steps polynomial in n, log W?

PLS What is the complexity of computing a pure Nash equilibrium? Non-Constructive Existence Argument: Potential Function Idea: Single players iteratively deviate until local maximum of Φ is reached. Potential function bounded by 0 Φ(s) < n 2 W Convergence in a number of steps polynomial in n, log W? Yes, if W = n c for some constant c.

PLS What is the complexity of computing a pure Nash equilibrium? Non-Constructive Existence Argument: Potential Function Idea: Single players iteratively deviate until local maximum of Φ is reached. Potential function bounded by 0 Φ(s) < n 2 W Convergence in a number of steps polynomial in n, log W? Yes, if W = n c for some constant c. For large W, however, there are games where convergence must take time c n for some constant c.

PLS What is the complexity of computing a pure Nash equilibrium? Non-Constructive Existence Argument: Potential Function Idea: Single players iteratively deviate until local maximum of Φ is reached. Potential function bounded by 0 Φ(s) < n 2 W Convergence in a number of steps polynomial in n, log W? Yes, if W = n c for some constant c. For large W, however, there are games where convergence must take time c n for some constant c. Can we instead compute PNE efficiently with a centralized algorithm? Complexity Class: PLS (polynomial local search) [Johnson, Papadimitriou, Yannakakis 1988] Completeness Idea: Define PLS-complete problem, construct polynomial reductions

PLS For a local search problem in PLS there are: Objective function and neighborhood relation for solutions Three polynomial-time algorithms: Alg A computes a starting solution Alg B computes the objective function value for each solution Alg C checks if a solution is a local optimum. If not, it computes a strictly better neighbor solution.

PLS For a local search problem in PLS there are: Objective function and neighborhood relation for solutions Three polynomial-time algorithms: Alg A computes a starting solution Alg B computes the objective function value for each solution Alg C checks if a solution is a local optimum. If not, it computes a strictly better neighbor solution. Many classic local search problems are PLS-complete: Circuit-Flip with FLIP Neighborhood MaxCut with FLIP neighborhood TSP with k-opt or Lin-Kernighan neighborhood Theorem Computing a PNE in correlation clustering games is PLS-complete. [Feldman, Levin-Eytan, Naor 2012]

Modularity Clustering Games Graph G = (V, E), n vertices, m edges Vertices are players, d i degree of i V Strategies are clusters, S i = [n] i Edge weight w ij = 2m (1 ij) d id j Utility for i of partition s: u i(s) = w ij. j:s i =s j Proposition Every improvement sequence reaches a PNE after O(m 2 ) steps. [Hoefer 2007]

Modularity Clustering Games 6-6 3 6-4 6-4 Graph G = (V, E), n vertices, m edges Vertices are players, d i degree of i V Strategies are clusters, S i = [n] i Edge weight w ij = 2m (1 ij) d id j Utility for i of partition s: u i(s) = w ij. j:s i =s j -6 8 6 Proposition Every improvement sequence reaches a PNE after O(m 2 ) steps. [Hoefer 2007]

Modularity Clustering Games 6 Graph G = (V, E), n vertices, m edges Vertices are players, d i degree of i V Strategies are clusters, S i = [n] i -6 8 6 Edge weight w ij = 2m (1 ij) d id j Utility for i of partition s: u i(s) = w ij. j:s i =s j Proposition Every improvement sequence reaches a PNE after O(m 2 ) steps. [Hoefer 2007]

Symmetric Additive-Separable Hedonic Games -2 n players, weights w ij = w ji Z 4-3 6 2-5 5 9 1 Strategies are clusters, S i = [n] i Utility for i of partition s: u i(s) = w ij. j:s i =s j Theorem Computing a PNE in symmetric additiveseparable hedonic games is PLS-complete. [Gairing, Savani 2010] 8

Symmetric Additive-Separable Hedonic Games -2 n players, weights w ij = w ji Z Strategies are clusters, S i = [n] i Utility for i of partition s: u i(s) = w ij. j:s i =s j 6 5 Theorem Computing a PNE in symmetric additiveseparable hedonic games is PLS-complete. [Gairing, Savani 2010] 8

Equilibrium Existence and Computation Concepts in Games: Pure Nash equilibrium Mixed Nash equilibrium Correlated equilibrium Coarse-correlated equilibrium Perfect equilibrium Strong equilibrium Core Shapley Value Nucleolus Market equilibrium Evolutionary Stable Strategies etc. Methods of Computation: Centralized Algorithms Oracle Models Tâtonnement Best-Response Dynamics Sequential Improvements Concurrent Improvements No-Regret Learning Logit-Response Dynamics etc.

Cooperative Games 4-3 2-2 -5 9 1 n players, weights w ij = w ji Z Partition C = (C 1,..., C k ) of [n] C(i) denotes coalition of i. Utility for i coalition C(i) is u i(c(i)) = w ij. j C(i) 6 8 5 C is in the core iff there is no C [n] such that for all i C u i(c ) > u i(c(i)).

Cooperative Games -2 n players, weights w ij = w ji Z Partition C = (C 1,..., C k ) of [n] C(i) denotes coalition of i. Utility for i coalition C(i) is u i(c(i)) = w ij. j C(i) 6 8 5 C is in the core iff there is no C [n] such that for all i C u i(c ) > u i(c(i)).

Cooperative Games n players, weights w ij = w ji Z -5 9 1 Partition C = (C 1,..., C k ) of [n] C(i) denotes coalition of i. Utility for i coalition C(i) is u i(c(i)) = w ij. j C(i) 6 8 5 C is in the core iff there is no C [n] such that for all i C u i(c ) > u i(c(i)).

Core? 6 5 4 5 6 4 4 6 5 Deciding non-emptiness of the core is hard: NP-hard in symmetric additive-separable hedonic games. [Aziz, Brandt, Seedig 2010] Σ p 2-complete in additive-separable hedonic games. [Woeginger 2013]

Core in Additive-Separable Games [Dimitrov, Borm, Hendrickx, Sung 2006] w ij {n, 1}: Consider digraph G with E = {(i, j) w ij = n}. Partition G into strongly connected components Core partition. Computation in P, recognition is an open problem.

Core in Additive-Separable Games [Dimitrov, Borm, Hendrickx, Sung 2006] w ij {n, 1}: Consider digraph G with E = {(i, j) w ij = n}. Partition G into strongly connected components Core partition. Computation in P, recognition is an open problem. w ij {1, n}: Consider graph G with E = {{i, j} w ij = w ji = 1}. Select a largest clique C from G as a separate cluster. Remove C from G and repeat Core partition. Special case of games with weak top-coalition property. [Banerjee, Konishi, Sönmez 2001] Computation and recognition are both strongly NP-complete.

Fractional Hedonic Games 4-3 -5 3-3 3 3 9 9 1 n players, weights w ij Z, w ij = w ji Utility of partition C for i is given by u i(c(i)) = w ij/ C(i). Theorem Deciding existence of PNE j C(i) non-emptiness of the core is NP-hard in symmetric fractional hedonic games. [Brandl, Brandt, Strobel 2015]

Fractional Hedonic Games -1.5-3 -1.5 n players, weights w ij Z, w ij = w ji Utility of partition C for i is given by u i(c(i)) = w ij/ C(i). j C(i) 4 3 9 3 2 4 Theorem Deciding existence of PNE non-emptiness of the core is NP-hard in symmetric fractional hedonic games. [Brandl, Brandt, Strobel 2015]

Fractional Hedonic Games 0 3-3 9 2 n players, weights w ij Z, w ij = w ji Utility of partition C for i is given by u i(c(i)) = w ij/ C(i). j C(i) 4.5 9 4 4.5 Theorem Deciding existence of PNE non-emptiness of the core is NP-hard in symmetric fractional hedonic games. [Brandl, Brandt, Strobel 2015]

Fractional Hedonic Games on Graphs If w ij = w ji {0, 1}, then w describes an unweighted, undirected graph. Max-Degree 2 Forests Bipartite with Perfect Matching Girth 5 Non-Empty Core Computation in P? [Aziz, Brandt, Harrenstein 2014]

Equilibrium Existence and Computation Concepts in Games: Pure Nash equilibrium Mixed Nash equilibrium Correlated equilibrium Coarse-correlated equilibrium Perfect equilibrium Strong equilibrium Core Shapley Value Nucleolus Market equilibrium Evolutionary Stable Strategies etc. Methods of Computation: Centralized Algorithms Oracle Models Tâtonnement Best-Response Dynamics Sequential Improvements Concurrent Improvements No-Regret Learning Logit-Response Dynamics etc.

Multi-Armed Bandits Learning to play optimally: m slot machines, T rounds In round t, first pick some machine i t Then learn reward u t (i t ) [0, 1] Reward chosen by adversary Goal: Minimize regret R(T ) = ( max j [m] t=1 No-Regret Algorithm picks i t such that ) T T u t (j) u t (i t ) lim R(T )/T = 0 T t=1

Full Information Assume after each round we learn u t (j) for all j [m]. Let η = (ln m)/t. Randomized Weighted Majority (RWM) Algorithm Initially, set w 1 j = 1, for every j [m]. At every time t, let W t = m j=1 wt j; choose machine j with probability p t j = w t j/w t ; set w t+1 j = w t j (1 η) 1 ut (j). The algorithm obtains R(T )/T = ε after T = Ω( ln m /ε 2 ) rounds. [Littestone, Warmuth 1994] Similar algorithms exist for incomplete information when we only learn u t (i t). They carefully trade exploration and exploitation.

Playing Games with Learning Algorithms Suppose a strategic game is played repeatedly for T rounds. Every player uses a no-regret learning algorithm to make his strategy choice in each round. Consider the sequence of states s 1, s 2,..., s T, then every player i obtains T t=1 u i(s t ) = max j S i T u i(j, s t i) R(T ) t=1

Playing Games with Learning Algorithms Suppose a strategic game is played repeatedly for T rounds. Every player uses a no-regret learning algorithm to make his strategy choice in each round. Consider the sequence of states s 1, s 2,..., s T, then every player i obtains 1 T T t=1 u i(s t 1 ) = max j S i T T t=1 u i(j, s t i) R(T ) T

Playing Games with Learning Algorithms Suppose a strategic game is played repeatedly for T rounds. Every player uses a no-regret learning algorithm to make his strategy choice in each round. Consider the sequence of states s 1, s 2,..., s T, then every player i obtains 1 T T t=1 u i(s t 1 ) = max j S i T T t=1 u i(j, s t i) R(T ) T Consider T and let D be the distribution of play resulting from averaging over s 1, s 2,... Let x D, then E [u i(x)] max j S i E [u i(j, x i)] Such a distribution D is called a coarse-correlated equilibrium.

Equilibrium Concepts PNE Mixed NE Correlated No-Regret

Price of Anarchy In many applications we are interested in the quality of equilibria. To measure quality of a state s, we use a social welfare function sw(s). A prominent choice is utilitarian social welfare with sw(s) = i ui(s). To quantify the deterioration in equilibrium we compare the cost to that of an optimum state s. The price of anarchy for an equilibrium concept in a single game is determined by the worst equilibrium. The price of anarchy for a class G of games is the worst ratio in any game: PoA = max I G sw(s, I) sw(s, I) max s is equil.

Price of Anarchy for Equilibrium Concepts Since the sets of equilibria are expanding, the worst-case ratios for the PoA are increasing for different equilibrium concepts: 1 Optimum PNE Mixed NE Correlated No-Regret

Robust Price of Anarchy A game is (λ, µ)-semi smooth if there exists a randomized strategy x i for each player i such that for every outcome s: E [u i(x i, s i)] λ sw(s ) µ sw(s) i

Robust Price of Anarchy A game is (λ, µ)-semi smooth if there exists a randomized strategy x i for each player i such that for every outcome s: E [u i(x i, s i)] λ sw(s ) µ sw(s) i Consider any PNE s of the game, then sw(s) = i u i(s) PNE semi smooth max u i(j, s i) j S i i E [u i(x i, s i)] i λ sw(s ) µ sw(s)

Robust Price of Anarchy A game is (λ, µ)-semi smooth if there exists a randomized strategy x i for each player i such that for every outcome s: E [u i(x i, s i)] λ sw(s ) µ sw(s) i Consider any PNE s of the game, then sw(s) = i u i(s) PNE semi smooth max u i(j, s i) j S i i E [u i(x i, s i)] i λ sw(s ) µ sw(s) Hence (1 + µ) sw(s) λ sw(s ) = sw(s ) sw(s) 1 + µ λ A similar calculation shows the same bound for coarse-correlated equilibria. [Caragiannis, Kaklamansis, Kanellopoulos, Kyropoulou, Lucier, Paes Leme, Tardos 2015].

Price of Anarchy MaxCut Game n players, weights w ij = w ji 0. Strategies are clusters, S i = {1, 2} i Utility of partition s for i is given by u i(s) = j:s i s j w ij.

Price of Anarchy MaxCut Game n players, weights w ij = w ji 0. Strategies are clusters, S i = {1, 2} i Utility of partition s for i is given by u i(s) = j:s i s j w ij. Let x i = (1/2, 1/2) for every i. For every s u i(1, s i) + u i(2, s i) = j w ij = w i. Thus, E [u i(x i, s i)] = i i w i 2 sw(s ), 2 i.e. λ = 1/2, µ = 0, and the robust price of anarchy is 2.

Conclusion Computational Game Theory Deciding Existence and Recognition of Equilibria Computing and Converging to Equilibria Inefficiency and Price-of-Anarchy Mechanism Design, Social Choice This talk: Strategic Games, PNE, Potential Functions, PLS Cooperative Games, Core, Hardness No-Regret Learning, Coarse-Correlated Equilibrium Robust Price of Anarchy