Pareto Efficiency and Approximate Pareto Efficiency in Routing and Load Balancing Games

Transcription

1 Pareto Efficiency and Approximate Pareto Efficiency in Routing and Load Balancing Games Yonatan Aumann Yair Dombb Abstract We analyze the Pareto efficiency, or inefficiency, of solutions to routing games and load balancing games, focusing on Nash equilibria and greedy solutions to these games. For some settings, we show that the solutions are necessarily Pareto optimal. When this is not the case, we provide a measure to quantify the distance of the solution from Pareto efficiency. Using this measure, we provide upper and lower bounds on the Pareto inefficiency of the different solutions. The settings we consider include load balancing games on identical, uniformly-related, and unrelated machines, both using pure and mixed strategies, and nonatomic routing in general and some specific networks. 1 Introduction Efficiency, and the efficient utilization of resources, is a key interest in economics. Efficiency can be defined in many ways, depending on the situation and goals, but perhaps one of the most rudimentary and basic efficiency notions is that of Pareto Efficiency. Pareto efficiency captures the idea that an outcome is clearly inefficient if it is possible to achieve an improvement on all fronts simultaneously; for example, in game theory an outcome of a game is (weakly) Pareto optimal if there is no other outcome in which all players are (strictly) better off. Unfortunately, it is well known that strategic behavior by players can frequently lead to Pareto inefficient outcomes, such as in the famous Prisoner s Dilemma. Thus, a Nash equilibrium may be Pareto inefficient. In this work, we study the Pareto efficiency, or inefficiency, of two well known games: routing games and load balancing games (also known aob scheduling games). These games have received a lot of attention in the past decade, mainly in the context of the Price of Anarchy and the Price of Stability, measures that quantify the loss in social welfare due to selfishness and inability of players to coordinate. We analyze the these games with respect to the Pareto efficiency of solutions to the games. Specifically, we focus on Nash equilibria solutions and greedy solutions, and analyze their Pareto efficiency. In some cases we can show that the solutions are necessarily Pareto optimal. When this is not the case, we wish to quantify how far the solution is from Pareto efficiency, since it would be different if all players can improve their outcome ten-fold or just by 10%. Thus, we introduce the notion of approximate Pareto efficiency, defined shortly. With this definition in hand, we show that while Pareto optimality is not always guaranteed, the inefficiency in the settings we consider can frequently be bounded by a constant. Department of Computer Science, Bar-Ilan University, Ramat Gan 52900, Israel. aumann@cs.biu.ac.il Department of Computer Science, Bar-Ilan University, Ramat Gan 52900, Israel. yair.biu@gmail.com 1

2 Approximate-Pareto-Efficiency We now present the formal definition for quantifying the distance of an outcome from Pareto efficiency. Conceptually, an outcome is α-pareto-inefficient if there is a different outcome which improves all players by at least an α factor. Formally, let G be a game, and A be some possible outcome of G. We denote by cost(i, A) the cost of player i in the outcome A. 1 Definition 1. Let G be a game with a set P of players. For outcomes A, A of G, we say that A α-pareto-dominates A if it holds that i P : α cost(i, A ) cost(i, A). Definition 2. Let A be an outcome of a game G. We say that A is α-pareto-deficient if there exists an alternative outcome A of G that α-pareto-dominates A. α-pareto-efficient (α-pe) if it is not β-pareto-deficient for any β > α. Thus, in an α-pareto-deficient outcome, all players can simultaneously improve their outcome by a factor of at least α. In an α-pareto-efficient outcome, it is impossible to improve all players simultaneously by more than α. Note that 1-Pareto-efficient coincides with Pareto optimality. This Work. As mentioned, in this work we consider routing and load balancing games, with several flavors of each. For each class of games, we consider the following issues: 1. Bounding the Pareto inefficiency of any Nash equilibrium: We seek the smallest possible α such that every Nash equilibrium in any game of the class is α-pareto-efficient. 2. Bounding the Pareto inefficiency of the best Nash equilibrium: We seek the smallest possible α such the for any game in the class there exists a Nash equilibrium that is α-pareto-efficient. 3. Bounding the Pareto inefficiency of a sequential greedy assignment process: The greedy solution is defined as follows. Assume that the players are (arbitrarily) ordered, and each player, in its turn, chooses a strategy that minimizes her cost at the time of choosing (ties are broken arbitrarily). We seek the smallest α such that every outcome achieved by a greedy solution is α-pareto-efficient. Results. We consider selfish load balancing and selfish routing games. For load balancing games we consider the settings of identical machines, uniformly-related machines, and unrelated machines. In addition, we consider both the case where only pure strategies are permitted and the case that mixed strategies are also allowed. We obtain: Pure strategies only: If only pure strategies are allowed, any Nash equilibrium is necessarily Pareto optimal for both identical and uniformly-related machines. For unrelated machines, the Pareto-deficiency of a Nash equilibrium can be arbitrarily large, but there necessarily exists a Pareto optimal Nash equilibrium. 1 Due to the nature of the routing and load balancing games that we consider, we use a cost formulation of the notions. Analogous definitions can be defined in a value/utility formulation. 2

3 The greedy solution is Pareto optimal for identical machines, while for uniformly-related machines we show that it is necessarily 2-Pareto-efficient. We were unable to show a bound on the Pareto-deficiency of the greedy solution for unrelated machines, but it can be shown that the bound of 2 shown for uniformly-related machines does not hold for this case (i.e. there are cases in which the Pareto-deficiency of the greedy solution is strictly larger than 2). Mixed strategies: if mixed strategies are allowed, then on identical machines any Nash equilibrium is necessarily (2 1 m )-Pareto-efficient, where m is the number of machines. This bound is tight, in the sense that for any m, there exists a setting with m machines that exhibits a Nash equilibrium which is (2 1 m )-Pareto-deficient. For uniformly-related machines with mixed strategies, we show that any Nash equilibrium is necessarily 4-Pareto-efficient. We do not know to say if this bound is tight, and suspect that it is not. For unrelated machines, the worst Nash can be arbitrarily Pareto-deficient. For the best Nash equilibrium in mixed strategies, we show that it is always Pareto optimal when the machines are identical. When the machines are not identical, we have an example in which it is not Pareto optimal; in fact, we show that there are cases in which any profile that Pareto-dominates the best equilibrium must use mixed strategies. However, we do not have upper bounds for the best Nash in uniformly-related and unrelated machines (although, of course, the upper bounds for the worst Nash apply for the best Nash as well). The greedy process is not well defined for such strategies. For selfish routing games we consider the case of nonatomic games with monotone cost functions. We show: For general networks, for any family of cost functions, the Pareto efficiency of any Nash equilibrium is necessarily bounded by the Price of Anarchy for this class of functions. This bound is tight, in the sense that there exists a game for which the only Nash equilibrium exhibits this level of Pareto-deficiency. Hence, the same bound also holds for the best Nash. We do not know to bound the Pareto-deficiency of the greedy solution for such games, although we have an example showing that there are cases in which such solutions are not Pareto optimal. We consider the special case of networks with linearly-independent routes, defined in [Mil06]. In that paper, Milchtaich proves that on such networks, equilibria are always Pareto optimal; we show here that the greedy solution on such networks is always an equilibrium and thus Pareto optimal as well. For the even more restricted case of networks with only parallel edges between a single source and a single sink (which we call parallel-edge networks), we show that if all cost functions are linear then any flow that uses all the edges is Pareto optimal. Our results are summarized in Table Related Work Pareto efficiency is a desirable property for solutions of games. In cooperative games, such as in Nash s famous bargaining game [Nas50], it is usually required that solutions be Pareto optimal. In non-cooperative game theory, it is well known that Nash equilibria are frequently Pareto inefficient, as illustrated by the famous prisoner s dilemma. During the years, several works aimed at developing 3

4 Setting Any Nash Best Nash Greedy Section Routing equals the General Networks POA for the class open 2.1 Routing Parallel-Edge Networks PO [Mil06] PO 2.2 Load Balancing (Pure) Identical Machines PO PO 3.1 Load Balancing (Pure) Uniformly-Related Machines PO 2-PE 3.1 Load Balancing (Pure) Unrelated Machines PO > 2 3 Load Balancing (Mixed) Identical Machines (2 1 m )-PE PO N/A 4 Load Balancing (Mixed) 4-PE Uniformly-Related Machines (not tight) > 1 N/A 4 Load Balancing (Mixed) Unrelated Machines > 1 N/A 3 Table 1: Summary of results (PO stands for Pareto Optimal, and PE for Pareto Efficient) a deeper understanding of this phenomenon. Examples include [Dub86], which gives sufficient conditions for inefficiency of equilibria, [Coh98], which computes the probability of inefficient (pure) Nash equilibria in finite random games, and [HS78, Mas99], which consider the Pareto optimality of different social choice rules. A work even more relevant to our case is that of Milchtaich [Mil06], in which the author gives a topological characterization of routing networks in which equilibrium may be Pareto inefficient with an appropriate choice of cost functions. Pareto efficient solutions are also sought in multi-objective optimization problems. In this case, the Pareto front is defined as the set of solutions from which not all objectives can be improved simultaneously. Several works (e.g. [Lor84, PY00, DY09, LGCM10]) have considered various approximation notions of the Pareto front, by additive or multiplicative terms, and provided algorithms for finding such solution sets. Also related is the line of research on the Price of Anarchy [KP99] and the Price of Stability [ADK + 04]. The Price of Anarchy bounds the distance of any Nash equilibrium from an optimal outcome, defined using a social welfare function. Likewise, the Price of Stability bounds the distance of the best Nash equilibrium from the optimal social welfare. Some of the issues we consider in this work (namely the Pareto inefficiency of any/the best equilibrium) resemble these concepts, although our Paretian efficiency concept is distinct from social welfare efficiency, and cannot be expressed using any real-valued social welfare (or cost) function. It is worth noting that if the utilitarian social welfare function is considered, it can be shown that the Pareto-deficiency of the worst and best equilibria provide lower bounds for the POA and POS (resp.). The same hold for the egalitarian social welfare function, in the case that only pure strategies are allowed. Finally, while our worst/best Nash questions cannot be expressed as special cases of the classical POA/POS, they can be formulated using the IR min measure, presented by Feldman and Tamir in [FT08]. For a Nash equilibrium s and a set Γ of players, IR min (s, Γ) is defined as the 4

5 maximal number α such that there exists a deviation of the coalition Γ from s in which every player in Γ reduces her cost by a factor of at least α. Therefore, if a game G has a set P of players and a set E of equilibria, the worst Nash is α-pareto-deficient iff α = sup E E IR min (E, P ), and the best Nash is β-pareto-deficient iff β = inf E E IR min (E, P ). However, while the results in [FT08] aim at bounding the simultaneous improvement of the players in any possible coalition, we focus on the special case of the grand coalition, involving all the players. In addition, we consider routing games, and various settings in load balancing games (including mixed strategies, uniformly-related machines and unrelated machines), whereas [FT08] focuses on load balancing games with pure strategies on identical machines. 2 Selfish Routing Games A multi-commodity network is a directed multigraph N = (V, E) (possibly containing parallel edges) together with a collection {(s 1, t 1 ),..., (s k, t k )} V V of source-sink vertex pairs, called commodities. We denote the set of edges E by [m] (where [m] = {1,..., m}), and with each edge j [m] we associate a cost function c j ( ); we denote by c the vector (c 1 ( ),..., c m ( )) of cost functions for N. (We assume throughout that c j ( ) is continuous for all j; for the results of Section 2.2 we additionally assume that c j ( ) is nondecreasing.) Finally, for each commodity i there is some amount r i of traffic that needs to be routed from s i to t i. Thus, a multi-commodity selfish routing game is simply a triple (N, r, c). The amount of traffic r i that commodity i needs to route from s i to t i can be routed through any of the paths going between these vertexes. The players in a selfish routing game are infinitesimallysmall traffic units that can make independent routing decisions; in particular, units of the same commodity may use different paths going from the commodity s source to its sink. A flow f in (N, r, c) is a vector, indexed by all the s i t i paths for all i, indicating the amount of traffic using each path. We denote by f j the total amount of traffic using the edge j (i.e. the total amount of traffic traveling through paths that include j). We say that a flow f is feasible if for every i, it routes an amount r i of traffic from the source s i to the corresponding sink t i. The cost incurred to a player p using a path P in the flow f is simply cost(p, f) = j P c j(f j ); an equilibrium flow (sometimes termed Wardrop equilibrium, first presented in [War52]) is defined naturally as a flow in which no unit of traffic can decrease its cost by unilaterally changing its path. A useful characterization of a Wardrop equilibrium is that all paths with nonzero flow of the same commodity i have the same cost γ, and all other paths from s i to t i have cost of at least γ. It is also well known that equilibrium flows exist for every network, and that all equilibrium flows on a network have the exact same cost (see [BMW59] and Chapter 18 in [NRTV07]). Since equilibrium is unique in this sense, there is no distinction between worst Nash and best Nash in nonatomic routing games. 2.1 General Networks The social cost traditionally associated with routing games is utilitarian, i.e. it is the average cost of a player. I.e. let P be the set of all s i t i paths for every commodity i, and let f P be the amount of traffic using the path P P, then the social cost of a flow f is C(f) = P P j P c j(f j )f P = j [m] c j(f j )f j. The Price of Anarchy for a game (N, r, c) is therefore defined as P OA(N, r, c) = max f C(f E ) C(f), where f E is an equilibrium flow, and the maximum is taken over all feasible flows f. Recall that when considering a utilitarian social welfare, equilibria are always ρ-pareto-efficient, 5

6 where ρ is the POA of the game at hand. We now show that for selfish routing games this is tight: Let C be a class of cost functions, and let ρ(c) be the highest possible POA for selfish routing games with cost functions from C. We construct a selfish routing game with cost functions from C, and show a feasible flow that Pareto dominates the equilibrium flow by a factor arbitrarily close to ρ(c). We begin by considering the single-commodity case: Theorem 1. Let (N, r, c) be a single-commodity selfish routing game in which all cost functions are continuous, and let ρ = P OA(N, r, c). Then for every ɛ > 0 there exists a single-commodity selfish routing game (N, r, c ) with cost functions c from the same set as c, such that (N, r, c ) has a (ρ ɛ)-pareto-deficient equilibrium. Proof. Let (N, r, c) be a single-commodity selfish routing instance with continuous cost functions and POA ρ, and fix some ɛ > 0. We denote by s and t the network s source and sink, respectively, and enumerate the s t paths in N by P 1,..., P M. (A flow f is thus a non-negative vector in R M.) W.l.o.g. we assume that the amount r that needs to be routed from s to t equals 1, and so in an equilibrium flow f E all players pay C(f E ). Recall that C(f) = C(f P1,..., f PM ) = M i=1 j P i c j (f j ) f Pi ; since for every e E, c e can be viewed as a continuous function of (f P1,..., f PM ) we get that C(f P1,..., f PM ) is also continuous (being defined as a sum of products of continuous functions). The idea behind the proof is the following. We create a new network N by concatenating q copies of N (for some large enough q), connecting every two adjacent copies by placing a zero-cost edge going from the sink of the first to the source of the second. A flow that routes all the traffic exactly as in equilibrium in each copy is an equilibrium in N, and its cost is q-times that of the original equilibrium flow. We now look at the optimal flow in N; since the total cost function C( ) is continuous, there is a flow ( p 1 q,..., p M q ) (for large enough q) routing rational amounts on the paths in N, and having a total cost larger than that of the optimal flow by at most ɛ. We can now use that latter flow in every copy of N in N, keeping the amounts routed on each path the same in every copy, but changing the sets of players routed in these paths. This can be done to achieve a flow in N with total cost of q-times the optimum (up to an additive factor of ɛ) in which all players are incurred the same cost. It then follows that this flow (ρ ɛ)-pareto-dominates the equilibrium flow. We now turn to the formal proof. Let f O be a minimum cost feasible flow vector, then ρ = C(f E ) C(f O ). Also, by the continuity of the global cost function C it holds that for every ɛ > 0 there exists δ(ɛ ) > 0 such that for every f R M with f f O < δ(ɛ ) it holds that C(f) C(f O ) < ɛ (note that by definition C(f) C(f O )). In particular, since f O is a feasible flow vector (for which M i=1 f O P i = r = 1) for every ɛ there exist integers q, p 1, p 2,..., p M such that M i=1 p i = q and such that the vector ˆf(ɛ ) = ( p 1 q,..., p M q ) has C( ˆf(ɛ )) C(f O ) < ɛ (note that ˆf(ɛ ) is also a feasible flow vector for (N, r, c)). 6

7 We now turn to construct the instance (N, r, c ). For some ɛ whose value we specify later, let p 1,..., p M be the numerators of the components of ˆf(ɛ ), and q their denominator. The graph N is comprised of q copies of N, which we label N 1,..., N q. The source s in N is s 1 (i.e. the source in N 1 ) and similarly t = t q (i.e. the sink in N q ). We connect the copies by creating a zero cost edge (t i, s i+1 ) for i = 1... (q 1), and for every other edge e i (which is the i-th copy of the edge e) we set c ei (x) = c e (x). An s t path in N can be uniquely described as a concatenation of q s t paths in N; we will use P (j) i to denote j concatenated copies of the path P i. Finally, we set r = r = 1. Now, observe that for an equilibrium flow f E in (N, r, c), the flow (f E ) (q) is an equilibrium flow in (N, r, c ) in which all players pay C ((f E ) (q) ) = q C(f E ). Recall that ˆf(ɛ ) = ( p 1 q,..., p M q ), and consider an alternative flow f (ɛ ) that partitions the players into q equally-sized sets and routes the players in the i-th set on the path that is the i-th shift of P (p 1) 1 P (p 2) 2 P (p M ) M (e.g. a player in the third set will be routed on the path P (3) M P (p 1) 1 P (p 2) 2 P (p M 3) M, assuming p M 3). It follows that for this flow, the congestion on each edge e i in N is exactly the same as the congestion on e in N when using ˆf(ɛ ). Furthermore, in the flow f (ɛ ) all players pay the exact same cost, which is q C( ˆf(ɛ )). Thus, the flow f (ɛ) dominates the equilibrium flow (f E ) (q) by and so the proof is completed by choosing q C(f E ) q C( ˆf(ɛ )) > C(f E ) ɛ + C(f O ) ; ɛ ɛ C(f O ) α ɛ Note that the construction above uses only edge cost functions that appeared in the original instance, aside from the zero cost functions between copies; however, those can be removed by unifying t i and s i+1 for every i = 1... (q 1). Theorem 1 shows that in single-commodity selfish routing games, an equilibrium may be ρ-pareto-deficient. Roughgarden, in [Rou02], proves that under some additional conditions on the class of allowable cost functions, the worst POA for multi-commodity instances can be achieved (up to an arbitrarily small additive factor) on single-commodity Pigou network instances. We therefore immediately get that under the same conditions (namely that the class of allowable cost functions is both standard and diverse, and that all cost functions are monotone) the Paretodeficiency of a Nash equilibrium in multi-commodity instances cannot be significantly worse than that of a single-commodity instance with cost functions from the same class. 2.2 Parallel-Edge Networks We have seen that for every single-commodity instance with POA ρ there exists a (possibly different) instance with an equilibrium far from Pareto optimality by a factor arbitrarily close to ρ. However, 7

8 the construction used in the proof of Theorem 1 may require a significantly larger network. This can be at least partly justified by the following result: Consider the restricted case of single commodity networks in which every route from the source to the sink has an edge that is not part of any other route. Such networks are said to have linearly-independent routes (or paths) and Milchtaich [Mil06] shows that in such networks, equilibria are guaranteed to be Pareto efficient whenever the edge cost functions are nondecreasing. Thus, in networks like the Pigou network, which clearly has linearly-independent routes, equilibria are never Pareto deficient. Another useful characterization of networks with linearly-independent routes is as the set of networks that do not contain a bad configuration [HLY97] a set of three paths P, Q, R from the source to the sink such that there exists an edge e 1 that is in P and Q but not in R, and an edge e 2 that is in P and R but not in Q. We use this characterization for proving that greedy flows in networks with linearly independent routes are also always Pareto efficient (assuming nondecreasing cost functions). Theorem 2. Let (N, r, c) be a single-commodity selfish routing game on a network with linearlyindependent routes, and with nondecreasing cost functions on all edges. Then if f is a flow obtained by a sequential greedy process, f is an equilibrium flow. Proof. Let P be a path in N going from the source to the sink, and let f be a flow on (N, r, c). We denote by C f (P ) = j P c j(f j ) the total cost payed by any unit of flow using the path P in the flow f. Now, assume that some feasible flow f obtained by a sequential greedy process is not an equilibrium flow. By a characterization of equilibrium flows, it has to be that in f there is some nonzero flow path P and another path Q such that C f (P ) > C f (Q). Now, consider the point t in time when the last traffic unit that chose the path P has made its choice. At the time t, the cost on the path Q could not have exceeded C f (Q), as the cost functions are nondecreasing and traffic arriving before time t could not have changed its routing later. It thus has to be that the cost on the path P at time t was at most C f (Q) (or this traffic unit would have used Q instead of P ), and that the cost on P was increased by traffic arriving after time t. However, this is only possible if there is some path R having some edges in common with P and on which the flow has increased after time t. We thus consider the set S of all paths Q that intersect P (i.e. have edges that are also part of P ), and for which C f (Q) < C f (P ). (W.l.o.g. we can choose P such that no other path with non-minimal cost has any traffic choosing it after the time t, so all the traffic that increased the cost on P necessarily uses paths in S.) Suppose first that for any two such paths Q and R we have P Q P R or P Q P R (i.e. all the edges that are common to P and Q are also in R, or vice versa). In this case, there is some path Q S such that for any R S has P R P Q. However, note that at time t, the cost on Q must have been at least the cost on P, and that the increase of the cost on Q after time t must have been at least the increase of the cost on P during this time, as Q shares with P all the edges on which the amount of traffic (and thus the cost) has increased. However, this implies that C f (Q) C f (P ), which contradicts Q S. It therefore has to be that there are Q, R S with P Q P R and P Q P R. However, this implies that there is some some edge e 1 in P Q and not in R, and an edge e 2 in P R and not in Q; thus P, Q, R constitute a bad configuration, implying that N does not have linearly-independent routes. Corollary 3. Let (N, r, c) be a single-commodity selfish routing game on a network with linearlyindependent routes, and with nondecreasing cost functions on all edges. Then if f is a flow obtained 8

9 by a sequential greedy process, f is Pareto optimal. We now turn to look at the seemingly simplest type of networks, that have only a source and a sink, and parallel edges directly connecting them. We call such networks parallel-edge networks, and note that these trivially have linearly-independent routes, and that the Pigou network is an example of such a network. The following result is concerned with parallel-edge networks with linear cost functions, and will be found to be especially useful for proving some of the results in Section 3. Simply put, the following theorem proves that any flow f on such networks cannot be Pareto dominated by a large factor by another flow f, unless f redirects a large portion of the traffic from some edge to edges unused by f. Theorem 4. Let N be a parallel-edge network with linear cost functions (of the form c(x) = a x, a > 0). Let f be a flow on N, and define I = {i [m] : f i > 0} as the set of edges on which f routes a positive amount of flow. Then a flow f that routes on the edges [m] \ I at most α-fraction of the flow from every edge i I may Pareto dominate f by at most 1 1 α. Proof. Assume, by contradiction, that there exists a flow f as above that Pareto dominates f by a factor larger than 1 1 α. Denote by xj i the amount of traffic moving from the edge i to the edge j when changing from f to f. It easily follows that for every h [m], (1 α)f h j I xj h and f h = i I xh i. We also denote by pj i the fraction of f j that originated from the edge i, i.e. p j i = xj i /f j. Since changing from f to f decreases the cost of every single player by more than 1 1 α, it has to be that for every two edges i, j with x j i > 0 it holds that a j xj i = a p j j fj < (1 α)a if i, which can i x be rearranged to write j i a i (1 α) < pj i f i a j. Now, consider the destinations set D i = {j I : x j i > 0} of the edges of I chosen in f by traffic leaving i. We get that f i a i j I x j i a i (1 α) = j D i j xi a i (1 α) < j pi f i = f i j pi = f i a j a j j D i j D i j I (note that p j i = xj i = 0 for every j I \ D i). This implies that 1 a i < j I for every i we can now sum over all i s and get 1 < p j i = ( 1 ) p j i = 1 a i a j a j a j i I j I j I i I j I i I p j i a j p j i a j, and since this holds (where the last equality follows from the fact that i I pj i equals 1 for every j). This is a contradiction, since clearly i I 1 a i = j I 1 a j ; we thus conclude that such f cannot exist. In the case where I = [m] it is immediate to observe that α = 0 and we obtain: Corollary 5. Let G be a parallel-edge network with linear cost functions, and f a flow such that f i > 0 for all i [m]. Then f is Pareto optimal. Thus, every flow on a parallel-edge network that uses all the edges is Pareto optimal, even if it is not an equilibrium flow. 9

10 3 Load Balancing Games Pure Strategies A load balancing game is defined by a set [m] of machines and a set [n] of jobs, where each job is associated with a weight function w i : [m] R such that w i (j) is the weight of job i on machine j. We say that the machines are uniformly-related if there are constants {w i } i [n] and { } j [m] such that for all i, j it holds that w i (j) = w i. The machines are identical if this holds with all = 1. If the machines are not uniformly-related, we say that the game is played on unrelated machines. A pure strategy profile is a function A : [n] [m] assigning every job i to a single machine j = A(i). The cost incurred to a job k assigned by A to machine j is cost(k, A) = i:a(i)=j w i(j) (i.e. we assume that on every machine, the jobs are executed in parallel). An assignment A is thus in Nash equilibrium if no player can benefit by unilaterally moving to another machine, i.e. if for every job k [n] and machine j A(k) it holds that cost(k, A) w k (j) + i:a(i)=j w i(j). Theorem 6. Let G be a load balancing game, then G has a Pareto optimal Nash equilibrium in pure strategies. Proof. The proof is essentially identical to the proof given in [EDKM03] to show that best reply dynamics converge to a Nash equilibrium, and we reiterate it here for completeness. Let A be an assignment in G, and denote by l(a) the sorted (in nonincreasing order) vector of the loads on the machines when the jobs are assigned according to A. We write l(a) l(a ) if l(a) is lexicographically smaller than l(a), i.e. if there is some index k [m] such that l(a) k < l(a ) k and l(a) i = l(a ) i for all i < k. It is easy to observe that any profitable deviation of a job i from an assignment A yields an assignment A with l(a ) l(a): Note that if a job i is assigned to machine j and finds it profitable to move to machine j, the load on j decreases, and the load on j increases, but remains lower than the original load on j (or the move would not have been profitable). Since the load on the remaining machines does not change, indeed l(a ) l(a). We thus have that the lexicographically minimal assignment is a Nash equilibrium; we now show that it is also Pareto optimal. Let A be the lexicographically minimal assignment, and let A be some other assignment, and assume by contradiction that it Pareto-dominates A. Let k be a job assigned in A to a machine with the maximum load, and so cost(k, A ) = l(a ) 1. By our contradiction assumption, cost(k, A) > cost(k, A ); however, since A A, l(a ) 1 l(a) j for all j. I.e. there is no machine in A on which the load is larger than what k experiences in A ; a contradiction. While the best Nash is always Pareto optimal, the worst Nash on unrelated machines may be arbitrarily Pareto-deficient, as the following example shows: Let ɛ > 0 be arbitrarily small and consider an instance with m machines and n = m jobs, such that for all i, w i (i) = 1 and for all i j, w i (j) = ɛ. It is easy to observe that the identity assignment A(i) = i is a Nash equilibrium that is 1 ɛ -Pareto-deficient. 3.1 Uniformly-Related and Identical Machines Load balancing games on uniformly-related machines can be viewed as atomic routing games (where each player controls a non-negligible amount of traffic) on parallel-edge networks with linear cost functions. However, we can use our results for nonatomic selfish routing games to derive bounds for load balancing games. Let G be a load balancing game on uniformly-related machines with total weight W = i [n] w i and speeds s = { } j [m]. We define a selfish routing game G (W, s) = 10

11 (N, r, c) on a parallel-edge network by creating a set of edges [m] with cost function c j (x) = x for every j [m], and r = W. Every assignment A : [n] [m] for G induces a feasible flow f A on G (W, s) in which the flow on an edge j is i:a(i)=j w i; furthermore, every player in the routing game originates from a single job i [n] in the load balancing game, and pays cost(i, A) in f A. Therefore, if an assignment A Pareto dominates A, then f A Pareto dominates f A by the same factor. This easily implies the following result. Theorem 7. Let G be a load balancing game on uniformly-related machines, and let A : [n] [m]. If either 1. A is an equilibrium assignment, or, 2. the machines of [m] are identical (i.e. s 1 = s 2 = = s m ) and A is the result of a sequential greedy assignment process, then A is Pareto optimal. Proof. First, assume that A is an equilibrium assignment, and define I = {j [m] i : A(i) = j} as the set of machine that some job uses. Then in G (W, s), I is the set of edges with nonzero flow in f A. Assume by contradiction that A is not Pareto optimal, so there exists another assignment A that Pareto dominates A. Define I = {j [m] i : A (i) = j}, then clearly, I I; otherwise let j I \ I and let i be such that A (i) = j. Since A is an equilibrium, it holds that cost(i, A) w i cost(i, A ), and thus A does not Pareto dominate A because player i pays in it at least as much as it payed in A. However, if I I then the flow f A routes all the traffic on the edges of I, and thus by applying Theorem 4 with α = 0 we get that A cannot Pareto dominate A; a contradiction. Now, assume that A is a result of a sequential greedy assignment process on identical machines, and define I as above. There are two cases: If I = [m], then by Corollary 5 we are done. Otherwise, there are machines that are not used by any of the jobs; however, since A was obtained by a greedy process and the machines have identical speeds, it has to be that on every machine j I there is only a single job (or the second job that arrived to j would have preferred to use some vacant machine l [m]\i). Thus, every job pays the minimum possible cost and there is no way to reduce the cost of any of the jobs, so again A is Pareto optimal. Unlike with identical machines, if A is the result of a sequential greedy assignment process on non-identical machines, A may be Pareto dominated by another assignment. For example, assume that we have three machines with speeds 2, 1 and 1, and three jobs with weights 1, 1 and 2. Consider the following scenario: A job of weight 1 arrives first, and chooses the fast (speed 2) machine. Then arrives the other unit weighted job, and (being indifferent about which machine to choose) chooses the fast machine as well. Finally, the heavy (weight 2) job arrives, and again chooses the fast machine (as it too would have the same cost on all the machines). In this assignment all the jobs pay a cost of = 2; however, if we assign each of the light jobs to a (distinct) slow machine and the heavy job to the fast machine we get that every job pays only 1. Thus, the sequential greedy assignment is 2-Pareto-deficient; we next show that this is the worst case possible. Theorem 8. Let G be a load balancing game on uniformly-related machines, and let A be the result of a sequential greedy assignment process. Then A is 2-Pareto-efficient. 11

12 Proof. Let A be such an assignment, and let A be an assignment that Pareto dominates it; we define I = {j [m] i : A(i) = j} and I = {j [m] i : A (i) = j}. For every j I let k(j) be the last (if any) of the jobs that arrived to j in the sequential process and have A (k(j)) I \ I. Denote by W j the total weight of jobs that arrived to j before k(j), and by W + j the total weight of jobs that arrived to j after k(j) (if there is no such job, we define W j = 0 and W + j = i:a(i)=j w i); we get that cost(k(j), A) = W j +w k(j)+w + j than (the vacant) l = A (k(j)) we also have that W j +w k(j) Now, if Wj + w k(j) W j +, then. Since upon its arrival, k(j) chose the machine j rather w j s l cost(k(j), A ). cost(k(j), A) = W j + w k(j) + W + j 2 W j + w k(j) 2 cost(k(j), A ), so k(j) reduces its cost by at most 2, and A Pareto dominates A by a factor of at most 2. Otherwise, we have that for all j, W j + > Wj + w k(j), i.e. W j + is more than half of the total weight assigned to j in A. Note that since k(j) is the last job in j that is assigned in A to a machine in I \ I, all the weight in W j + is assigned in A to machines in I. Thus, looking at the flows f A and f A in G (W, s) we get that f A routes on edges in I \I less than 1 2 of the flow routed by f A on each edge, and by Theorem 4 we get that A Pareto dominates A by a factor of less than 2. 4 Load Balancing Games Mixed Strategies A mixed strategy of a player i [n] in a load balancing game is a distribution p i = (p 1 i,..., pm i ) over the set of machines, so that i chooses to use machine j with probability p j i. We therefore get that in a mixed strategy profile p = (p 1,..., p n ), cost(i, p) = ( j [m] pj i w i (j) + ) h [m] i p j h w h(j). As one would expect, a profile p is in equilibrium if no player can benefit by unilaterally switching to a different distribution p i. Note that mixed strategies are a superset of pure strategies; therefore, we immediately obtain that for unrelated machines the worst Nash in mixed strategies may be arbitrarily Pareto-deficient. However, note that we do not get immediately from the results for pure strategies that the best equilibrium is Pareto optimal; while every such profile is still an equilibrium, it may be Pareto dominated by some mixed strategy profile (that was not possible in the pure strategies setting). This is indeed the case with uniformly-related and unrelated machines, as the following example illustrates. Proposition 9. There exists a load balancing game G with uniformly-related machines in which no Pareto optimal Nash equilibrium exists when mixed strategies are allowed. Proof. Assume a load balancing with two identical jobs of weight 1, and three machines: the speed of the first two machines is 1, and the speed of the third machine is 2 + ɛ for some small ɛ > 0. It is immediate to observe that in the unique equilibrium in this game both jobs use the fast machine (with probability 1) and thus each pay 2 2+ɛ 1. On the other hand, consider the mixed profile in which the first task gives probability 1 2 to the first machine and 1 2 to the third (fast) machine, and the second task gives probability 1 2 to the second machine and 1 2 to the third one. In this case, each 12

13 ( of the jobs pays ) ɛ 7 8. Thus, this profile Pareto-dominates the unique equilibrium by a factor arbitrarily close to 8 7. In contrast, we will show that for identical machines, the best equilibrium remains Pareto optimal even when mixed strategies are allowed. However, before getting to the proof itself, we introduce some further notation. Assume a set [m] of uniformly-related machines, a set [n] of jobs and a (perhaps mixed) strategy profile p. We denote by E p [W j ] = h [n] pj h w h the total expected weight on the machine j in the profile p. We will further denote E p [W j ] i = h [n] i p j h w h the total weight on the machine j, excluding the job i. Thus, we can write cost(i, p) = j [m] p j i Ep[W j ] + (1 p j i )w i = j [m] p j i Ep[W j ] i + w i. We can now easily prove the following theorem. Theorem 10. Let G be a load balancing game with identical machines, then G has a Pareto optimal Nash equilibrium in pure strategies. Proof. For any strategy profile p, define Φ(p) = i [n] w i cost(i, p) as the weighted sum of the players costs. It is thus clear that if a profile q dominates the profile p, then it has to be that Φ(q) < Φ(p). Our proof will show that for any (possibly mixed) strategy profile q there exists a pure equilibrium profile p with Φ(p) Φ(q), i.e. an equilibrium profile p not dominated by q. This implies that the pure equilibrium profile p that minimizes Φ(p ) in undominated, i.e. Pareto optimal. We begin with some strategy profile q. If q is a pure equilibrium, we are done. Otherwise, we will repeatedly pick some player i having a beneficial deviation from the current profile or using a mixed strategy, and change her strategy to some best response, i.e. a pure strategy that minimizes her cost given the current strategies of the other players. As load balancing games are congestion games, it is guaranteed that this process of best-response dynamics will result in a pure Nash equilibrium p. It thus remains to prove that indeed Φ(p) Φ(q); we will do so by showing that by whenever a player k plays a best response in the profile q, resulting in some new profile r, we have Φ(r) Φ(q). We begin by noting that k s choice of strategy contributes to the value of Φ in two ways: The first type of contribution is directly via the her own cost; the second type of contribution is via the costs of the other players (since k increases the load on the machines to which she gives positive probability). Now, assume that each player i k plays a strategy (distribution) q i. From the definition, it is clear that if k plays a best response to this profile, it minimizes her first type of contribution to Φ; we now show that it also minimizes her second type of contribution. Suppose that k chooses to play a distribution r k, then her contribution of the second type to Φ is w i q j i rj k w k = w k r j k q j i w i = w k r j k E q[w j ] k i [n] k j [m] j [m] i [n] k j [m] However, this is clearly minimized when k gives probability 1 to a machine l minimizing E q [W l ] k, which is exactly her best response. Thus, playing best response can never increase the value of Φ, as required. 13

14 For a load balancing game with identical machines we show that every equilibrium profile is (2 1 m )-Pareto-efficient, and that this is tight, i.e. the worst equilibrium in a game may indeed be (2 1 m )-Pareto-deficient. Theorem 11. In load balancing games with identical machines and mixed strategies, all Nash equilibria are (2 1 m )-Pareto-efficient, and this bound is tight. Proof. First, consider a game with m identical machines and n = m jobs, all with unit weight. The strategy profile in which every job gives probability 1 m to each of the machines is clearly an equilibrium, and the cost to each job in this profile is 2 1 m. However, the (equilibrium) profile in which job i uses machine i with probability 1 results in cost 1 for every job, and thus the other equilibrium is (2 1 m )-Pareto-deficient. We complete the proof by showing that in any equilibrium profile p no job i every pays more than W i m + w i (where W i = W w i ), and that in every profile q there exists a job k paying at least max { W m, w k}. This implies the theorem as any Pareto improvement is bound from above by cost(k, p) sup q S,p E cost(k, q) W k m + w k max { W m, w k} W + (m 1)w k m max{ W m, w k} 2m 1 m = 2 1 m (where S is the set of all strategy profiles and E the set of all Nash equilibria). We first upper-bound the cost of every job i in equilibrium. In equilibrium, every job i gives nonzero probability only to machines minimizing the cost function w i +E p [W j ] i, which is similar to minimizing the function E p [W j ] i. Since j [m] E p[w j ] i = W i, every machine j that minimizes this objective has E p [W j ] i W i m. Therefore, a player i never pays more than W i m + w i in equilibrium. Now, let q be some strategy profile, and consider a weighted sum of the jobs costs, where each cost is weighted by the fraction of the total weight contributed by the job paying it. We have: i [n] w i W cost(i, q) 1 W = 1 W = 1 W w i q j i E q[w j ] i [n] j [m] j [m] j [m] E q [W j ] q j i w i (E q [W j ]) 2 i [n] ( j [m] E q[w j ]) 2 1 W m = 1 W W 2 m = W m. (For the first inequality recall that cost(i, q) = ( ) j [m] qj i E q [W j ] + (1 q j i )w i and q j i 1; the other inequality follows from Cauchy-Schwarz.) Since the (weighted) average of the costs is at least W m, it cannot be that for every job i, cost(i, q) < W m. We therefore conclude that in every profile q, 14

15 there exists a job k paying at least W m ; since k also never pays less than its own weight, we get that cost(k, q) max{ W m, w k}. This completes the proof, as shown above. For the case of uniformly-related machines with mixed strategies, we show that every equilibrium is 4-Pareto-efficient; however, we do not know to show that this bound is tight. Theorem 12. In load balancing games with uniformly related machines and mixed strategies, all Nash equilibria are 4-Pareto-efficient. Proof. Assume that there exist a set [m] of machines, a set [n] of jobs, and strategy profiles p, q such that the profile p is an equilibrium and for every player i [n], cost(i, q) 1 4cost(i, p). The idea is to show that in this case there exists a job k that can unilaterally improve its cost from the profile p, implying that p cannot be an equilibrium. To that end, we first define another strategy profile r, which is a variation of the profile q. For a job i, let B i be the set of bad machines to which i gives nonzero probability and on which it pays over twice its expected cost in the profile q, i.e., { B i = j q j i > 0 E q[w j ] + (1 q j i )w } i > 2 cost(i, q). We also denote the remaining ( good ) machines to which i gives nonzero probability will be denoted by G i, so { G i = j q j i > 0 E q[w j ] + (1 q j i )w } i 2 cost(i, q). Since all q j i are non-negative, it holds that the total probability every i gives to bad machines is b i = j B i q j i < 1 2 (or the expected cost for i would have exceeded cost(i, q)). We create the new strategy profile r as follows. First, for every i, j with q j i = 0 we set rj i = 0 as well. For every i, j such that j B i, we also set r j i = 0. However, in order to keep the vector r i a distribution, we normalize the probabilities given to machines in G i to compensate for the lost probability b i ; specifically, for every i, j with j G i we set r j i = We now have w i j [m] E p [W j ] = i [n] qj i 1 b i. j [m] p j i = i [n] w i = i [n] w i j [m] r j i = j [m] E r [W j ], so there is bound to be a machine l with E r [W l ] > 0 and E p [W l ] E r [W l ]. Let k be a job such that rk l > 0. We now show that in the strategy profile p, player k can reduce its cost to less than 4 cost(k, q), by choosing to use machine l with probability 1. The cost incurred to k when doing so is E p [W l ] + (1 p l k )w k s l E r[w l ] + w k s l Recall that for every i, j it holds that r j i i [n] k r l i w i + r l k w k + w k s l qj i 1 b i i [n] k ri lw i + 2w k i [n] < k 2qi lw i + 2w k = 2 s l s l i [n] k r l i w i + 2w k s l. and that b i < 1 2 ; this implies that rj i < 2qj i. Thus, 15 i [n] k q l i w i + w k s l 2 2 cost(k, q),

16 where the last inequality holds since h [n] k q l h w h+w k s l is exactly the cost k pays on machine l in the profile q. We chose k such that r l k > 0 and so it must be that l G k; this implies that in the profile q, k pays on l at most 2 cost(i, q). Combining the two inequalities above we get that E p[w l ]+(1 p l k )w k s l < 4 cost(k, q); however, we assumed that the original cost of k in the profile p was cost(k, p) 4 cost(k, q). Therefore, in the profile p, player k can benefit from unilaterally moving to machine l, and thus p is not an equilibrium. 5 Conclusions and Open Problems In this paper, we introduced the question of the distance from Pareto optimality and analyzed it in the context of selfish routing and load balancing games. Our results concentrated mainly on equilbria of such games, answering both the question of how Pareto-deficient may an arbitrary Nash equilibrium get, and what bounds on the Pareto-deficiency of the best equilibrium of a game can be guaranteed. We have shown that in some cases, deficiency may be large (selfish routing games on general networks, and as an extreme example, Prisoner s Dilemma), in some cases equilibria are in fact Pareto optimal (load balancing with pure strategies), and in other cases deficiency can be bounded by a constant (load balancing on uniformly-related machines with mixed strategies). We have also shown cases where the worst and best equilibria are equally Pareto-deficient, and cases in which the worst equilibria are strictly less efficient (an extreme example is load balancing on unrelated machines, where the best Nash is always Pareto optimal, while the worst Nash may be arbitrarily deficient). We have also analyzed the deficiency of non-equilibrium outcomes: Namely, flows that use all edges in selfish routing games on parallel-edge networks, and outcomes obtained by a sequential greedy process. A natural direction for further research is the analysis of the Pareto efficiency/deficiency of solutions in other games, as well as other solution concepts in these and other games. In addition, there are a few cases left open in this work, including: Flows obtained by sequential greedy processes on general routing networks. Unlike with parallel-edge networks, such flows in general networks need not be equilibrium flows, even in single-commodity instances. However, the lower bound of the POA value ρ still holds for such flows. Sequential greedy assignments for load balancing on unrelated machines. It can be shown that there are instances in which greedy assignments are not 2-Pareto-efficient. What is the Pareto efficiency/deficiency this case? (Worst) mixed equilibrium for load balancing on uniformly-related machines. We have shown that such equilibria are always 4-PE, but suspect that the real bound may be lower. References [ADK + 04] Elliot Anshelevich, Anirban Dasgupta, Jon M. Kleinberg, Éva Tardos, Tom Wexler, and Tim Roughgarden. The price of stability for network design with fair cost allocation. In FOCS, pages ,

17 [BMW59] [Coh98] [Dub86] [DY09] Martin Beckmann, Charles B. McGuire, and Christopher B. Winsten. Studies in the economics of transportation. Yale University Press, Joel E. Cohen. Cooperation and self-interest: Pareto-inefficiency of Nash equilibria in finite random games. Proceedings of the National Academy of Sciences of the United States of America, 95(17): , Pradeep Dubey. Inefficiency of nash equilibria. Mathematics of Operations Research, Vol. 11, No. 1, Ilias Diakonikolas and Mihalis Yannakakis. Small approximate pareto sets for biobjective shortest paths and other problems. SIAM Journal on Computing, 39(4): , [EDKM03] Eyal Even-Dar, Alexander Kesselman, and Yishay Mansour. Convergence time to nash equilibria. In ICALP, pages , [FT08] [HLY97] [HS78] [KP99] Michal Feldman and Tami Tamir. Approximate strong equilibrium in job scheduling games. In SAGT, pages 58 69, Ron Holzman and Nissan Law-Yone. Strong equilibrium in congestion games. Games and Economic Behavior, 21(1-2):85 101, October Leonid Hurwicz and David Schmeidler. Construction of outcome functions guaranteeing existence and pareto optimality of nash equilibria. Econometrica, Vol. 46, No. 6, Elias Koutsoupias and Christos H. Papadimitriou. Worst-case equilibria. In STACS, pages , [LGCM10] Julien Legriel, Colas Le Guernic, Scott Cotton, and Oded Maler. Approximating the pareto front of multi-criteria optimization problems. In TACAS, pages 69 83, [Lor84] P. Loridan. ɛ-solutions in vector minimization problems. Journal of Optimization Theory and Applications, Vol 43, No. 2, [Mas99] [Mil06] Eric Maskin. Nash equilibrium and welfare optimality. The Review of Economic Studies, Vol. 66, No. 1, Special Issue: Contracts, Igal Milchtaich. Network topology and the efficiency of equilibrium. Games and Economic Behavior, 57(2): , [Nas50] John F. Nash. The bargaining problem. Econometrica, Vol. 18 No. 2, [NRTV07] [PY00] [Rou02] Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V. Vazirani. Algorithmic Game Theory. Cambridge University Press, New York, NY, USA, Christos H. Papadimitriou and Mihalis Yannakakis. On the approximability of tradeoffs and optimal access of web sources (extended abstract). In Proceedings 41st Annual Symposium on Foundations of Computer Science, pages 86 92, Tim Roughgarden. The price of anarchy is independent of the network topology. In Journal of Computer and System Sciences, pages ,

18 [War52] John G. Wardrop. Some theoretical aspects of road traffic research. Proceedings of the Institution of Civil Engineers, Part II, 1(36): ,