Large Population Potential Games

Transcription

1 Large Populaton Potental Games Wllam H. Sandholm January 8, 2009 Abstract We offer a parsmonous defnton of large populaton potental games, provde some alternate characterzatons, and demonstrate the advantages of the new defnton over the exstng defnton, but also show the equvalence of the two defntons. 1. Introducton Potental games admt a varety of practcal applcatons and possess appealng theoretcal propertes, most notably the convergence of myopc adjustment processes to Nash equlbrum. Sandholm (2001), buldng on the work of Beckmann et al. (1956), Monderer and Shapley (1996), and Hofbauer and Sgmund (1988), defnes and analyzes potental games n settngs wth large populatons of anonymous agents. In ths paper, we provde a new defnton of potental games for these large populaton settngs. At frst glance t may seem that the defnton proposed here s more general but more complcated than the one from Sandholm (2001); however, we show that f populaton szes are fxed, the two defntons are equvalent. Equally mportantly, the new defnton s both more parsmonous and easer to verfy than the old one: whle under the old defnton, checkng whether a game s a potental game requres constructng extended payoff functons that satsfy certan propertes, the new defnton can be verfed drectly. Whle our analyss allows for multple populatons (.e., multple player roles), t s easer to descrbe the man deas nformally by focusng on the sngle populaton case. In ths settng, an n-strategy populaton game s defned by a vector of payoff functons F = (F 1,..., F n ) defned on the set of populaton states that s, the smplex X n R n. I thank an anonymous referee, an anonymous Assocate Edtor, and the Edtors for helpful comments, and Katsuhko Aba for able research assstance. I also gratefully acknowledge fnancal support from NSF Grants SES and SES Department of Economcs, Unversty of Wsconsn, 1180 Observatory Drve, Madson, WI 53706, USA. e-mal: whs@ssc.wsc.edu; webste: whs.

2 In Sandholm (2001), potental games are defned usng standard concepts from multvarable calculus. A populaton game F s called a potental game f there s a scalar-valued functon f, the potental functon, whose gradent always equals the vector of payoffs; put dfferently, the payoff to strategy must always be gven by the th partal dervatve of f. By standard results, F s a potental game n ths sense f and only f ts dervatve matrces DF(x) are symmetrc, so that correspondng cross partal dervatves of F are equal. Because the statements above nvolve partal dervatves, they only make sense f the relevant functons are defned on a full-dmensonal subset of R n for nstance, on the postve orthant R n +. In partcular, checkng the symmetry of the dervatve matrx DF(x) requres us to extend the payoff functons from the smplex X to the orthant R n +. Snce ponts outsde the smplex are not populaton states, extendng payoffs n ths way s not parsmonous. Worse stll, ths approach makes t dffcult to decde whether a gven populaton game s a potental game. Even f the extensons one has checked do not satsfy the symmetry condton, an extenson that one has not checked may do so. Smple examples n Sectons 3 and 4 show that ths possblty s real. We therefore offer a defnton of potental games that does not requre such extensons: payoffs and potental need only be defned on the smplex. Later on, n Theorem 4.3, we characterze potental games usng a dervatve symmetry condton, but one that contnues to make sense when payoffs are only defned on the smplex. From a game theoretc pont of vew, the key dfference between the new and old defntons s ths: Whle under the old defnton, the gradent f completely descrbes payoffs under F, under the new defnton, the gradent need only descrbe relatve payoffs under F n other words, dfferences between the payoffs of pars of strateges. In ths sense, the new defnton accords more closely wth that of Monderer and Shapley (1996), as ther normal form potental games are defned by the property that dfferences n payoffs are captured by dfferences n potental. The new defnton and characterzaton employ the orthogonal projecton Φ. When appled to a payoff vector π, Φ returns the demeaned verson of that vector, so that Φπ only retans nformaton about relatve payoffs. Usng Φ n the defnton of potental games s an uncomplcated way of ensurng that only relatve payoffs are regulated by the defnton. What s the lnk between the old and new defntons of large populaton potental games? Surprsngly, Theorem 4.1 shows that the two defntons are equvalent: gven a payoff functon F and potental functon f defned on the smplex, there s always an extenson of these functons to R n + that satsfes the defnton of potental games from Sandholm (2001). In partcular, the nformaton about payoffs not ntally captured 2

3 by f namely, nformaton about average payoffs can always be ncorporated nto the extenson of f. Ths result tells us that the old defnton of potental games does not ental a loss of generalty. But the new approach s superor: snce t does not requre extensons, t allows us to decde whether a populaton game s a potental game n a systematc way. Secton 2 ntroduces populaton games. Secton 3 revews the defnton of potental games from Sandholm (2001), explans the nadequaces of ths defnton, and proposes a new one. Secton 4 relates the old and new defntons, and also descrbes stuatons n whch the old defnton s more useful than the new one. 2. Games Played by Large Populatons 2.1 Populaton Games Let P = {1,..., p} be a socety consstng of p 1 populatons of agents. Agents n populaton p form a contnuum of mass m p > 0. The set of strateges avalable to agents n populaton p s denoted S p = {1,..., n p }, and has typcal elements, j, and (n the context of normal form games) s p. We let n = p P n p equal the total number of pure strateges n all populatons. Durng game play, each agent n populaton p selects a strategy from S p. The set of populaton states (or strategy dstrbutons) for populaton p s thus X p = {x p R np + : S p xp = m p }. The scalar x p R + represents the mass of players n populaton p choosng strategy S p. Elements of X = p P X p = {x = (x 1,..., x p ) R n + : x p X p }, the set of socal states, descrbe behavor n all p populatons at once. We generally take the sets of populatons and strateges as fxed and dentfy a game wth ts payoff functon. A payoff functon F: X R n s a contnuous map that assgns each socal state a vector of payoffs, one for each strategy n each populaton. F p : X R denotes the payoff functon for strategy S p, whle F p : X R np denotes the payoff functons for all strateges n S p. When p = 1, we omt the redundant superscrpt p from all of our notaton. Example 2.1. Random matchng n normal form games. A p-player normal form game s defned by a strategy set S p = {1,..., n p } and a utlty functon U p : S R for each player p P = {1,..., p}, p 2. The doman of U p s the set of pure strategy profles S = q P S q. Suppose that agents from p unt-mass populatons are randomly matched to play the normal form game U = (U 1,..., U p ), wth one agent from each populaton q beng drawn to serve n player role q. Then the probablty that an agent from populaton p s matched wth opponents who jontly play strategy profle s p S p = r p S r s r p x r s. The populaton r 3

4 game defned by ths random matchng procedure therefore has the multlnear payoff functon (1) F p s p (x) = 2.2 Full Populaton Games U p (s 1,..., s p ) x r sr for all x X. s p S p r p In searchng for nterestng propertes of populaton games, t seems natural to consder the margnal effect of addng new agents choosng strategy j S q on the payoffs of agents currently choosng strategy S p. Ths effect s captured by the partal dervatve Fp. x q j But f F p s only defned on X, the payoffs that would arse were newcomers to enter the populaton are not defned. For ths reason, even f F p s dfferentable (n a sense to be made explct below), ts partal dervatves do not exst. To ensure that partal dervatves exst, we can extend the doman of payoff functons from the state space X = {x = (x 1,..., x p ) R n + : S p xp = m p } to the entre postve orthant R n +. We call the resultng game F : R n + R n a full-dmensonal populaton game, or smply a full populaton game. As suggested above, one can sometmes nterpret ths extended defnton of payoffs as descrbng the values that payoffs would take were the populaton masses to change. For nstance, ths nterpretaton s the natural one n congeston games: here t makes sense to consder the effects of changng the number of drvers commutng between a certan orgn/destnaton par, and t s clear how one ought to defne the payoffs at any newly feasble state x R n + X. 1 But n other settngs, there may be no one choce of extenson that s obvously correct from a game-theoretc pont of vew. Example 2.2. Random matchng n normal form games revsted. Suppose we would lke to defne a full populaton game verson of the random matchng game (1). Many extensons of defnton (1) from X to R n + are equally good n prncple. But of these, the easest to wrte down s the extenson that retans the multlnear functonal form from (1): (2) F p s p (x) = U p (s 1,..., s p ) x r s for all x r Rn +. s p S p r p 1 In partcular, the cost of choosng path should equal the sum of the delays on the lnks n the path, where the delay on a lnk s descrbed by a nondecreasng functon of the number of drvers on that lnk see Beckmann et al. (1956) or Sandholm (2001). 4

5 3. Large Populaton Potental Games 3.1 Full Potental Games We can now present a defnton of potental games used n Sandholm (2001). 2 Defnton. We call the full populaton game F : R n + R n a full potental game f there exsts a contnuously dfferentable functon f : R n + R satsfyng (FP) f (x) = F(x) for all x R n +. Property (FP) can be stated more explctly as f x p (x) = F p (x) for all p P, Sp and x R n +. We call the functon f, whch s unque up to an addtve constant, the full potental functon for the game F. That payoffs are fully captured by the scalar-valued functon f s the source of full potental games attractve propertes. Nash equlbra of F can be characterzed n terms of Kuhn-Tucker condtons for maxmzers of f. Moreover, reasonable evolutonary dynamcs namely, dynamcs n whch strateges growth rates are postvely correlated wth ther payoffs ascend potental, and therefore converge to sets of rest ponts from all ntal condtons. Snce under many dynamcs the rest ponts are dentcal to the Nash equlbra of F, ths global convergence result provdes a strong justfcaton of the predcton of Nash equlbrum play. Suppose that F s contnuously dfferentable (C 1 ). Then by the standard ntegrablty condton from multvarable calculus (see, e.g., Lang (1997)), F s a full potental game f and only f t satsfes full externalty symmetry: (FES) D F(x) s symmetrc for all x R n +. More explctly, F s a full potental game f and only f (3) F p (x) = x q j F q j (x) for all S p, j S q, p, q P, and x R n +. x p 2 Actually, whle we suppose here that payoffs and potental are defned on R n +, n Sandholm (2001) we assumed that these were defned on the smaller set X = p P X p R n +, where X p = {x p R np + : S p xp [m p ε, m p + ε]} for some ε > 0. As both of these domans are full-dmensonal, they accomplsh the same purpose. 5

6 Thus, full externalty symmetry has a natural game-theoretc nterpretaton: the effect on the payoffs to strategy S p of ntroducng new agents choosng strategy j S q must always equal the effect on the payoffs to strategy j of ntroducng new agents choosng strategy. 3.2 An Example The dffcultes that the defnton of full potental games allows are llustrated n the followng example. Example 3.1. Random matchng n normal form potental games. The normal form game U s a potental game (Monderer and Shapley (1996)) f there s a potental functon V : S R such that (4) U p (ŝ p, s p ) U p (s p, s p ) = V(ŝ p, s p ) V(s p, s p ) for all ŝ p, s p S p, s p S p, and p P. In words, U s a potental game f any unlateral devaton has the same effect on the devator s payoffs as t has on potental. Equvalently, U s a potental game f there s a potental functon V and auxlary functons W p : S p R such that (5) U p (s) = V(s) + W p (s p ) for all s S and p P, so that each player s payoff s the sum of a common payoff term and a term that only depends on opponents behavor. Suppose that p populatons of agents are randomly matched to play the normal form potental game U. The full populaton game we obtan from equaton (2) s (6) F p (x) = (V(s) + W p (s p )) x r s p s for all x r Rn +, s p S p r p where V and W p are the potental and auxlary functons from equaton (5). Snce the normal form game U admts a potental functon, t seems natural to expect F to admt one as well. But f we compute the partal dervatves of payoffs, we fnd that F p s p x q s q (x) = F q s q x p s p (x) = s {p,q} S {p,q} s {p,q} S {p,q} ( V(s) + W p (s q, s {p,q} ) ) r p,q ( V(s) + W q (s p, s {p,q} ) ) x r s r. r p,q x r sr, whle 6

7 Thus, wthout further assumptons about W p and W q, F s not a full potental game. If each W p s dentcally zero, so that U s a common nterest game, then the partal dervatves above are equal, and so F s a full potental game. In ths case, f we let f : R n + R equal aggregate payoffs n F dvded by p, (7) f (x) = 1 p x p F p (x) = V(s) x r s p s p s r, s p S p s S we fnd that f x p (x) = s p p P V(s) x r s = F p (x). r s p s p S p r p Thus, f (x) = F(x) for all x R n +, and so f s a full potental functon for F. r P Ths example seems to suggest that random matchng n a normal form potental game does not generate a full potental game. But we wll soon see that the example s msleadng. Whle equaton (6) s the only way to defne the payoffs of the random matchng game on the state space X, equaton (6) mplctly uses extenson (2) to defne payoffs on the rest of R n +. Theorem 4.1 shows that by choosng the extenson more carefully, one can ensure that condtons (FP) and (FES) are satsfed. More fundamentally, Example 3.1 llustrates the dffcultes nherent n defntons that requre the exstence of extensons satsfyng certan propertes. If we only know how payoffs must be defned on the state space X, t s not obvous how to determne whether an extenson to R n + satsfyng condtons lke (FP) and (FES) exsts: the fact that we have not found an extenson satsfyng these propertes does not necessarly mean that no such extenson exsts. The most parsmonous approach to ths problem s to work drectly wth payoff and potental functons defned on the state space X. Snce n ths case partal dervatves of payoffs and potental are not defned, the usual defntons from multvarate calculus do not apply, and so must be replaced by notons of dfferentaton for functons defned on affne spaces. In geometrc terms, ths means that dervatves can only be evaluated wth respect to drectons tangent to X, whch represent feasble changes n the socal state. As we wll see, the game theoretc consequence of ths restrcton s that potental functons defned on X only provde nformaton about relatve payoff levels. Once ths parsmonous analyss has been carred out, t s not dffcult to ntroduce alternatve defntons that use payoff extensons, and thus avod affne calculus, but that do not cause the dffcultes llustrated n the example above. We show how ths s done 7

8 n Secton 3.4 and Corollary 4.6 below. 3.3 Dervatves and Tangent Spaces We now ntroduce the defntons we need to proceed wth the analyss proposed above. For a more thorough treatment of these deas, see Akn (1990). The tangent space TX of the state space X s the smallest subspace of R n contanng all drectons of moton through X. One can verfy that TX = p P TX p, where the set TX p = {z p R np : S p zp = 0} s the tangent space of X p. Let f : X R be a scalar-valued functon on X. Then f s dfferentable at x X f there s a lnear map D f (x) : TX R, the dervatve of f at x, satsfyng f (x + z) = f (x) + D f (x)z + o(z) for all z TX. The gradent of f at x s the unque vector f (x) TX such that D f (x)z = f (x) z for all z TX. It s worth emphaszng that the unqueness requrement here s wth respect to the set TX. More generally, f v R n s any vector that represents D f (x) n the sense above, then f (x) = Φv, where the matrx Φ R n n s the orthogonal projecton of R n onto TX defned below. Analogous defntons apply to vector-valued functons on X. If F : X R n s C 1, ts dervatve at x X s a lnear map from TX to R n ; whle many matrces n R n n can represent ths dervatve, applyng the logc above to each component of F shows that there s a unque such matrx, the dervatve matrx DF(x), whose rows are n TX. A leadng role n our analyss s played by the orthogonal projecton onto TX p, represented by the symmetrc matrx Φ R np n p. Ths matrx can be wrtten explctly as Φ = I 1 n p 11, where I s the dentty matrx and 1 R np s the vector of ones. Snce TX = p P TX p, t follows that the orthogonal projecton onto TX s gven by the block dagonal matrx Φ = dag(φ,..., Φ) R n n. If π p R np s a payoff vector for populaton p, then the projected payoff vector Φπ p = π p 1 1 n p S p π p π p 1 π p subtracts the average payoff π p from each component of π p. In other words, Φ preserves the dfferences between components of π p whle normalzng ther sum to zero. The 8

9 resdual from the projecton, (I Φ)π p = 1 π p, s a constant vector whose components all equal π p, the average payoff of strateges belongng to populaton p. By the same logc, f π = (π 1,..., π p ) R n s a payoff vector for the socety, then Φπ = (Φπ 1,..., Φπ p ) normalzes each of the p peces of the vector π separately, and (I Φ)π = (1 π 1,..., 1 π p ) descrbes average payoffs n each populaton. 3.4 Potental Games Wth these prelmnares n hand, we present our new defnton. Defnton. Let F: X R n be a populaton game. We call F a potental game f t admts a potental functon: a C 1 functon f : X R that satsfes (P) f (x) = ΦF(x) for all x X. Snce the potental functon f has doman X, the gradent vector f (x) s by defnton an element of the tangent space TX. Condton (P) requres that ths gradent vector always equal ΦF(x), the projecton of the payoff vector F(x) onto TX. Put dfferently, the gradent of the potental functon f must capture relatve payoffs n F. At the cost of sacrfcng parsmony, one can avod affne calculus by usng a functon defned throughout R n + to play the role of the potental functon f. To do so, one smply ncludes the projecton Φ on both sdes of the analogue of equaton (P). Observaton 3.2. If F s a potental game wth potental functon f : X R, then any C 1 extenson f : R n + R of f satsfes (8) Φ f (x) = ΦF(x) for all x X. Conversely, f the populaton game F admts a functon f satsfyng condton (8), then F s a potental game, and the restrcton f = f X s a potental functon for F. Ths observaton s mmedate from the relevant defntons. In partcular, f f and f agree on X, then for all x X the gradent vectors f (x) and f (x) defne dentcal lnear operators on TX, mplyng that Φ f (x) = Φ f (x). But snce Φ f (x) = f (x) by defnton, t follows that Φ f (x) = f (x); ths equalty and defnton (P) yeld the result. Observaton 3.2 can be restated n a way that uses smple dfferences rather than the orthogonal projecton Φ. 3 Note that condton (8) s equvalent to the dual condton z f (x) = z F(x) for all z TX and x X. 3 I thank an anonymous referee for suggestng condton (9). 9

10 Snce the set of vectors of the form e p e p j equvalent to span the subspace TX, ths condton s tself (9) f x p j (x) f x p (x) = F p (x) j Fp(x) for all p P,, j Sp and x X. We therefore have Proposton 3.3. Observaton 3.2 remans true f condton (8) s replaced by condton (9). For an applcaton of ths result, see Secton 3.5 below. We conclude wth a representaton of potental games and potental functons that uses multpopulaton versons of condtons due to Hofbauer and Sgmund (1988, p. 243). Let X p = {x p R np 1 + : n p 1 =1 x p m p } denote the projecton of X p R np onto R np 1, and let X = p P X p. Then ψ(x p ) = (x p,..., x p, 1 n p 1 mp n p 1 =1 x p ) s a bjecton from X p to X p, and ψ(x ) = (ψ 1 (x 1 ),..., ψ p (x p )) s a bjecton from X to X. Proposton 3.4. Let F : X R n be a populaton game, and defne the functon G : X R n p by (10) G p (x ) = Fp (ψ(x )) Fp n p (ψ(x )) for all p P, < n p and x X. Then F s a potental game wth potental functon f : X R f and only f there exsts a functon g : X R such that (11) g x p (x ) = G p (x ) for all p P, < np and x X. When these statements are true, f and g are related by g(x ) = f (ψ(x )) + c. Proof. Suppose frst that F s a potental game wth potental functon f. Then defnng g : X R by g(x ) = f (ψ(x )), and lettng e p denote the (, p)th standard bass vector n R n, we fnd that g (x ) = x p f (e p e p ) (ψ(x )) = Fp(ψ(x )) Fp (ψ(x )) = G p (x ), n p n p so equaton (11) holds. Conversely, suppose that G and g satsfy equaton (11). If we defne f : X R by 10

11 f (x) = g(ψ 1 (x)), then snce ψ 1 (x) = (..., x p 1,..., xp n p 1, xp+1 1,...) t follows that f (e p e p n p ) g (x) = x p (ψ 1 (x)) = G p (ψ 1 (x)) = F p (x) Fp (x). n p As vectors of the form e p e p n p potental functon f. span TX, we conclude that F s a potental game wth 3.5 Example Revsted For a frst applcaton of our new defnton, we revst random matchng n normal form potental games (Example 3.2). 4 Let U be a normal form potental game wth potental functon V and auxlary functons W p, and let F : X R n be the correspondng populaton game: U p (s) = V(s) + W p (s p ); F p (x) = (V(s) + W p (s p )) s p s p S p r p x r s r Ths F s the restrcton to X of the full potental game F : R n + R n from equaton (6). If we defne the functon f : R n + R as n equaton (7), we fnd that f (x) = s S V(s) x r s r, r P f x p (x) f x p (x) = ŝ p s p s p S p ( V(ŝ p, s p ) V(s p, s p ) ) = F p ŝ p (x) F p s p (x). Therefore, by Proposton 3.3, F s a potental game wth potental functon f = f X. Conversely, let U be a normal form game, and let F : X R be the correspondng populaton game as defned n equaton (1). Snce the payoffs to an agent n populaton p do not depend on the choces of other agents n populaton p, we can wrte F p (x) = F p (x p ) whenever t s convenent to do so. 4 I thank an anonymous referee for suggestng the arguments to follow, whch mprove sgnfcantly on my ntal proof. r p x r s r 11

12 Suppose that F s a potental game wth potental functon f : X R as defned n equaton (P). Fx a player p and two strateges ŝ p, s p S p, and let e p denote the (, p)th standard bass vector n R n. Snce e p e p TX, defnton (P) mples that ŝ p s p (e p e p ) f (x) = (e p e p ) F(x) = F p (x) F p (x) = F p (x p ) F p (x p ) ŝ p s p ŝ p s p ŝ p s p ŝ p s p for all x X. Thus, lettng ι p denote the th standard bass vector n R np, and settng y p (t) = t ι p + (1 t) ι p, we have that ŝ p s p f (ι p ŝ p, x p ) f (ι p s p, x p ) = 1 0 f (y p (t), x p ) (e p ŝ p e p s p ) dt = F p ŝ p (x p ) F p s p (x p ). Now defne V : S R by V(s) = f (ι 1,..., ι p s 1 s p ). Fx a strategy s q S q for each player q p; then settng y q = ι q for each such q, we fnd that s q U p (ŝ p, s p ) U p (s p, s p ) = F p (y p ) F p (y p ) ŝ p s p = f (ι p, y p ) f (ι p, y p ) ŝ p s p = V(ŝ p, s p ) V(s p, s p ). Therefore, V s a potental functon for U as defned n equaton (4). In summary, we have establshed Proposton 3.5. Let U be a normal form game, and let F : X R n be the populaton game generated by random matchng n U. Then F s a potental game f and only f U s a potental game. 4. Characterzatons of Large Populaton Potental Games 4.1 Creatng Full Potental Games from Potental Games What s the relatonshp between full potental games and potental games? Under the former defnton, condton (FP) requres that payoffs be completely determned by the potental functon, whch s defned on R n +. Under the latter, condton (P) asks only that relatve payoffs be determned by the potental functon, now defned just on X. To understand the relatonshp between the two defntons, take a potental game F : X R n wth potental functon f : X R as gven, and extend f to a full potental functon f : R n + R. Theorem 4.1 shows that the lnk between the full potental game F f and the orgnal game F depends on how the extenson f s chosen. 12

13 Theorem 4.1. Let F : X R n be a potental game wth potental functon f : X R. Let f : R n + R be any C 1 extenson of f, and defne the full potental game F : R n + R n by F(x) = f (x). Then () () The populaton games F and F X have the same relatve payoffs: ΦF(x) = Φ F(x) for all x X. One can choose the extenson f n such a way that F and F X are dentcal. Part () of the theorem shows that the full potental game F generated from an arbtrary extenson of the potental functon f exhbts the same relatve payoffs as F on ther common doman X. It follows that F and F have the same best response correspondences and Nash equlbra, but may exhbt dfferent average payoff levels. Part () of the theorem shows that by choosng the extenson f approprately, we can make F and F dentcal on X. To accomplsh ths, we construct the extenson f n such a way (equaton (12) below) that ts dervatves at states n X evaluated n drectons orthogonal to TX encode nformaton about average payoffs from the orgnal game F. In concluson, Theorem 4.1() demonstrates that f populaton masses are fxed, so that the relevant set of socal states s X, then defnton (FP), whle more dffcult to check, does not ental a loss of generalty relatve to defnton (P). Proof of Theorem 4.1. Part () follows from the fact that Φ F(x) = Φ f (x) = f (x) = ΦF(x) for all x X; compare the dscusson followng Observaton 3.2. To prove part (), we frst extend f and F from the state space X to ts affne hull aff(x) = {y = (y 1,..., y p ) R n : S p yp = m p }. Let ˆ f : aff(x) R be a C 1 extenson of f : X R, and let ĝ p : aff(x) R be a contnuous extenson of populaton p s average payoff functon, 1 n p 1 F p : X R. (The exstence of these extensons follows from the Whtney Extenson Theorem see Krantz and Parks (1999).) Then defne Ĝ : aff(x) R n by Ĝ p (x) = 1ĝ p (x), so that F(x) = ΦF(x) + (I Φ)F(x) = f ˆ (x) + Ĝ(x) for all x X. If after ths we defne ˆF : aff(x) R n by ˆF(x) = f ˆ (x) + Ĝ(x), then ˆF s a contnuous extenson of F, and f ˆ (x) = Φ ˆF(x) for all x aff(x). Wth ths groundwork complete, we can extend f to all of R n + va (12) f (y) = ˆ f (ξ(y)) + (y ξ(y)) ˆF(ξ(y)), where ξ(y) = Φy + z TX s the closest pont to y n aff(x). (Here, z TX s the orthogonal translaton vector that sends TX to aff(x): namely, (z TX )p = mp 1.) Snce F f, the chan n p and product rules mply that F(y) = f (y) 13

14 = f (ξ(y)) Φ + (y ξ(y)) DF(ξ(y))Φ + F(ξ(y)) (I Φ) = (ΦF(ξ(y))) Φ + (y ξ(y)) DF(ξ(y))Φ + F(ξ(y)) F(ξ(y)) Φ = F(ξ(y)) ΦΦ + (y ξ(y)) DF(ξ(y))Φ + F(ξ(y)) F(ξ(y)) Φ = (y ξ(y)) DF(ξ(y))Φ + F(ξ(y)). In partcular, f x X, then ξ(x) = x, and so F(x) = F(x). Ths completes the proof of the theorem. Example 4.2. Random matchng n symmetrc normal form potental games. If a sngle populaton of agents are randomly matched to play the symmetrc two-player game U = (U 1, U 2 ) = (A, A ), the resultng populaton game F : X R n has payoff functon F(x) = Ax. If U s a normal form potental game, then any potental functon V of U s a symmetrc matrx. In fact, Sandholm (2008) shows that U s a potental game f and only f ΦAΦ s symmetrc, and that n ths case V = ΦAΦ+ΦA(I Φ)+(I Φ)A Φ s a potental functon for U. If we defne f : X R by f (x) = 1 2 x Vx, then f (x) = ΦVx = ΦAx = ΦF(x) for all x X, so F s a potental game wth potental functon f. Moreover, f we defne f : R n + R by (13) f (x) = 1 2 ξ(x) Vξ(x) + (x ξ(x)) Aξ(x), where ξ(x) = x (1 1 x)1, then the proof of Theorem 4.1 shows that f (x) = F(x) for all x X, so that f encodes all nformaton about payoffs n F, not just nformaton about relatve payoffs. For a smple example, suppose that 4 0 A = 3 2, so that V = s a potental functon for U = (A, A ). Then f (x) = 1 2 x Vx = 1 ( (x1 ) 2 + 2(x 2 2 ) 2) satsfes f (x) = ΦF(x) = ΦAx for all x X, whle applyng equaton (13) shows that f (x) = 1 ( ) 13(x1 ) 2 7(x 2 ) 2 6x 1 x 2 + 6x x 2 8 satsfes f (x) = F(x) = Ax for all x X. 4.2 Dfferental Characterzatons of Potental Games Lke full potental games, potental games can be characterzed by a symmetry condton on the payoff dervatves DF(x). 14

15 Theorem 4.3. Suppose that the populaton game F : X R n s C 1. Then F s a potental game f and only f t satsfes externalty symmetry: (ES) DF(x) s symmetrc wth respect to TX TX for all x X. Condton (ES) requres that at each state x X, the dervatve DF(x) be a symmetrc blnear form on TX TX. Ths condton can be stated more explctly as (14) z DF(x)ẑ = ẑ DF(x)z for all z, ẑ TX and x X. Proof of Theorem 4.3. To begn, suppose that F s a potental game wth potental functon f : X R as n condton (P). Ths means that for all x X we have f (x) = ΦF(x), or, equvalently, that for all x X, f (x) = F(x) as lnear forms on TX. Takng the dervatve of each sde of ths dentty wth respect to drectons n TX, we fnd that 2 f (x) = DF(x) as blnear forms on TX TX. But snce 2 f (x) s a symmetrc blnear form on TX TX (by vrtue of beng a second dervatve see Lang (1997, Theorem )), DF(x) s as well. Next, suppose that F satsfes condton (ES). Defne ξ : R n aff(x) by ξ(y) = Φy+z TX, as n the proof of Theorem 4.1. Let X = {y R n : ξ(y) X}, and defne F : X R n by F(y) = ΦF(ξ(y)). Then dfferentatng yelds (15) DF(y) = D ( ΦF(ξ(y)) ) = ΦDF(ξ(y))Φ. By the standard ntegrablty condton, F admts a potental functon f and only f DF(y) s symmetrc for all y X. Equaton (15) tells us that the latter statement s true f and only f F satsfes condton (ES). Now let f be a potental functon for F, and let f = f. Then snce ξ(x) = x for all X x X, we fnd that f (x) = Φ f (x) = ΦF(x) = Φ ( ΦF(ξ(x)) ) = ΦF(x), and so f s a potental functon for F n the sense of condton (P). Ths completes the proof of the theorem. Example 4.4. Two-strategy games. If F: X R n s a two-strategy game (.e., f p = 1, m = 1, and n = 2), the state space X s the smplex n R 2, so the tangent space TX s spanned by 15

16 the vector d = e 2 e 1. Thus, f z and ẑ are vectors n TX, then z = kd and ẑ = ˆkd for some real numbers k and ˆk. Ths mples that that z DF(x)ẑ = k ˆk d DF(x) d = ẑ DF(x)z for all x X, and so that F s a potental game. x1 0 Indeed, t s easy to verfy that the functon f : X R defned by f (x 1, 1 x 1 ) = (F 1 (t, 1 t) F 2 (t, 1 t)) dt s a potental functon for F. Whle one can arrve at ths f by an educated guess, one can also construct f from F mechancally usng Proposton 3.4: The functon G : X = [0, 1] R defned by G(x ) = G 1 (x ) = F 1 (x, 1 x ) F 2 (x, 1 x ) has a one-dmensonal doman, and so s ntegrable; f we let g(x ) = x 0 G(t) dt, then f (x) = g(x 1) s the potental functon for F ntroduced above. Let e p denote the (, p)th standard bass vector n R n. Snce the equalty n (14) s lnear both n z and n ẑ, and snce the set of vectors of the form e p j e p span the subspace TX, condton (14) can be expressed equvalently as (16) (F p j Fp) q (F F q l (e q e q (x) = ) k ) (e p l k j ep) (x) for all, j Sp, k, l S q, p, q P, and x X. The left hand sde of equaton (16) captures the change n the payoff to strategy j S p relatve to strategy S p as agents swtch from strategy k S q to strategy l S q. Ths effect must equal the change n the payoff of l relatve to k as agents swtch from to j, as expressed on the rght hand sde of (16). Evdently, ths condton and ts nterpretaton are akn to those of full externalty symmetry (FES) (see equaton (3) and the subsequent dscusson); however, equaton (16) only refers to relatve payoffs and to feasble changes n the socal state. In the sngle populaton case, condtons (14) and (16) are equvalent to a condton ntroduced by Hofbauer (1985) (also see Hofbauer and Sgmund (1988, Theorem 24.4)). Whle the three condtons are mathematcally equvalent, condton (16) seems to admt the smplest game theoretc nterpretaton. Proposton 4.5. Suppose that F s a C 1 sngle populaton game (p = 1). Then F s a potental game f and only f t satsfes trangular ntegrablty: (17) F (e j e k ) (x) + F j (e k e ) (x) + F k (x) = 0 for all, j, k S and x X. (e e j ) Proof. In the sngle populaton context, condton (16) can be wrtten as (e j e ) DF(x) (e l e k ) = (e l e k ) DF(x) (e j e ) for all, j, k, l S and x X. 16

17 Snce for each fxed j S, the sets {(e e j )} j and {(e j e k )} k j each span TX, ths condton s tself equvalent to the requrement that (e e j ) DF(x) (e j e k ) = (e j e k ) DF(x) (e e j ) for all, j, k S and x X. Expandng the above equalty and collectng terms yelds F (e j e k ) (x) + F j ( e j + e k e + e j ) (x) + Ths condton s clearly dentcal to condton (17). F k (x) = 0 for all, j, k S and x X. (e e j ) The condtons stated above characterze potental games n terms of the dervatves of the payoff functon F, whose doman s the set of socal states X. Can we obtan a smlar characterzaton that only requres dfferentaton of the usual sort that s, dfferentaton of functons defned on the full-dmensonal set R n +? Corollary 4.6, a drect consequence of Theorem 4.3 and the relevant defntons, provdes such a result. Corollary 4.6. Suppose that the populaton game F : X R n s C 1, and let F : R n + R n be any C 1 extenson of F. Then F s a potental game f and only f ΦD F(x)Φ s symmetrc for all x X. 4.3 Uses of Full Potental Functons Despte the prevous results, there are stll stuatons n whch the use of full potental functons as defned by condton (FP) s warranted. For one example, consder populaton games wth entry and ext, n whch the mass m p of agents n populaton p not only can choose among the strateges n S p, but also can select an outsde opton whose payoff s always zero. In such games, the relevant set of socal states s Y = p P Y p, where Y p = {x p R np + : S p xp m p }: that s, the number of actve agents n populaton p vares between 0 and m p, and all remanng agents choose the outsde opton. 5 In ths settng, we call F a potental game f there exsts a functon f : Y R satsfyng condton (FP) on Y. It s not dffcult to show that the basc results from Sandholm (2001) extend to potental games wth entry and ext: the Nash equlbra of F are the states that satsfy the Kuhn-Tucker frst-order condtons for maxmzng f on Y, and evolutonary dynamcs under whch growth rates are postvely correlated wth payoffs converge to connected sets of rest ponts from all ntal condtons. 5 Of course, one could also model such an nteracton as game wth fxed populaton masses m p, augmented strategy sets S p 0 = Sp {0}, and state space Y 0 = p P Y p 0, where Yp 0 = {xp R np +1 + : S p x p = m p }. 0 17

18 The next example shows why condton (FP) s the approprate defnton of potental when entry and ext are allowed. Example 4.7. A congeston model wth (and wthout) entry and ext. Suppose that agents n a sngle unt-mass populaton select ether an actve strategy from the set S = {1,..., n} or the outsde opton, so that the set of populaton states s Y = {x R n + : S x 1}. Each actve strategy S has a C 1 payoff functon F : Y R of the form (18) F (x) = u (x ) + v (x T ), where we wrte x T k S x k. The payoff to strategy s thus the sum of two terms, one dependng on the mass of agents playng strategy, and the other on the total mass of actve agents. By defnton, the payoff to the outsde opton s dentcally zero. For the moment, assume that there s no entry or ext: that s, restrct the populaton state to the set X m = {x R n + : S x = m} Y, where m (0, 1] represents the (fxed) mass of actve players. In ths case, dfferences between the payoffs of actve strateges completely determne ncentves, and so are enough to characterze equlbra and evolutonary dynamcs. These payoff dfferences are precsely what s captured by a potental functon f : X m R satsfyng condton (P): 6 f (x) = Φ F(x) for all x X m. It s easy to verfy that the game defned n equaton (18) satsfes externalty symmetry (ES) throughout the state space Y, and so admts a potental functon on X m n the sense of condton (P). Indeed, such a functon s gven by (19) f (x) = S x 0 u (a) da + x v (x T ) for all x X m. S If agents are free to choose the outsde opton, then understandng ncentves requres us to know not only the relatve payoffs of dfferent actve strateges, but also the absolute payoffs of each strategy, so that they can be compared to the the outsde opton payoff of zero. Ths requres a full potental functon f : Y R, one that satsfes condton (FP): f (x) = F(x) for all x Y. Even f we extend ts doman to Y n the obvous way, the potental functon f from 6 If f were defned throughout Y, the approprate analogue of condton (P) would requre that Φ f (x) = Φ F(x) at all x Y; see Observaton

19 equaton (19) s not a full potental functon for the game F from equaton (18): f x (x) = u (x ) + v (x T ) + x k v k (x T) F (x). k S In fact, a full potental functon for ths game does not exst n general, snce F volates condton (FES): (20) F x j (x) = v (x T) v j (x T) = F j x (x). But F does satsfy condton (FES) under addtonal assumptons. Suppose that v v for all S, so that the payoff to each actve strategy depends on the number of actve players n the same way. Then t s clear from equaton (20) that condton (FES) s satsfed, and n ths case f (x) = S x 0 u (a) da + xt defnes a full potental functon for F. 0 v(a) da Snce condton (FP), unlke condton (P), ensures that potental provdes nformaton about average payoffs, t leaves open the possblty of usng potental functons to nvestgate effcency. Hofbauer and Sgmund (1988, p ) and Sandholm (2001, Secton 5) show that f the full potental functon f of a full potental game F s homogeneous of postve degree, then f s proportonal to aggregate payoffs n F; 7 thus, snce evoluton ncreases potental, t ncreases aggregate payoffs as well. Of course, homogenety s defned n terms of a functon s behavor along rays from the orgn, and so s not meanngful for functons defned only on X. Consequently, even n games wthout outsde optons, the full force of defnton (FP) s needed to prove effcency results. References Akn, E. (1990). The dfferental geometry of populaton genetcs and evolutonary games. In Lessard, S., edtor, Mathematcal and Statstcal Developments of Evolutonary Theory, pages Kluwer, Dordrecht. 7 An example s provded by random matchng n a common nterest game see Example

20 Beckmann, M., McGure, C. B., and Wnsten, C. B. (1956). Transportaton. Yale Unversty Press, New Haven. Studes n the Economcs of Hofbauer, J. (1985). The selecton mutaton equaton. Journal of Mathematcal Bology, 23: Hofbauer, J. and Sgmund, K. (1988). Theory of Evoluton and Dynamcal Systems. Cambrdge Unversty Press, Cambrdge. Krantz, S. G. and Parks, H. R. (1999). The Geometry of Domans n Space. Brkhäuser, Boston. Lang, S. (1997). Undergraduate Analyss. Sprnger, New York, second edton. Monderer, D. and Shapley, L. S. (1996). Potental games. Games and Economc Behavor, 14: Sandholm, W. H. (2001). Potental games wth contnuous player sets. Journal of Economc Theory, 97: Sandholm, W. H. (2008). Decompostons and potentals for normal form games. Unpublshed manuscrpt, Unversty of Wsconsn. 20