12 Evolutonary Dynamcs Through the anmal and vegetable kngdoms, nature has scattered the seeds of lfe abroad wth the most profuse and lberal hand; but has been comparatvely sparng n the room and nourshment necessary to rear them. T. R. Malthus Ffteen months after I had begun my systematc enqury, I happened to read for amusement Malthus on Populaton... t at once struck me that... favorable varatons would tend to be preserved, and unfavorable ones to be destroyed. Here, then, I had at last got a theory by whch to work. Charles Darwn Our study of evolutonary dynamcs s bult around the replcator equatons. We begn by defnng the replcator dynamc, dervng t n several dstnct ways, and explorng ts major characterstcs ( 12.1 12.6). The next several sectons make good on our promse to justfy Nash equlbrum n terms of dynamcal systems, as we exhbt the relatonshp between dynamc stablty of evolutonary models, on the one hand, and domnated strateges ( 12.7), Nash equlbra ( 12.8), evolutonarly stable strateges ( 12.9), and connected sets of Nash equlbra, on the other. Many of the results we obtan reman vald n more general settngs (e.g., when the dynamc has an aggregate tendency toward favorng more ft strateges, but not necessarly as strongly as the replcator dynamc). We next turn to asymmetrc evolutonary games ( 12.17), whch have the surprsng property, a property that s extremely mportant from the pont of vew of understandng real-world evolutonary dynamcs, that strctly mxed Nash equlbra are never asymptotcally stable but are often neutrally stable, leadng to genercally stable orbts (the Lotka-Volterra model has orbts, but t s not generc, as small changes n the coeffcents lead to the equlbrum beng ether a stable or an unstable focus). In asymmetrc games, the lmt ponts of dynamc processes are generally Nash equlb- 271
272 Chapter 12 ra, but vrtually nothng stronger than ths can be asserted, ncludng the elmnaton of weakly domnated strateges (Samuelson and Zhang 1992). 12.1 The Orgns of Evolutonary Dynamcs The central actor n an evolutonary system s the replcator, whch s an entty havng some means of makng approxmately accurate copes of tself. The replcator can be a gene, an organsm (defnng accurate copy approprately n the case of sexual reproducton), a strategy n a game, a belef, a technque, a conventon, or a more general nsttutonal or cultural form. A replcator system s a set of replcators n a partcular envronmental settng wth a structured pattern of nteracton among agents. An evolutonary dynamc of a replcator system s a process of change over tme n the frequency dstrbuton of the replcators (and n the nature of the envronment and the structure of nteracton, though we wll not dscuss these here), n whch strateges wth hgher payoffs reproduce faster n some approprate sense. In addton to havng an evolutonary dynamc, evolutonary systems may generate novelty f random errors ( mutatons or perturbatons ) occur n the replcaton process, allowng new replcators to emerge and dffuse nto the populaton f they are relatvely well adapted to the replcator system. The stunnng varety of lfe forms that surround us, as well as the belefs, practces, technques, and behavoral forms that consttute human culture, are the product of evolutonary dynamcs. Evolutonary dynamcs can be appled to a varety of systems, but we consder here only two-player evolutonary games, whch consst of a stage game played by pars of agents n a large populaton, each wred to play some pure strategy. We assume the game s symmetrc ( 10.1), so the players cannot condton ther actons on whether they are player 1 or player 2. In each tme perod, agents are pared, they play the stage game, and the results determne ther rate of replcaton. We generally assume there s random parng, n whch case the payoff to an agent of type playng aganst the populaton s gven by equaton (10.1); note that ths assumes that the populaton s very large, so we treat the probablty of an agent meetng hs own type as equal to fracton p of the populaton that uses the th pure strategy. More generally, we could assume spatal, knshp, or other patterns of assortaton, n whch case the probablty of type meetng
Evolutonary Dynamcs 273 type j depends on factors other than the relatve frequency p j of type j n the populaton. There are varous plausble ways to specfy an evolutonary dynamc. See Fredman (1991) and Hofbauer and Sgmund (1998) for detals. Here we dscuss only replcator dynamcs, whch are qute representatve of evolutonary dynamcs n general. Our frst task s to present a few of the ways a replcator dynamc can arse. 12.2 Strateges as Replcators Consder an evolutonary game where each player follows one of n pure strateges s for D 1; : : : ; n. The play s repeated n perods t D 1; 2; : : :. Let p t be the fracton of players playng s n perod t, and suppose the payoff to s s t D.p t /, where p t D.p1 t ; : : : ; pt n /. We look at a gven tme t, and number the strateges so that 1 t t 2 : : : t n. Suppose n every tme perod dt, each agent wth probablty dt > 0 learns the payoff to another randomly chosen other agent and changes to the other s strategy f he perceves that the other s payoff s hgher. However, nformaton concernng the dfference n the expected payoffs of the two strateges s mperfect, so the larger the dfference n the payoffs, the more lkely the agent s to perceve t, and change. Specfcally, we assume the probablty pj t that an agent usng s wll shft to s j s gven by p t j D ˇ. t j t / for t j > t 0 for t j t where ˇ s suffcently small that p j 1 holds for all ; j. The expected fracton Ep tcdt of the populaton usng s n perod t C dt s then gven by X Ep tcdt D p t dt p t p t j ˇ.t j t / C dt p t j pt ˇ.t t j / D p t C dt pt j DC1 p t j ˇ.t t j / D p t C dt pt ˇ.t N t /; where N t D 1 tpt 1 C : : : C t n pt n s the average return for the whole populaton. If the populaton s large, we can replace Ep tcdt by p tcdt. Subtractng p t from both sdes, dvdng by dt, and takng the lmt as dt! 0, we
274 Chapter 12 get Pp t D ˇpt.t N t /; for D 1; : : : ; n; (12.1) whch s called the replcator dynamc. Because the constant factor ˇ merely changes the rate of adjustment to statonarty but leaves the stablty propertes and trajectores of the dynamcal system unchanged, we often smply assume ˇ D 1 ( 12.5). Several ponts are worth makng concernng the replcator dynamc. Frst, under the replcator dynamc, the frequency of a strategy ncreases exactly when t has above-average payoff. In partcular, ths means that the replcator dynamc s not a best-reply dynamc; that s, agents do not adopt a best reply to the overall frequency dstrbuton of strateges n the prevous perod. Rather, the agents n a replcator system have lmted and localzed knowledge concernng the system as a whole. Some game theorsts call such agents boundedly ratonal, but ths term s very msleadng, because the real ssue s the dstrbuton of nformaton, not the degree of ratonalty. Second, f we add up all the equatons, we get P P Pp t D 0, so f pt D 1 at one pont n tme, ths remans true forever. Moreover, although a partcular replcator can become extnct at t! 1, a replcator that s not represented n the populaton at one pont n tme wll never be represented n the populaton at any later pont n tme. So, replcator dynamcs deal poorly wth nnovaton. A more general system adds a term to the replcator equaton expressng the spontaneous emergence of replcators through mutaton. Thrd, our dervaton assumes that there are no mstakes; that s, players never swtch from a better to a worse strategy. We mght suspect that small probabltes of small errors would make lttle dfference, but I do not know under what condtons ths ntuton s vald. Note that takng expected values allows us to average over the possble behavors of an agent, so that even f there s a postve probablty that a player wll swtch from better to worse, on average the player wll not. The replcator dynamc compels a dynamcal system always to ncrease the frequency of a strategy wth above average payoff. If we do not take expected values, ths property fals. For nstance, f there s a probablty p > 0, no matter how small, that a player wll go from better to worse, and f there are n players, then there s a probablty p n > 0 that all players wll swtch from better to worse. We would have a stochastc dynamc n whch movement over tme probably, but not necessarly, ncreases the
Evolutonary Dynamcs 275 frequency of successful strateges. If there s a sngle stable equlbrum, ths mght not cause much of a problem, but f there are several, such rare accumulatons of error wll nevtably dsplace the dynamcal system from the basn of attracton of one equlbrum to that of another (see chapter 13). It follows that the replcator dynamc, by abstractng from stochastc nfluences on the change n frequency of strateges, s an dealzed verson of how systems of strategc nteracton develop over tme, and s accurate only f the number of players s very large n some approprate sense, compared to the tme nterval of nterest. To model the stochastc dynamc, we use stochastc processes, whch are Markov chans and ther contnuous lmts, dffuson processes. We provde an ntroducton to such dynamcs n chapter 13. It s satsfyng that as the rate of error becomes small, the devaton of the stochastc dynamc from the replcator dynamc becomes arbtrarly small wth arbtrarly hgh probablty (Fredln and Wentzell 1984). But the devl s n the detals. For nstance, as long as the probablty of error s postve, under qute plausble condtons a stochastc system wth a replcator dynamc wll make regular transtons from one asymptotcally stable equlbrum to another, and superor mutant strateges may be drven to extncton wth hgh probablty; see chapter 13, as well as Foster and Young (1990) and Samuelson (1997) for examples and references. 12.3 A Dynamc Hawk-Dove Game There s a desert that can support n raptors. Raptors are born n the evenng and are mature by mornng. There are always at least n raptors alve each mornng. They hunt all day for food, and at the end of the day, the n raptors that reman search for nestng stes (all raptors are female and reproduce by clonng). There are two types of nestng stes: good and bad. On a bad nestng ste, a raptor produces an average of u offsprng per nght, and on a good nestng ste, she produces an average of u C 2 offsprng per nght. However, there are only n=2 good nestng stes, so the raptors par off and ve for the good stes.
276 Chapter 12 There are two varants of raptor: hawk raptors and dove raptors. When a dove raptor meets another dove raptor, they do a lttle dance and wth equal probablty one of them gets the good ste. When a dove raptor meets a hawk raptor, the hawk Hawk Hawk u 1 u 1 u Dove u C 2 Dove u C 2 u u C 1 u C 1 raptor takes the ste wthout a fght. But when two hawk raptors meet, they fght to the pont that the expected number of offsprng for each s one less than f they had settled for a bad nestng ste. Thus the payoff to the two strateges hawk and dove are as shown n the dagram. Let p be the fracton of hawk raptors n the populaton of n raptors. We Assume n s suffcently large that we can consder p to be a contnuous varable, and we also assume that the number of days n the year s suffcently large that we can treat a sngle day as an nfntesmal dt of tme. We can then show that there s a unque equlbrum frequency p of hawk raptors and the system s governed by a replcator dynamc. In tme perod dt, a sngle dove raptor expects to gve brth to f d.p/dt D.u C 1 p/dt lttle dove raptors overnght, and there are n.1 p/ dove raptors nestng n the evenng, so the number of dove raptors n the mornng s n.1 p/.1 C.u C 1 p/dt/ D n.1 p/.1 C f d.p/dt/: Smlarly, the number of hawk raptors n the evenng s np and a sngle hawk raptor expects to gve brth to f h.p/dt D.u C 2.1 p/ p/dt lttle hawk raptors overnght, so there are np.1 C.u C 2.1 p/ p/dt/ D np.1 C f h.p/dt/ hawk raptors n the mornng. Let f.p/ D.1 p/f d.p/ C pf h.p/; so f.p/dt s the total number of raptors born overnght and n.1cf.p/dt/ s the total raptor populaton n the mornng. We then have p.t C dt/ D np.t/.1 C f h.p/dt/ n.1 C f.p/dt/ D p.t/ 1 C f h.p/dt 1 C f.p/dt ;
Evolutonary Dynamcs 277 whch mples p.t C dt/ dt If we now let dt! 0, we get p.t/ fh.p/ f.p/ D p.t/ : 1 C f.p/dt dp dt D p.t/œf h.p/ f.p/ : (12.2) Ths s of course a replcator dynamc, ths tme derved by assumng that agents reproduce genetcally but are selected by ther success n playng a game. Note that p.t/ s constant (that s, the populaton s n equlbrum) when f h.p/ D f.p/, whch means f h.p/ D f d.p/ D f.p/. If we substtute values n equaton (12.2), we get dp D p.1 p/.1 2p/: (12.3) dt Ths equaton has three fxed ponts: p D 0; 1; 1=2. From our dscusson of one-dmensonal dynamcs ( 11.8), we know that a fxed pont s asymptotcally stable f the dervatve of the rght-hand sde s negatve, and s unstable f the dervatve of the rght-hand sde s postve. It s easy to check that the dervatve of p.1 p/.1 2p/ s postve for p D 0; 1 and negatve for p D 1=2. Thus, a populaton of all dove raptors or all hawk raptors s statonary, but the ntroducton of even one raptor of the other type wll drve the populaton toward the heterogeneous asymptotcally stable equlbrum. 12.4 Sexual Reproducton and the Replcator Dynamc Suppose the ftness (that s, the expected number of offsprng) of members of a certan populaton depends on a sngle genetc locus, at whch there are two genes (such creatures, whch ncludes most of the hgher plants and anmals, are called dplod). Suppose there are n alternatve types of genes (called alleles) at ths genetc locus, whch we label g 1 ; : : : ; g n. An ndvdual whose gene par s.g ; g j /, whom we term an j -type, then has ftness w j, whch we nterpret as beng the expected number of offsprng survvng to sexual maturty. We assume w j D w j for all,j. Suppose sexually mature ndvduals are randomly pared off once n each tme perod, and for each par.g ; g j / of genes, g taken from the frst and g j taken from the second member of the par, a number of offsprng of type j are born, of whch w j reach sexual maturty. The parents then de.
278 Chapter 12 THEOREM 12.1 For each D 1; : : : ; n let p.t/ be the frequency of g P n the populaton. Then, ftness of a g allele s gven by w.t/ D n w j p j.t/, the average ftness n the populaton s gven by w.t/ D P n D1 p w.t/, and the followng replcator equatons hold: Pp D p Œw.t/ w.t/ for D 1; : : : ; n: (12.4) PROOF: For any D 1; : : : ; n, let y be the number of alleles of type g, and let y be the total number of alleles, so y D P n y j and p D y =y. Because p j s the probablty that a g allele wll meet a g j allele, the expected number of g genes n the offsprng of a g gene s just P j w j p j, and so the total number of g alleles n the next generaton s y Pj w jp j. Ths gves the dfferental equaton Py D y w j p j : Dfferentatng the dentty ln p D ln y get Pp p D Py y Py j y D w j p j Py j y j p j D ln y wth respect to tme t, we w j p j j;kd1 w jk p j p k ; whch s the replcator dynamc. The followng mportant theorem was dscovered by the famous bologst R. A. Fsher. THEOREM 12.2 Fundamental Theorem of Natural Selecton. The average ftness w.t/ of a populaton ncreases along any trajectory of the replcator dynamc (12.4), and satsfes the equaton Pw D 2 p.w w/ 2 : D1 Note that the rght-hand sde of ths equaton s twce the ftness varance. PROOF: Let W be the n n matrx.w j / and let p.t/ D.p 1.t/; : : : ; p n.t// be the column vector of allele frequences. The ftness of allele s then w D w j p j ;
Evolutonary Dynamcs 279 and the average ftness s w D p w D p w j p j : D1 ; Then, Pw D 2 D 2 ; p j w j Pp D 2 ; p.w w/w D 2 D1 p j w j p.w w/ p.w w/ 2 ; where the last equaton follows from P n D1 p.w w/w D 0. The above model can be extended n a straghtforward manner to a stuaton n whch the parents lve more than one generaton, and the fundamental theorem can be extended to nclude many genetc loc, provded they do not nteract. However, t s a bad mstake to thnk that the fundamental theorem actually holds n the real world (ths s often referred to as the Panglossan fallacy, named after Voltare s Dr. Pangloss, who n Candde declared that all s for the best n ths, the best of all possble worlds ). Genes do nteract, so that the ftness of an allele depends not just on the allele, but on the other alleles n the ndvdual s genetc endowment. Such genes, called epstatc genes, are actually qute common. Moreover, the ftness of populatons may be nterdependent n ways that reduce ftness over tme (see, for nstance, secton 11.4, whch descrbes the Lotka-Volterra predator-prey model). Fnally, stochastc effects gnored n replcator dynamcs can lead to the elmnaton of very ft genes and even populatons. D1 12.5 Propertes of the Replcator System Gven the replcator equaton (12.1), show the followng: a. For 1 < j n, show that d p p D. j /: dt p j p j
280 Chapter 12 b. Suppose that there s an n n matrx A D.a j / such that for each D 1; : : : ; n, D P j a j p j ; that s, a j s the payoff to player when pared wth player j n the stage game. Show that addng a constant to a column of A does not change the replcator equaton and hence does not change the dynamc propertes of the system. Note that ths allows us to set the dagonal of A to consst of zeros, or set the last row of A to consst of zeros, n analyzng the dynamcs of the system. c. How does the column operaton descrbed n the prevous queston affect the Nash equlbra of the stage game? How does t affect the payoffs? A more general form of (12.1) s Pp t D a.p; t/pt.t N t / for D 1; : : : ; n; (12.5) where p D.p ; : : : ; p n /, t and Nt are defned as n (12.1) and a.p; t/ > 0 for all p; t. We wll show that for any trajectory p.t/ of (12.5) there s an ncreasng functon b.t/ > 0 such that q.t/ D p.b.t// s a trajectory of the orgnal replcator equaton (12.1). Thus, multplyng the replcator equatons by a postve functon preserves trajectores and the drecton of tme, alterng only the tme scale. 12.6 The Replcator Dynamc n Two Dmensons Suppose there are two types of agents. When an agent of type meets an agent of type j, hs payoff s j, ; j D 1; 2. Let p be the fracton of type 1 agents n the system. a. Use secton 12.5 to show that we can assume 21 D 22 D 0, and then explan why the replcator dynamc for the system can be wrtten Pp D p.1 p/.a C bp/; (12.6) where a D 12 and b D 11 12. b. Show that n addton to the fxed pont p D 0 and p D 1, there s an nteror fxed pont p of ths dynamcal system (that s, a p such that 0 < p < 1) f and only f 0 < a < b or 0 < a < b. c. Suppose p s an nteror fxed pont of (12.6). Fnd the Jacoban of the system and show that p s an asymptotcally stable equlbrum f and only f b < 0, so 0 < a < b. Show n ths case that both of the other fxed ponts of (12.6) are unstable.