Analysis of Coalition Formation and Cooperation Strategies in Mobile Ad hoc Networks

Transcription

1 Aalss of oalo Formao ad ooperao Sraeges Moble Ad hoc ewors Pero Mchard ad Ref Molva Isu Eurecom 9 Roue des rêes Sopha-Apols, Frace Absrac. Ths paper focuses o he formal assessme of he properes of cooperao eforceme mechasms used o deec ad preve selfsh behavor of odes formg a moble ad hoc ewor. I he frs par, we demosrae he requreme for a cooperao eforceme mechasm usg cooperave game heor ha allows us o deerme a lower boud o he sze of coalos of cooperag odes. I he secod par, usg o-cooperave game heor, we compare our cooperao eforceme mechasm ORE o oher popular mechasms. Uder he hpohess of perfec moorg of ode behavor, ORE appears o be equvale o a wde rage of hsor-based sraeges le -for-a. Furher, adopg a more realsc assumpo ag o accou mperfec moorg due o probable commucao errors, he o-cooperave model pus evdece he superor of ORE over oher hsor-based schemes.. Iroduco ooperao eforceme mechasms have bee developed recel he aemp o cope wh he selfsh behavor of odes moble ad hoc ewors MAET. As defed [5] [6], a ode s cosdered selfsh whe does o parcpae he basc ewor operao order o save eerg. As opposed o malcousess, selfshess s a passve hrea ha does o volve a eo o damage he operao of eworg fucos b acve aacs le roue subverso, amperg wh daa, ec. I hs paper we prese wo dffere approaches o assess he feaures of our cooperao eforceme mechasm ORE [6] [7]. Usg ORE, ever ode locall raes s eghbors hrough a moorg mechasm. The observaos colleced b he moorg mechasm are processed o evaluae a repuao value assocaed o each eghbor. The repuao value s used b ORE a sep-le cooperao polc: ol odes wh a repuao ha sasf he requreme of beg greaer ha a defed hreshold are served.e. daa ad roug paces are forwarded, whle odes wh low repuao values are graduall solaed from he ewor. Sce a large fraco of exsg cooperao eforceme schemes are based o prcples a o decso mag ad ecoomc modelg, a aural ool ha emerged o be suable for he valdao of such mechasms s game heor. I he frs par of hs paper we prese a model ha aes o accou boh a ode-cerc ad a ewor-cerc percepo of he eracos bewee odes ha parcpae a MAET b usg cooperave game heor. We frs demosrae he requreme for a cooperao eforceme mechasm order o promoe cooperao bewee self-eresed odes b showg ha he absece of such a mechasm he bes sraeg for a ode would be o free rde. Moreover, we aalze whch would be he sze of a coalo of cooperag odes based o he mporace gve b a ode o he ode-cerc ad ewor-cerc perspecve of he game. We fall sugges how he ORE mechasm could be used o smulae a ode o o he coalo of cooperaors. The beef from usg cooperave GT derves from he abl of hs mehod o seze he damcs of large group of plaers: he sraeg chose b a plaer does o ol deped o a self-eresed percepo of he game bu also aes o accou a group-wde polc of he coalo he plaer belogs o. Alhough he cooperave games approach appears o be approprae o model he damcs of large coalos of odes formg a MAET, he ma lmao of hs mehod s ha s based o a hgh-level represeao of he repuao mechasm ha does o ae o accou he feaures of ORE. To Ths research was parall suppored b he Iformao Soce Techologes program of he Europea ommsso, Fuure ad Emergg Techologes uder he IST MOILEMA proec ad b he Isu Eurecom.

2 overcome hs weaess, we prese he secod par of hs paper a alerave approach based o ocooperave game heor [8] [9]. I hs secod mehod we use a model ha descrbes he sraeg of a selferesed ode ha has o ae he decso wheher o cooperae or o wh a radoml chose eghbor. Uder hs model, he ORE mechasm ca be raslaed o a sraeg profle ha ca be compared o oher popular sraeges. Uder he commol used hpohess of perfec moorg, we demosrae he equvalece bewee ORE ad a wde rage of hsor-based sraeges le -for-a. Furher, b adopg a more realsc assumpo ha aes o accou urelable observaos of odes behavor due o commucao errors, he o-cooperave model pus evdece he superor erms of sabl ad robusess of ORE over oher hsor-based schemes. Alhough he wo mehods descrbed hs paper focus o ORE as a specfc mechasm, some geeral coclusos ca be draw from hs aalss owards he desg of cooperao eforceme mechasms geeral.. Relaed wor Recel, much aeo has bee dedcaed o game heorecal models for MAET geeral ad for cooperao eforceme mechasms parcular ad a creasg umber of models have bee preseed o he commu. I s however ou of he scope of hs paper o propose a exesve sae of he ar of game heorecal models of cooperao MAET, hus we wll focus o some approaches ha we deem relaed o our seg. I a eresg approach preseed [] he auhors propose a game heorecal model whch eergec formao s ae o accou o descrbe he coflcg eraco bewee heerogeeous odes volved a forwardg game,.e. a game whch odes ha belog o a pah from a source o a desao have o collaboravel rela daa paces. The auhors sud he properes of a well ow sraeg geerous--for-a, G-TFT ad demosrae ha uder he eerg cosras mposed o he odes, G-TFT promoes cooperao f ever ode of he ewor coforms o. The model [] provdes a accurae descrpo of he eergec cosra of a ode, whch s he ma reaso for a selfsh behavor, bu provdes ol hgh-level gudeles owards he desg of a cooperao eforceme mechasm based o he G-TFT sraeg. The ma dfferece bewee he wor preseed hs paper ad he research coduced [] s ha our model we ae o accou a more realsc scearo where he observaos made b a ode o her eghbors ca be affeced b errors. The moorg mechasm s deed he e feaure of a cooperao sraeg based o he observao of he oppoe s move such as G-TFT ad we beleve ha a more accurae descrpo of how hese observaos are made s fudameal. Aoher eresg wor owards he defo of a geerc game heorecal framewor o sud cooperao MAET has bee preseed [3]. The auhors propose a model ha aes o accou boh he avalable eerg o a ode ad he raffc geeraed ad/or dreced o ha ode ad helps derve some eresg gudeles owards he defo of a cooperao mechasm. The auhors o ol aalze some exsg cooperao mechasms cludg ORE bu also propose o use he -for-a TFT as a cooperao sraeg. Smlarl o he wor preseed [3], our paper we are able o accurael descrbe o ol our cooperao sraeg ORE bu also a wde-rage of hsor-based cooperao sraeges such as TFT. The performace aalss of he TFT sraeg preseed [3] s exeded our wor ad we prove ha ORE ouperforms all oher sraeges whe he mperfec moorg assumpo s made. I [4] he auhors propose a alerave model for he forwardg behavor of a ode ha s par of a specfc ewor opolog. usg her model, he auhors are able o express he equlbrum forwardg sraeg of a selfsh ode as a fuco of opolog ad roug pah legh formao. The also propose a pushme mechasm ha eforces a cooperave behavor amog selfsh odes. Alhough he resuls obaed [4] provde a ver useful descrpo of he relao bewee roug, ewor opolog ad he cooperave behavor of a ode, he proposed pushme mechasm s lmed o a specfc sace of he cosdered ewor opolog ad does o ae o accou he mperfec moorg of he ode behavor. The research preseed [], [3], [4] ad he secod par of hs paper, s based o o-cooperave game heor: eve whe mulple plaers are cosdered, he sraeg seleco phase s alwas drve b a ode-cerc percepo of he game. As a resul, he cooperao sraeges obaed hrough he proposed models ae o accou ol he paoffs obaed b a sgle plaer. Hece, he frs par of hs paper we propose a alerave approach based o a geeral model usg cooperave game heor as a

3 framewor o sud cooperao as a group ave raher ha a sraeg adoped b sgle plaers. We beleve ha he cooperave games approach provdes a approprae wa of descrbg he damcs of group formao MAET bu eeds furher research order o roduce he model a more formal descrpo of cooperao eforceme mechasm. 3. ooperave games approach I a aemp o expla cooperao ad coalo formao, mos heorecal models use a wo-perod srucure as roduced [, 3]. Plaers mus frs decde wheher or o o o a coalo. I a secod sep, boh he coalo ad he remag ages choose her behavor o-cooperavel. A coalo s sable f o age has a ceve o leave. Smulaos preseed [6], [7], [8] have show ha, alhough here s cooperao, he coalo sze s raher small. I hs paper we sugges a approach based o a preferece srucure as defed b he ER-heor []. Ths heor explas mos of he behavor of ages observed dverse expermes bu devaes lle from he radoal ul cocep. The ul of a age s o solel based o he absolue paoff bu also o he relave paoff compared o he overall paoff o all ages. Gve a cera relave paoff share, he ul s srcl creasg he ow absolue paoff of he age. Gve a fxed absolue paoff, he age s bes off whe recevg us he equal far share. To boh sdes of hs equal share,.e. whe recevg less or more ha he far amou, ul s lower, eve f he absolue paoff does o chage 3. I seco 3., we frs sud a smmerc -ode prsoer s dlemma PD game a o-cooperave seg, whch he ages have ol wo opos avalable cooperae or defec. We aalze ashequlbrum of he o cooperave game whe ages prefereces ca be descrbed b ER,.e. plaers value boh her absolue ad her relave paoff. I parcular, we loo a he umber of ages who pla cooperavel. We show ha o-cooperao s alwas a equlbrum, sce f o oher ode cooperaes a ode would maxmze s absolue paoff ad receve he equal share b choosg o defec. Addoall, however, here ma be ash-equlbrum whch odes cooperae: f, for example, he res of he ages pla cooperavel, a plaer ca ge he equal share b choosg o cooperae as well. Hece, f values s relave paoff beg close o he equal share more ha s absolue paoff, wll choose o complee he grad coalo. learl, paral cooperao ca also occur, whereb some odes cooperae whle ohers defec. For such equlbrum, we show ha he umber of cooperag odes s raher large: sce cooperao leads o a lower absolue paoff, for a ode o choose o cooperae, plag cooperavel mus move closer o he equal share ha defecg would. As we show, hs ca ol be he case f a leas half of he odes cooperae. Ths resul corass wh he sadard resul preseed [6] whch saes ha he coalo sze s raher small. oe, however, ha he prsoer s dlemma, he odes have ol he dscree choce of cooperag or defecg, bu wh respec o he cooperao eforceme problem, he odes of a ad hoc ewor mgh choose her cooperao level 4 couousl. We herefore roduce seco 3.3 a smmerc couous PD-game based o he ER preferece srucure. A eresg fdg of hs aalss s ha ER aloe cao mprove upo he o-cooperave ash-equlbrum wh sadard prefereces whch ol he absolue paoff maers o a ode. As a furher refeme, we propose he cooperave-games approach cossg a combao bewee he ER preferece srucure ad he wo-sage coalo formao mehod []. I coras o he radoal models from he game heor leraure, he ER preferece srucure allows coalos o volve a raher large fraco of plaers. Furhermore, hs model allows for a precse characerzao of codos uder whch eve a grad coalo ca be obaed. Fall, seco 3.5 we propose a dscusso o he relao bewee a coalo formao process ad our cooperao eforceme mechasm ORE, used as a effecve complemear ool o mpose a specfc ER-pe for ever ode parcpag a cooperave seg as a ad hoc ewor. The defo of sabl also mples ha o age was o o he coalo. 3 oe ha such a preferece for equ s self-ceered ol ad s dsc from alrusm. 4 The defo of cooperao level wll be gve seco [3.5]: here s suffce o ow ha cooperao level sads for he fraco of paces daa or roug ha are forwarded b a ode of he ewor plag he cooperao game.

4 3.. The preferece srucure Our aalss reles o a preferece srucure whch plaers, alog wh her ow absolue paoff, are movaed o-moooousl b he relave paoff share he receve,.e. how her sadg compares o ha of ohers. We use he ER model preseed [4] ad ehace wh a complee formao framewor. Le he o-egave paoff o ode be deoed b,,...,, ad he relave share b σ. The ER-ul fuco s defed as follows: α u βr σ where α, β 0 ad u s dffereable, srcl creasg ad cocave, ad r s dffereable, cocave ad has s maxmum σ. Throughou hs paper we assume ha odes dsul from dsadvaageous equal s larger f he ode s beer off ha average,.e. r x r x, x 0,. The pes of odes are characerzed b he relave weghs α,. β umber of plaers ER global ul fuco for plaer α u β r σ ER-pes for plaer α, β 0 Absolue paoff for plaer Absolue ul fuco for plaer u Relave paoff for plaer σ, dffereable, srcl creasg, cocave Relave ul fuco for plaer r σ, dffereable, cocave, maxmum Table. Summarzg able defg he game based o ER heor. σ 3.. The prsoer s dlemma wh a dscree sraeg space I hs seco we sud a smple smmerc -ode prsoer s dlemma where each moble ode ca cooperae, c, or defec, d : hs mples ha he sraeg se avalable o each plaer s dscree ad ol wo acos are allowed. I erms of he ode msbehavor problem, hs meas ha he ode eher correcl execues he ewor fucos or does. Le he oal umber of cooperag odes be deoed b. For a gve, he paoff o a ode s gve b f he ode defecs res o free-rde. If a ode plas cooperavel, mus bear some addoal coss. Is paoff s herefore gve b -. We assume decreasg margal beefs for a ode f he umber of moble odes rses,.e. s creasg ad cocave. Furhermore, he oal cos of cooperao,, creases. I order o geerae he sadard ceve srucure of a PD game, we mae he followg assumpo. Assumpo. PD srucure: - < Assumpo mples ha plag cooperavel reduces he absolue paoff, gve a arbrar umber of c -odes. To mae cooperao more aracve from boh he socal ad he dvdual po of vew, we mae he followg assumpos: Assumpo. Socall desrable : Assumpo 3. Idvduall desrable :. Furhermore, we assume ha paoffs for boh cooperag ad defecg odes are o-egave for all

5 3... The ash equlbrum I he followg seco we aalze he ash equlbrum he oe sho PD game uder he assumpo ha all he odes og a exsg ewor choose smulaeousl. Assume ha odes, asde from ode, pla cooperavel. We wa o sud he codo uder whch ode, whch s o par of he se of cooperag odes, chooses o cooperae; plaer chooses o pla c f ad ol f her ul s hgher ha whe plag d,.e.: α u[ ] β r α u[ ] β r 3 Ths s equvale o ode plag c f ad ol f: α r r δ where β [ ] [ δ ] 4 u u I order o choose c he ode mus be overcompesaed for he loss absolue ga b movg closer o he average ga. The geeral codos for a ash equlbrum of a ER-PD game [] of odes whereb he umber of cooperag odes s * ca be used o sud expresso 4: α δ * β for * odes plag c 5 α δ * for he remag -* odes plag d 6 β odos 5 ad 6 ca be used o evaluae he umber of odes * ha ma possbl cooperae a ash equlbrum. O oe had, as log as δ * < 0, here s o chace of havg a coalo of sze * α because > δ * for all pes ad codo 5 cao hold for a ode. O he oher had, he β codos for a ash equlbrum gve b 5 ad 6 mpl ha f δ * > 0 he here are pes α of odes such ha * odes cooperae ad -* odes free-rde. oe ha for a gve β,..., dsrbuo of ER-pes, δ * > 0 s a ecessar bu o suffce codo o ge a coalo sze of *. For a gve paoff srucure wh δ * > 0, however, here exs ER-pes such ha * s he equlbrum for a coalo sze. I order o fd feasble coalo szes, we mus herefore sud codos uder whch δ s posve. oe ha 4 he deomaor of δ s posve due o assumpo. The sg of he umeraor, however, depeds o he umber of cooperag odes. 0 For 0 he sg of he umeraor s egave, sce r r < r r 0 For - he sg of he umeraor s posve, sce r r > r r

6 Therefore, δ 0 < 0 < δ ad o odes ulaerall cooperae whereas all odes plag c ca α esablsh a equlbrum, provded ha all odes pes are smaller ha δ. β I geeral, here are equlbra where ol a cera umber * of odes cooperae. The crucal po s o fd wheher or o he umeraor s posve. Remember ha we prevousl assumed ha r x r x, x 0,. I s ecessar, order o oba δ > 0, ha a ode choosg d furher devaes from he equal share / ha b plag c,.e.: > 7 I s possble o show ha equal 7 s sasfed for >/ 5. Assumpo ad mpl ha he codo δ > 0 s ecessar bu o suffce o sae ha, for a gve vecor of pes, f a ode plas c a he equlbrum, he a leas half of he odes cooperae. Proposo. For a gve paoff srucure of he PD game wh ER prefereces, here s alwas a equlbrum whch all odes defec. Proposo. Gve Assumpo ad Assumpo, here s a ash equlbrum where a leas / odes cooperae. ased o proposo, f here s a coalo of cooperag odes he s raher large The prsoer s dlemma wh a couous sraeg space I seco 3.., we assumed ha odes ol have a dscree opo as o wheher o cooperae or o. ow, we ur o a prsoer s dlemma game where odes ca couousl choose her cooperao levels. As we wll see, ER aloe cao mprove upo he o-cooperave ash-equlbrum wh sadard prefereces whereb ol he absolue paoff maers. However, roducg more srucure o he game,.e. f odes pla a coalo game seco 3.4, ER ma eld a raher large coalo sze or eve suppor he grad coalo. Le he umber of odes aga be deoed b. We defe he cooperao level q [ 0,] as he fraco of paces boh daa ad roug paces ha ode forwards o s eghborg odes or o he desao ode. Each ode mus choose s cooperao level q,,. ooperao duces some coss q ha are assumed o be creasg ad covex he cooperao level > 0, > 0. ooperao also elds some beef Q erms of ewor coecv ad aggregae cooperao effor made avalable b cooperag odes, where Q deoes he aggregae cooperao level. eefs from cooperao are creasg ad cocave, 0, < 0. The paoff o a ode s herefore deermed b: Q q The ash equlbrum I order o fd he ash equlbrum pos of he game f oe exss s ecessar o def he sraeg q ha correspods o he sgular pos of he global ER ul fuco,.e. fdg he roos of he frs order dervave of he ul fuco ad mae sure ha hose pos are maxmum of he ul fuco. I should be oed ha he assumpos made o he covex of he ul fuco see seco 3. allevae he problem of sudg he border codos he fuco doma. I he res of he paper we refer o he frs order codo whe descrbg he process of fdg he ash equlbrum. q 5 The proof of hs affrmao s gve Appedx.

7 We aalze he ash equlbrum whe odes ac smulaeousl. ode chooses q o maxmze s ul fuco r u σ β α, where: q q q Absolue paoff o plaer σ Relave paoff o plaer oe ha he defo used o express he absolue paoff o ode emphaszes he sraeg space for ode q as compared o he sraeg space avalable o he oher odes of he ewor. choosg q, each ode deermes s ow cooperao coss ad he beefs from cooperao. The choce of q also mpacs he paoff of he remag odes ha ur s fed bac o he ode s ow ul hrough he relave paoff. The frs order codo s herefore gve b: [ ] 0 Q r q Q r u β β α The frs order codo ca rewre as: [ ] 0 q Q r q Q u β α 8 or [ ] 0 q Q r q Q u σ σ β α 9 The sraeg q of ode o a gve cooperao polc for he res of he ewor ca be calculaed from hs frs order codo. Proposo 3. ouous game I he couous PD-game based o ER prefereces, he ash equlbrum s gve b solvg he followg expresso: 0 * * q q. I s smmercal as log as a leas oe ode draws ul from s absolue paoff α>0. 6 Iroducg ER prefereces, herefore, does o crease he cooperao effor chose b he odes whe plag he PD-game wh a couous aco se. I does o eve chage he equlbrum cooperao levels. I coras o he dscree prsoer s dlemma, ER does o add a equlbrum whch here s more cooperao effor. The exsece of equlbrum he PD game ha mmcs cooperave behavor, herefore, ol arses he presece of dscree aco ses. Havg a couous decso varable, ER does o chage he se of equlbrum. The reaso s ha ER does o esablsh a preferece for beg cooperave, bu for beg smlar o oher odes wh respec o he paoff. I hs seco, however, we used he ER heor a classcal o-cooperave seg: le us see how he sraeg seleco of a selfsh ode chage whe roducg more srucure o he game,.e. whe cosderg a cooperave-game seg. 6 The proof of proposo 3 s gve Appedx

8 3.4. oalo formao: he cooperave-game approach As a furher refeme, we ow propose a cooperave-games approach cossg a combao of he ER preferece srucure ad he wo-sage coalo formao mehod as roduced [3]. Le us assume ha all odes are decal wh respec o her paoff fuco.e. he use he same defo of ul fuco. I a frs sage, odes decde wheher or o o o he coalo. he prcple of raoal, each ode s assumed o ow he decsos of he oher odes. The cooperao levels.e. he sraeg ha wll be chose he secod sage deped o wheher he odes ae par he coalo or o. The coalo hereb maxmzes s collecve beefs ad plas agas he odes ha do ae par he coalo, whch smulaeousl maxmze her dvdual ul. We frs sud he case of odes ha have decal ER-pes. We demosrae ha wh he coalo formao game, ER-prefereces ca eforce cooperao ad eve resul he grad coalo. We he loo a he case of heerogeeous ER-pes. sudg he exreme scearo of odes ha are solel eresed eher her absolue paoff or equ, we wll explore he effecs of he exsece of some equ-oreed odes he ewor oalo of decal ER-pes We wll ow solve he coalo formao game bacwards, ha s, for a coalo sze, we frs sud he frs order codos for he choce of he cooperao level sde ad ousde he coalo. The, he secod sep, he equlbrum coalo sze s deermed b a sabl codo. Ths meas ha he equlbrum, mus sasf he codo ha here s o ceve o leave he coalo 7. For sadard prefereces usg ER-prefereces hs resuls he specal case β0, he game heor leraure shows ha he coalo sze s raher small. Usg ER prefereces, however, he umber of odes wh a coalo ca be much hgher equlbrum. Isead of solvg he game geeral, we wll show ha f odes ol value he relave paoff hgh eough,.e. α/β s below a cera boud he eve he grad coalo ca be sable. The frs order codo for odes ousde he coalo S s gve b 0, whereas he cooperao sraeg of odes ha ae par he coalo s chose b maxmzg he ul fuco of a represeave member: deed all odes wh he coalo S selec he same sraeg q s sce he are assumed o be of he same pe. Ths mples ha all members of he coalo have decal absolue paoff S Q qs ad relave paoff σ S S. The frs order codo s gve b: S [ α u βr ] [ Q q ] βr S Q 0 0 σ S σ S α u [ Q q ] βr q Q S 0 S σ S S S σ For odes ha do o belog o he coalo S we ow from seco 3.4 ha f σ < > for S he Q > < q. For he coalo, we oba from 0 ad ha f σ S < > he Q > < q S 8. Q > Q 9, he frs order codos mpl ha for odes wh he coalo Sce 7 The orgal wor roduced [3] saes ha he sabl codo s such ha here s a ceve o eher leave or o he coalo. 8 Assumg ha S > 9 For. σ he r < 0 ad mples Q < q S.

9 σ S ad hus: Q q S. To prove ha sde he coalo σ S, assume o he corar ha σ S > ad ha σ < for some odes ousde he coalo. Iequales 0 ad mpl ha q < Q < Q < q S whch coradcs he assumpo of creasg ad covex cooperao coss. Iequales 0 ad ca be used o show he followg proposo: Proposo 4. oalo game I he smmerc coalo game for decal ER prefereces pe α/β, he grad coalo s sable f α/β s suffcel small,.e. odes are eresed eough beg close o he equal share. oe frs, ha wh he grad coalo, he cooperao level sasfes he codo * * q q depedel of he ER-pes ad odes ha receve he equal share. If ode leaves he coalo, he from he frs order codos we oba:, [ q q ] q q > [ q q ] S S S Le us ow loo a he cooperao levels ha would resul f he ER-pe α/β goes o zero. I hs case, odes ge more ad more eresed geg her equal share, ad her cooperao levels wll coverge: he lm q ~ q S q. However, he lm, equal sll mus hold,.e. q q. I he lm he absolue paoff of a ode leavg he coalo s smaller ha wh he grad coalo, ~ whereas he relave paoff s he same,.e. Q > q ~. Therefore, as log as α/β s small eough, he absolue paoff remas lower ad he ul derved from he relave paoff s also smaller ha he grad coalo. Thus, o ode has a ceve o leave he grad coalo f α/β s small eough oalo of heerogeeous ER-pes Whe odes wh heerogeeous ER-pes are allowed o ae par he coalo S, hose odes ha have he larges α /β wll have he greaes eres o leave he coalo order o oba a larger absolue paoff. We wll ow cocerae o he exreme case whch odes are eher eresed her absolue paoff β 0 or equ α 0. The former are referred o as A-odes, he laer as -odes. I oal, here are a A-odes ad b -odes; a of hese A-odes ad b -odes form he coalo. The cooperao levels are deoed b q as, q bs for odes sde S, q a ad q b for odes ousde he coalo. Le us frs loo a he behavor of -odes. Ousde he coalo, a -odes ca arrve a he equal share b choosg he average cooperao cos level. Thus, qb [ a qas b qbs a a qa ] a b 3 A -ode sde he coalo has o ceve o leave f also receves he equal share: qbs [ a qas b b qb a a qa ] a b 4 I equlbrum, all -odes choose he same cooperao level, q ˆ q q ad receve he equal share: qb a as a a a a [ q q ] 5 A-odes ousde he coalo maxmze her absolue paoff, Q qa. The frs order codo s gve b: Q qa. 6 Wh he coalo, he ul of a represeave A-pe-member s maxmzed b guaraeeg ha he -members ge he equal share,.e. q bs. The frs order codo for choosg q as s gve b: b b bs

10 q bs b q Q as a b qas Q a qas 0 7 qas b qbs cosruco, for a gve a ad b, ever -ode s dffere o beg eher sde or ousde he coalo. For a coalo o be sable, a A-ode mus o have a ceve o leave he coalo. I geeral, for a b here wll be a cera umber of A-odes, a, ha wll o he coalo. We have mulple equlbra. Iequales 3-7 ca be used o fer he followg resuls: Resul 5. The larger he oal umber of equ-oreed odes b, he hgher he ceves for A-odes o o he coalo. Hece, for a gve b, he umber of cooperag A-odes a creases b. Resul 6. The more -odes o he coalo, he smaller he ceve for A-odes o do so. I equlbrum, b ad a are egavel correlaed. Resul 7. The oal cooperao level creases wh he umber of -pes ousde he coalo. A og -ode mproves he paoffs ol f does o drve ou a A-ode. The raoale of resuls 5 ad 6 s he followg: f a A-ode eers he coalo ad he coalo creases s cooperao effors, -odes ousde he coalo crease her cooperao acves as well ad hereb addoall reward he eerg ode. If he umber of such equ-oreed -odes ousde he coalo ges larger, hs exeral reward for og a coalo creases ad, herefore, he equlbrum coalo sze creases. Aalogousl, f - odes o he coalo, fewer odes ousde he coalo reward he eerg A-ode b a crease of her cooperao acves. Hece, he ceves for A-odes o eer he coalo decrease ad he umber of A-odes ha are sde he coalo equlbrum ges smaller. Resul 7 reflecs he fac ha he more odes cooperae, he hgher he effcec gas are ad he closer he aggregae cooperao level s o he effce oe. The mpac of A- ad -odes o he decso of he coalo, however, dffers he followg wa: a og A-ode s eresed he absolue paoff ad, cosequel, he re-opmzg coalo creases s cooperao effor because he posve effec o oe more ode s ow ae o accou. A og -ode, however, s o prmarl eresed he absolue paoff, bu he equal share. Therefore, he coalo wll o crease he oal cooperao level ha much because he -ode refras from devag from he cooperao level of o-cooperag odes. osequel, he effcec gas are larger f a A-ode eers he coalo ha f a -ode os. Therefore, -odes are welcome sde a coalo ol f her eerg does o drve ou a A-ode Dscusso: coalo formao process ad he cooperao eforceme mechasm ORE Self-eresed, auoomous moble odes of a ad hoc ewor ma erac raoall o ga ad share beefs sable emporar coalos: hs s o save coss b coordag acves wh oher odes of he ewor. For hs purpose, each ode deermes he ul of s acos a gve evrome b a dvdual ul fuco. I seco 3. we roduced a more sophscaed model whch o ol self-ceered prefereces are ae o accou o derve he dvdual paoff of a aco bu also relave formao s used order o fd a exeded se of possble equlbrum pos. Resuls obaed wh he proposed model are promsg: a damc ewor formed b odes ha follow he defo of ul gve b he ER heor, depedg o he ode pes, s possble o oba sable coalos of a relavel large sze ad uder cera crcumsaces, eve he grad coalo becomes feasble. ode pes are deermed b he wo parameers α ad β whch represe he e facor of he coalo formao process. We beleve ha he repuao echque mplemeed ORE ca be used as a effecve mechasm o mpose a specfc decal ER pe for ever ode parcpag a cooperave seg as a ad hoc ewor. Ideed, he repuao measure roduced [5] s compla wh he ceve srucure gve b ad. ooperao s made aracve from a dvdual po of vew because he cos of parcpag o he ewor operao s compesaed wh a hgher repuao value, whch s he prerequse for a ode o esablsh a commucao wh oher odes he ewor. O he oher had, whe

11 he umber of cooperag odes creases, he cos for parcpao s compesaed b a more coeced ewor ha ur creases he beef of cooperao. ow, f he wo parameers α ad β are represeed as fucos of he repuao r as defed [5], he s possble o eforce a parcular value o he α/β rao. Specfcall s possble o damcall adus he α/β rao order o be compable wh proposo 4. Thus, eve he grad coalo s sable ad ever ode of he ewor cooperaes bearg he same coss ad geg equal beefs b choosg a far operag po whch o oe devaes from he average cooperao level chose b he coalo. The relao bewee α, β ad r s drecl proporoal: he lower he repuao value meag ha he pas sraeg seleced b he ode has bee o reduce he cooperao level he hgher wll be facor β ad he lower wll be facor α hus reducg he α/β rao, ad vce-versa. The relao bewee he repuao value ad he ER pe of a ode becomes more complcaed f we allow he presece of odes wh dffere ER pes: modelg a ewor ha allows dffere ER pes s eresg whe cosderg moble odes wh dffere capables such as dffere baer power ad dffere compuaoal power. However, order o provde a formal assessme of he effcec of he repuao mechasm proposed ORE s ecessar o evaluae he ode model preseed he prevous secos a damc seg: he repuao value s compued based o he pas sraeges seleced b he odes of he ewor ad have a fluece o hose odes fuure acos. Furhermore a varao o he sraeg seleco phase of a ode has a mpac o he sraeges seleced b eghborg odes: soluos o he damc coalo formao process sll have o be examed. We beleve ha he research we have coduced so far has gve some eresg resuls ad proposes a useful bass o sud he coalo formao process of auoomous self-eresed moble odes b meas of repuao mechasms whch s, o he bes of our owledge, a raher uexplored doma. However, we h ha s possble o express he damc coalo formao process usg a more elega ad smple mehodolog, whch s a e requreme for sudg damc games. The relavel rece leraure o he subec saes ha he models of coalo formao ma be classfed o wo ma caegores: ul-based models, as s largel favored b game heor, ad complemear-based models. Up o ow, mos classc mehods ad proocols for he formao of sable coalos amog raoal ages follow he ul-based approach ad cover wo ma acves whch ma be erleaved: he geerao of coalo srucures, ha s parog or coverg he se of ages o coalos, ad he dsrbuo of gaed beef amog he parcpas o each of he coalos. The fuure research dreco we wll ae s o prove ha repuao mechasms geeral are compla o he so called oalo Formao Algorhm. oalo formao algorhms are hose mechasms ha provde a feasble soluo o a cooperave game coaloal srucure: here are several soluo coceps ad we wll focus o he so called Kerel-oreed soluos [4], [5]. Kerel-oreed coalos are he mos suable for our purpose because he relaed leraure gves precse codos for a coalo formao algorhm o be erel-sable wh a polomal complex, as opposed o oher soluo/algorhms ha are ol of heorecal relevace sce he have expoeal complex. 4. o-cooperave games approach I a alerave approach, we vesgaed o he characerscs of ORE b modelg he eracos bewee he odes of a MAET as a o-cooperave game. I he followg secos, we wll roduce a specfc ad well-ow game he prsoer s dlemma, PD ad expla how ad wh hs model s suable o descrbe he decso mag process ha a moble ode would uderae whe parcpag o he ad hoc ewor operao. Subseque o he defo of he model ha descrbes he eraco bewee decso-maers odes volved he game pla, we wll exed our aalss o a parcular sace of games ha goes uder he ame of repeaed games. Repeaed or eraed games have bee exhausvel reaed he game heorec leraure [8], [9], [0], [], [3], [4], [5] ad eresg resuls cocerg he esablshme of a cooperave behavor wll be preseed. I parcular, we wll focus o he sraeg ha a plaer 0 adops o deerme wheher o cooperae or o a each of he moves he eraed game ad descrbe a mpora sraeg ow as -for-a TFT whch has bee cosdered b a lo of game-heors o be oe of he bes sraeges o ol o promoe cooperao bu also for he evoluo of cooperao a defo of evoluo of cooperao wll be gve he followg 0 I hs paper we wll adop he word plaer ad ode as soms.

12 secos. We wll he descrbe how he ORE cooperao eforceme mechasm ca be raslaed o a sraeg for a plaer ad compare o he TFT sraeg o umercall prove he equvalece bewee ORE ad TFT. furher exedg he game heorec cocep appled o he classcal eraed PD game we wll show how he performaces of TFT ad s dervaes.e. geerous-tft, GTFT degrade as ose s roduced he model. I he followg secos we wll descrbe how he roduco of a ose facor allows graspg he udesrable effecs of usg he promscuous mode operao of a wreless card as a bass for he moorg mechasm he wachdog mechasm ad prove ha he ORE sraeg ouperforms all oher ow sraeges boh for promog cooperao ad for he evoluo of cooperao. The umercal resuls obaed hrough a smulao sofware desged b [3] are smulag he more dffcul as of provdg a formal aalss of he ORE sraeg, whch s par of our fuure wor. 4.. Ssem model I order o descrbe he eraco bewee odes of a MAET ad he decso mag process ha resuls a cooperave or selfsh behavor of he odes we wll use a classcal game roduced b A. Tucer [4, pages 7-8]. I he classcal PD game, wo plaers are boh faced wh a decso o eher cooperae or defec D. The decso s made smulaeousl b he wo plaers wh o owledge of he oher plaer s choce ul he choce s made. If boh cooperae, he receve some beef R. If boh defec he receve a specfc pushme P. However, f oe defecs, ad oe cooperaes, he defecg sraeg receves o pushme T ad he cooperaor a pushme S. The game s ofe expressed he caocal form erms of pa-offs: Plaer Plaer ooperae Defec Plaer ooperae Defec ooperae R,R S,T Plaer ooperae 3,3 -,4 Defec T,S P,P Defec 4,- 0,0 Table. Prsoer s Dlemma paoff marx: a geeral, b example. The PD game s a much suded problem due o s far-reachg applcabl ma domas. I game heor, he prsoer s dlemma ca be vewed as a wo-plaers, o-zero-sum, o-cooperave ad smulaeous move game. I order o have a dlemma he followg expressos mus hold: T > R > P > S S T R > 8 I our model, a MAET formed b odes s cosdered as a -plaer plagroud whch radoml, a wo odes ca mee. We suppose ha ever ode of he ewor has some daa raffc o be se hrough some source roue ha s he resul of he execuo of some roug proocol as a example he DSR proocol. We also suppose ha whe a wo odes mee, a some me perod, he boh eed o sed some daa paces hrough each oher,.e. usg each oher as a rela ode. efore he acual process of sedg a pace, he wo odes have o ae he decso wheher o cooperae or defec. cooperag a ode wll forward oe or more daa pace for he requesg ode, whereas b defecg a ode wll o rela daa paces o behalf of he requesg ode. Isead of cludg a accurae descrpo of eergec coss, opolog formao, possble erferece ad pah formao we wll base our model o some basc ecoomc modelg. As a llusrave ad uve example, le us cosder wo plaers odes wh some leers daa messages o sed. For each leer leavg a plaer, a samp eerg cos for sedg oe daa pace has o be used. Whe a leer s forwarded owards s desao he plaer beef s arbrarl fxed o 5: of course, he beef for a successful commucao should be hgher ha he eergec cos for sedg he leer. So, for example, f wo plaers mee ad boh have a leer o sed, f he decso of a plaer s o cooperae, she wll have o sped wo samps oe for her leer, ad oe for her oppoe s leer ad eveuall receve a beef of 5 f her oppoe cooperaed, leadg o a paoff equals o 3 case he oppoe decded o cooperae ad o a paoff equals o -

13 case he oppoe decded o defec. Ths suao ca be raslaed a paoff marx whch maches he oe llusraed Table of he classcal PD game. Of course, s arguable ha such a smple model ca represe a real MAET, bu we beleve ha he lmaos mposed b our model are greal compesaed b he cosoldaed heorecal resuls avalable he leraure for he prsoer s dlemma. Furhermore, we pla o exed he model order o cope wh a T-plaer smulaeous move game where T < hus ag o accou he cooperave sraeg of odes ha are par of a ere pah from a daa source o her seleced desao. However, he e of he model preseed hs paper ad a furher exesos s he wllgess o commucae assumpo: durg ever pla of he game boh he basc PD ad he eraed PD, as we wll see he ex seco boh plaers egaged he decso mag process cooperae or o are supposed o have some daa paces o be se hrough he oppoe plaer. As we wll see laer hs assumpo s ecessar order o mpleme a pushme mechasm for a o-cooperag ode. 4.. The eraed Prsoer s dlemma The eraed verso of he PD game, ad geeral repeaed games have bee exesvel suded he leraure ad he eresed reader could refer o [0] order o fd a basc e complee roduco o he heor of games, equlbrum coceps ad eraed games. I hs paper we wll o focus o he basc resuls from game heor appled o he PD e.g. ash equlbrum of he oe sho PD game bu we wll roduce some coceps ha wll be used he res of he paper. Oe surprsg feaure of ma oe-sho games.e. games ha are plaed ol oce cludg he PD game, s ha he ash equlbrum s o-cooperave: each plaer would prefer o f defec raher ha o cooperae. However, a more realsc scearo e.g. a MAET a parcular oe sho game ca be plaed more ha oce; fac, a realsc game could eve be a correlaed seres of oe sho games. I such eraed games a aco chose b a plaer earl o ca affec wha oher plaers choose o do laer o: repeaed games ca corporae a pheomea whch we beleve are mpora bu o capured whe resrcg our aeo o sac, oe sho games. I parcular, we ca srve o expla how cooperave behavor ca be esablshed as a resul of raoal behavor. I hs seco we ll dscuss repeaed games whch are fel repeaed. Ths eed o mea ha he game ever eds, however. We wll see ha hs framewor ca be approprae for modelg suaos whch he game eveuall eds wh probabl oe bu he plaers are ucera abou exacl whe he las perod s ad he alwas beleve here s some chace he game wll coue o he ex perod. I he followg subsecos we ll roduce a more formal wa some basc coceps relaed o repeaed games ad fel repeaed games. We wll he show he defo of a sraeg for a plaer ad expla how o verf f a smple sraeg s a equlbrum for a game. A reader who s famlar wh game heor s ved o sp he followg wo secos 4.. ad Repeaed games heor osder a game G whch we ll call he sage game or he cosue game. Le he plaer se be I{,,}. I our prese repeaed-game coex wll be clarfg o refer o a plaer s sage game choces as acos raher ha sraeges. We ll reserve sraeg for choces he repeaed game. So each plaer has a pure-aco space A. The space of aco profles s A X I A. Each plaer has a vo euma-morgeser ul fuco defed over he oucomes of G, g : A R, ha he parcular case of he wo plaers PD game aes he form of a paoff marx as Table. Le G be plaed several mes perhaps a fe umber of mes ad award each plaer a paoff whch s he dscoued sum of he paoffs she go each perod from plag G. The hs sequece of sage games s self a game: a repeaed game. Two saemes are mplc whe we sa ha each perod we re plag he same sage game: a for each plaer he se of acos avalable o her a perod he game G s he same regardless of whch perod s ad regardless of wha acos have ae place he pas ad b he paoffs o he plaers from he sage game a perod deped ol o he aco profle for G whch was plaed ha perod, ad hs sage-game paoff o a plaer for a gve aco profle for G s depede of whch perod s I hs seco we ofe refer o or use ex ad examples avalable o he Jm Ralff oes. Tex ad examples wll o be quoed us for clar of preseao.

14 plaed. Saemes a ad b are sag ha he evrome for our repeaed game s saoar or, aleravel, depede of me ad hsor. Ths does o mea he acos hemselves mus be chose depedel of me or hsor. We ll lm our aeo here o cases whch he sage game s a oe-sho, smulaeous-move game. The we erpre a ad b above as sag ha he paoff marx s he same ever perod. We mae he pcal observable aco or sadard prvae moorg assumpo ha he pla whch occurred each repeo of he sage game s revealed o all he plaers before he ex repeo. Therefore eve f he sage game s oe of mperfec formao as s smulaeous-move games so ha durg he sage game oe of he plaers does ow wha he ohers are dog/have doe ha perod each plaer does lear wha he ohers dd before aoher roud s plaed. Ths allows subseque choces o be codoed o he pas acos of oher plaers. We ll see laer he paper ha f we mae he assumpo of mperfec prvae moorg resuls ca be sgfcal dffere. efore we ca al abou equlbrum sraeges repeaed games, we eed o ge precse abou wha a sraeg a repeaed game s. We ll fd useful whe sudg repeaed games o cosder he semexesve form. Ths s a represeao whch we accep he ormal-form descrpo of he sage game bu sll wa o rea he emporal srucure of he repeaed game. Le he frs perod be labeled 0. The las perod, f oe exss, s perod T, so we have a oal of T perods our game. We allow he case where T,.e. we ca have a fel repeaed game. We ll refer o he aco of he sage game G whch plaer execues perod as a. The aco profle plaed perod s us he -uple of dvduals sage-game acos: a a a,..., 9 We wa o be able o codo he plaers sage-game aco choces laer perods upo acos ae earler b oher plaers. To do hs we eed he cocep of a hsor: a descrpo of all he acos ae up hrough he prevous perod. We defe he hsor a me o be: 0 a, a,..., a h 0 I oher words, he hsor a me specfes whch sage-game aco profle.e., combao of dvdual sage-game acos was plaed each prevous perod. oe ha he specfcao of h cludes wh a specfcao of all prevous hsores h 0, h,, h -. For example, he hsor h s us he cocaeao of h - wh he aco profle a - ;.e. h h - ;a -. The hsor of he ere game s h T a 0,a,, a T. oe also ha he se of all possble hsores h a me s us: A X 0 A, he -fold aresa produc of he space of sage-game aco profles A. To codo our sraeges o pas eves, he, s o mae hem fucos of hsor. So we wre plaer s perod- sage-game sraeg as he fuco s, where a s h s he sage-game aco she would pla perod f he prevous pla had followed he hsor h. A plaer s sage-game aco a perod ad afer a hsor mus be draw from her aco space for ha perod, bu because he game s saoar her sage-game aco space A does o chage wh me. The perod- sage game sraeg profle s s: s s s,..., So far we have bee referrg o sage-game sraeges for a parcular perod. ow we ca wre, usg hese sage-game ees as buldg blocs, a specfcao for a plaer s sraeg for he repeaed game. We wre plaer s sraeg for he repeaed game as:

15 0 T s, s s s,..., 3.e. a T-uple of hsor-coge plaer- sage-game sraeges. Each s aes a hsor h A as s argume. The space S of plaer- repeaed-game sraeges s he se of all such T-uples of plaer- sage game sraeges s : A A. We ca wre a sraeg profle s for he whole repeaed game wo was. We ca wre as he - uple profle of plaers repeaed-game sraeges: s s,..., 4 s as defed 3. Aleravel, we ca wre he repeaed-game sraeg profle s as: 0 T s, s s s,..., 5.e., as a colleco of sage-game sraeg profles, oe for each perod, as defed. Le s see how hs repeaed game s plaed ou oce ever plaer has specfed her repeaed-game sraeg s. I s more covee a hs po o vew hs repeaed-game sraeg profle as expressed 5,.e. as a sequece of T hsor-depede sage-game sraeg profles. Whe he game sars, here s o pas pla, so he hsor h s degeerae: ever plaer execues her a sage-game sraeg from s 0 0 a 3. Ths zero-h perod pla geeraes he hsor h a 0 0, where a a,...,. Ths hsor s he revealed or moored b he plaers hemselves o he plaers so ha he ca codo her perod- pla upo he perod-0 pla. Each plaer he chooses her sage-game sraeg s h. osequel, he sage game he sraeg profle a s h s h,..., s h s plaed. I order o form he updaed hsor hs sage-game sraeg profle s he cocaeaed oo he prevous hsor: h a 0,a. Ths ew hsor s revealed o all he plaers ad he each he choose her perod- sage-game sraeg h T, ad so o. We sa ha h s he pah geeraed b he repeaed-game sraeg profle s. s Le us ow cosder he paoff fuco of he repeaed game. We ca h of he plaers as recevg her sage-game paoffs perod-b-perod. Ther repeaed game paoffs wll be a addvel separable fuco of hese sage-game paoffs. Rgh awa we see a poeal problem: f he game s plaed a fe umber of mes, here s a fe umber of perods ad, hece, of sage-game paoffs o be added up. I order ha he plaers repeaed-game paoffs be well defed we mus esure ha hs fe sum does o blow up o f. We esure he feess of he repeaed-game paoffs b roducg dscoug of fuure paoffs relave o earler paoffs. Such dscoug ca be a expresso of me preferece ad/or ucera abou he legh of he game. We roduce he average dscoued paoff as a coveece whch ormalzes he repeaed-game paoffs o be o he same scale as he sage game paoffs. Ife repeo ca be he e for obag behavor he sage games whch could o be equlbrum behavor f he game were plaed oce or a ow fe umber of mes. For example, defeco ever perod b boh plaers s he uque equlbrum a fe repeo of he PD. Whe repeaed a fe umber of mes, however, cooperao ever perod s a equlbrum f he plaers are suffcel pae. Whe sudg fel repeaed games we are cocered abou a plaer who receves a paoff each of fel ma perods. I order o represe her prefereces over varous fe paoff sreams we wa o meagfull summarze he desrabl of such a sequece of paoffs b a sgle umber. A commo assumpo s ha he plaer was o maxmze a weghed sum of her per-perod paoffs, where she weghs laer perods less ha earler perods. For smplc hs assumpo ofe aes he parcular δ 0,, each weghg form ha he sequece of weghs forms a geomerc progresso: for some fxed See heorem 4 Repeaed Games hadous b J. Ralff [0].

16 facor s δ mes he prevous wegh. δ s called her dscou facor. If each perod plaer receves he 0 paoff u, we could summarze he desrabl of he paoff sream u,,... b he umber: u 0 u δ 6 Such a eremporal preferece srucure has he desrable proper ha he fe sum of he weghed paoffs wll be fe sce he sage-game paoffs are bouded. A plaer would be dffere bewee a paoff of x τ a me ad a paoff of x recevedτ perods laer f: τ τ x δ x 7 s: A useful formula for compug he fe ad fe dscoued sums we wll use laer hs seco T T δ δ δ δ T T 9 whch, parcular, s vald for T. If we adoped he summao 6 as our plaers repeaed-game ul fuco, ad f a plaer receved he same sage-game paoff v ever perod, her dscoued repeaed-game paoff, usg 9, would be v / δ. I s however more covee o rasform he repeaed-game paoffs o be o he same scale as he sage-game paoffs, b mulplg he dscoued paoff sum from 6 b δ. So 0 u we defe he average dscoued value of he paoff sream u,,... b: 0 δ δ u 30 I s ofe covee o compue he average dscoued value of a fe paoff sream erms of a leadg fe sum ad he sum of a ralg fe subsream. For example, sa ha he paoffs v a plaer receves are some cosa paoff v for he frs perods,.e. 0,,,,-, ad hereafer she receves a dffere cosa paoff v each perod,,,. The average dscoued value of hs paoff sream s: τ τ v v v τ τ τ τ v v δ δ δ δ δ v v δ δ δ δ δ 3 δ δ τ 0 τ 0 τ I s possble o see ha he average dscoued value of hs sream of bvalued sage-game paoffs s a covex combao of he wo sage-game paoffs. We ca erae hs procedure order o evaluae he average dscoued value of more complcaed paoff sreams. Aoher useful example s whe a plaer receves v for he frs perods, he receves v ol perod ad receve v ever perod hereafer. The average dscoued value of he sream begg perod dscoued o perod s: δ v δv. Subsug hs for v 3, we fd ha he average dscoued value of hs hree-valued paoff sream s: [ δ v δv ] δ v δ 3

17 We have ow defed all he formalsm eeded o exame he equlbrum of a fel repeaed PD game ad o verf f a predefed sraeg cosues a equlbrum. The varous defos of equlbrum ad he relaed heorems ca be foud [0] ooperao he Repeaed Prsoer s dlemma I he oe-sho PD, he plaers cao avod choosg her doma sraeg Defec see Table. I order o mae he followg aalss smpler, cosder he followg paoff marx: Plaer Plaer ooperae Defec ooperae, -, Defec,- 0,0 Table 3. Modfed PD paoff marx. I s eas o verf ha codos 8 hold. Eve whe hs game s fel repeaed, because he sage game has a uque ash equlbrum, he uque subgame-perfec equlbrum has boh plaers defecg ever perod. However, whe he plaers are suffcel pae s possble o susa cooperao.e. eepg ooperae ever perod as a subgame-perfec equlbrum of he fel repeaed game. Frs we wll see ha such cooperao s a ash equlbrum of he repeaed game. We wll he show ha hs cooperao s a subgame-perfec equlbrum. Whe a fel repeaed game s plaed, each plaer has a repeaed-game sraeg s, whch s a sequece of hsor-depede sage-game sraeges 0 s ;.e. s, s,... s, where each s : A A. The s s s,...,. -uple of dvdual repeaed-game sraeges s he repeaed-game sraeg profle As a fudameal example, le us cosder a parcular sraeg ha a plaer could follow ad whch s suffce o susa cooperao. Ths sraeg s also ow as he speful sraeg. Speful ooperae he frs perod. I laer perods, cooperae f boh plaers have alwas cooperaed. However, f eher plaer has ever defeced, defec for he remader of he game. 0 More precsel ad formall, s possble o wre plaer s repeaed-game sraeg s s, s,... he sequece of hsor-depede sage-game sraeges such ha perod ad afer hsor h,, 0 or h, h s 33 D, oherwse Frs, we wll show ha for suffcel pae plaers he sraeg profle s s,s s a ash equlbrum of he repeaed game. The we wll show ha for he same requred level of paece hese sraeges are also a subgame-perfec equlbrum. ow, f boh plaers coform o he alleged equlbrum prescrpo, he boh pla cooperae a 0. Therefore a, he hsor s h,; so he boh pla cooperae aga. Therefore a, he hsor s h,,,, so he boh pla cooperae aga. Ad so o. The pah of s s he fe sequece of cooperave aco profles,,,,. The repeaed-game paoff o each plaer correspodg o hs pah s rval o calculae: he each receve a paoff of each perod, herefore he average dscoued value of each plaer s paoff sream s. a plaer ga from devag from he repeaed-game sraeg s gve ha plaer s fahfull followg s? Le be he perod whch plaer frs devaes. She receves a paoff of he frs perods 0,,,-. I perod, she plas defec whle her coformg oppoe plaed cooperae,, s as

18 eldg plaer a paoff of ha perod. Ths defeco b plaer ow rggers a ope-loop defec - alwas respose from plaer. Plaer s bes respose o hs ope-loop sraeg s o defec ever perod herself. Thus she receves zero ever perod,,. To calculae he average dscoued value of hs paoff sream o plaer we ca refer o 3, ad subsue v, v, ad v 0. Ths elds plaer s repeaed-game paoff whe she defecs perod he mos advaageous wa o be δ δ. Ths s weal less ha he equlbrum paoff of, for a choce of defeco perod, as log as δ. Thus we have defed wha we mea b suffcel pae: cooperao hs PD game s a ash equlbrum of he repeaed game as log asδ. To verf ha s s a subgame-perfec equlbrum of he repeaed prsoers dlemma s ecessar o chec ha hs sraeg profle s resrco o each subgame s a ash equlbrum of ha subgame. osder a subgame, begg perod τ wh some hsor h τ. Wha s he resrco of s o hs subgame? Deog he resrco b ŝ we have: τ τ ˆ τ ˆ, h, ad h, h ; h ˆ ˆ τ s h s 34 D, oherwse We ca paro he subgames of hs game, each defed b a begg perod τ ad a hsor o wo classes: A hose whch boh plaers chose cooperae all prevous perods,.e. τ τ h,, ad hose whch a defeco b eher plaer has prevousl occurred. For hose subgames class A, he sequece of resrcos ˆ ˆ s h from 34 reduces o he sequece of orgal sage-game sraeges h τ s from 33,.e. for all τ ad h, τ we have: τ h, τ, ad hˆ,, hˆ, s h τ, h s ˆ hˆ D, oherwse D, oherwse 35 τ ecause s s a ash equlbrum sraeg profle of he repeaed game, for each subgame h class A, he resrco ŝ s a ash equlbrum sraeg profle of he subgame whe δ. For a subgame h τ τ τ class, h,. Therefore he resrco ŝ of s specfes sˆ D for all. I oher words, a subgame reached b some plaer havg defeced he pas, each { 0,,... } plaer chooses he ope-loop sraeg defec alwas. Therefore he repeaed-game sraeg profle ŝ plaed such a subgame s a ope-loop sequece of sage-game ash equlbra. From Theorem of [0] we ow ha hs s a ash equlbrum of he repeaed game ad hece of hs subgame. We have show ha for ever subgame he resrco of s o ha subgame s a ash equlbrum of ha subgame for δ. Therefore s s a subgame-perfec equlbrum of he fel repeaed PD whe δ omplex sraeges he Ieraed Prsoer s Dlemma I subseco 4.., we dealed he aalss of a parcular sraeg called speful ha was show o be a equlbrum sraeg boh a ash equlbrum for he whole repeaed game ad a subgame perfec equlbrum for he prsoer s dlemma. Axelrod ad Hamlo [9], [0], [] used a compuer ourame o umercall deec sraeges ha would favor cooperao amog plaers egaged he eraed PD. I a frs roud, 4 more or less sophscaed sraeges ad oe oall radom sraeg compeed agas each oher for he hghes average scores a eraed PD of 00 moves. Uexpecedl, a ver smple sraeg dd ousadgl well:

19 TIT-FOR-TAT ooperae o he frs perod ad he cop our oppoe s las move for all subseque perods Ths sraeg was called T-for-a TFT ad became he fouder of a ever growg amou of successful sraeges. To sud he behavor of sraeges from a umercal po of vew, wo ds of compuao ca be doe. The frs oe s a smple roud rob ourame, whch each sraeg mees all oher sraeges. Is fal score s he he sum o he dscoued sum of all scores doe each cofroao. A he ed, he sraeg s sregh measureme s gve b s rage he ourame. The secod pe of umercal aalss s a smulaed ecologcal evoluo, whch a he begg here s a fxed populao cludg he same qua of each sraeg. A roud rob ourame s made ad he he populao of bad sraeges s decreased whereas good sraeges oba ew elemes. The smulao s repeaed ul he populao has bee sablzed,.e. he populao does o chage amore. A good sraeg s he a sraeg whch sas alve he populao for he loges possble me, ad he bgges possble proporo. Ths d of evaluao quoes he robusess of sraeges. efore he roduco of ORE as a sraeg for he eraed PD, s mpora o deal he compuao mehod for ecologcal evoluo, for example volvg hree sraeges. Suppose ha, all, he populao s composed of hree sraeges A,,. A geerao each sraeg s represeed b a cera umber of plaers: W A usg A, W usg ad W usg. The paoff marx of wo-b-wo meeg bewee A, ad s compued ad s hus ow see Table. V A s he score of A whe mees, ec Le us suppose ha he oal sze of he populao s fxed ad cosa. Le s oe Π : [, [, Π W A W W 36 The compuao of he score dsrbued pos of a plaer usg a fxed sraeg a geerao s he: g A W g W g W A V A A W A V A W A V A W V A W V W V W V A V A A V V V V 37 oe ha because of he subracos he compuao of g cao be smplfed. The oal pos dsrbued o all volved sraeges are: W A g A W g W g 38 The sze of each sub-populao a geerao s fall: W W ΠW A ΠW A g g A ΠW g W All dvso beg rouded o he eares lower eger. 39

20 lasscal resuls o he eraed PD, whch have bee emphaszed b Axelrod [38] show ha o bee good a sraeg has o: o be he frs o defec e reacve Forgve e smple The TFT sraeg whch sasfes all hose crera, has, sce Axelrod s boo, bee cosdered o be oe of he bes sraeges o ol for cooperao bu also for evoluo of cooperao ORE as a complex sraeg for he Ieraed Prsoer s Dlemma I s ow mpora o defe he scope of our aalss. Afer a bref roduco o he heor behd he sud of he eraed PD game, we are focusg o he umercal aalss hrough a smulao sofware [34] of he feaures preseed b some specfc sraeges ha he plaers of he eraed PD should follow order o promoe cooperao. Furhermore, we wa o compare some of he sraeges avalable he game heorec leraure ad ow o be he bes sraeges boh from a cooperao po of vew ad from a evoluoar po of vew wh he sraeg derved from he ORE cooperao eforceme mechasm. We sugges he reader o refer o [6] order o grasp he deals ad he fucog of ORE. Our clam s ha he ORE sraeg ca be cosdered equvale o he TFT sraeg uder cera crcumsaces amel whe he repuao buffer s of sze. Furhermore, we wll show hrough he evoluoar smulao ouled seco 4.3, ha he ORE sraeg ouperforms over all he oher aalzed sraeges whe he assumpo of perfec prvae moorg s replaced b he mperfec prvae moorg assumpo. The ORE sraeg 3 ca be defed as follow: ORE ooperae o he frs move. I each perod, observe he pas oppoe s moves ad buld a vecor b b,..., b,..., b where each eleme equals for a cooperao ad - for a defeco. Evaluae repuao as repuao b. If repuao 0 ooperae else Defec We wa o show ow ha he TFT sraeg represes a parcular case of he ORE sraeg. Ideed, f we se meas ha ol oe observao over he oppoe s pas moves s ae o accou o buld he repuao formao. Ths mples ha f he oppoe cooperaed he las move her repuao wll be posve ad he plaer wll chose oo cooperae. Vce versa, f he las oppoe s move was a defeco, he repuao would be egave ad he respose of he plaer would be o defec. Ths s exacl wha he TFT sraeg mples: cooperae o he frs move ad do wha he oppoe dd he prevous move. I hs paper a aalcal resul sag ha he ORE sraeg s a equlbrum sraeg wll o be preseed as he wor hs dreco s progress: however we beleve ha he aalss wll be faclaed has o he equvalece of he TFT ad he ORE sraeg. I he followg subsecos we prese some resuls obaed hrough evoluoar smulaos usg he eraed PD sofware avalable [34]. The ORE sraeg has bee coded ad added up o he ls of avalable sraeges he sofware. 3 The reader should be formed ha hs paper we cosder a lmed verso of he ORE mechasm whch repuao s evaluaed hrough a smple average over he pas observaos made hrough he wachdog mechasm. A more fahful defo of he ORE sraeg s reserved for our fuure wor.