Noisy mean field stochastic games with network applications

Size: px
Start display at page:

Download "Noisy mean field stochastic games with network applications"

Transcription

1 Noisy mea field stochastic games with etwork applicatios Hamidou Tembie LSS, CNRS-Supélec-Uiv. Paris Sud, Frace Pedro Vilaova AMCS, KAUST, Saudi Arabia Mérouae Debbah Chaire Alcatel-Lucet e radio-flexible, Supélec, Frace Abstract We cosider a class of stochastic games with fiite umber of resource states, idividual states ad actios per states. At each stage, a radom set of players iteract. The states ad the actios of all the iteractig players determie together the istataeous payoffs ad the trasitios to the ext states. We study the covergece of the stochastic game with variable set of iteractig players whe the total umber of possible players grow without boud. We show that the optimal payoffs, the mea field equilibrium payoffs are solutio of coupled system of backward-forward equatios. The limitig games are equivalet to discrete time aoymous sequetial games or to differetial populatio games. Usig multidimesioal diffusio process, a geeral mea field covergece to stochastic differetial equatio is give. We illustrate the cotrolled mea field limit i wireless etworks. CONTENTS I Itroductio 2 II Stochastic game with idividual states 4 II-A The settig II-B Policies ad Strategies II-C Differet types of payoffs II-D Computatio of equilibria II-E Q-values II-F Dyamic team problems ad stochastic potetial games III The classic mea field approach 8 III-A Mea Field Iteractio model [] III-B Mea Field Asymptotic of Markov Decisio Evolutioary Games ad Teams model [2]... 9 IV Cotrolled mea field iteractio V Noisy mea field approach VI Covergece to discrete time mea field 3 VI-A Big step-size VI-A Mea field coordiatio games VI-B Vaishig step-size VI-B Cetralized mea field cotrol VII Applicatio to malware propagatio 6 VII-A Homogeeous system VII-A Ucotrolled behaviour VII-A2 Cotrolled behaviour VII-B Heterogeeous system

2 VII-C Optimal strategy for the homogeeous system VII-D Noisy mea field VII-E Cyclig behavior i IEEE 82. CSMA based Cogitive Networks VIII Cocludig remarks 22 Refereces 23 Appedix 23 I. INTRODUCTION Dyamic Game Theory deals with sequetial situatios of several decisio makers (ofte called players) where the objective for each oe of the players may be a fuctio of ot oly its ow preferece ad decisio but also of decisios of other players. Dyamic games allow to model sequetial decisio makig, time-varyig iteractio, ucertaity ad radomess of iteractio by the players. They allow to model situatios i which the parameters defiig the games vary i time ad players ca adapt their strategies (or policies) accordig the evolutio of the eviromet. At ay give time, each player takes a decisio (also called a actio) accordig to some strategy. A strategy of a player is a collectio of history-depedet maps that tell at each time the choice (which ca be probabilistic) of that player. The vector of actios chose by players at a give time (called actio profile) may determie ot oly the payoff for each player at that time; it ca also determie the state evolutio. A particular class of dyamic games widely studied i the literature is the class of stochastic games. Those are dyamic games with probabilistic state trasitios (stochastic state evolutio) cotrolled by oe or more players. The discrete time state evolutio is ofte modeled as iteractive Markov decisio processes while the cotiuous time state evolutio is referred to stochastic differetial games. Discouted stochastic games have bee itroduced i [3]. Stochastic games ad iteractive Markov decisio processes are widely used for modelig sequetial decisio-makig problems that arise i egieerig, computer sciece, operatios research, ad social scieces. However, it is well kow that may real-world problems modeled by stochastic games have huge state ad/or actio spaces, leadig to the well-kow curse of dimesioality that makes solutio of the resultig models itractable. I additio, if the size of the system grows, the umber of parameters: states, actios, trasitios explode expoetially. I this paper we develop a oisy mea field limit for stochastic games with variable umber of iteractig players. Cetralized ad decetralized mea field solutios are obtaied by idetifyig a cosistecy relatioship betwee the idividualstate-mass iteractio such that i the populatio limit each idividual optimally respods to the mass effect ad these idividual strategies also collectively produce the same mass effect presumed iitially. This leads to a coupled system forward/backward equatios. Related work Mea field iteractios with spatially distributed players ad types ca be described as a sequece of dyamic games. Sice the populatio profile ivolves may players for each type or class, a commo approach is to igore idividual players ad to use cotiuous variables to represet the aggregate average of type-locatio-secodary actios. The validity of this method has bee prove oly uder specific time-scalig techiques ad regularity assumptios. The mea field limit is the modeled by state ad locatio-depedet time process. This type of aggregate models, also kow as o-atomic or populatio games, have bee studied by Wardrop (952) i a determiistic ad statioary settig of idetical players. We describe below some recet advaces i mea field cotrol ad games. Lasry & Lios itroduced i recet papers [4], [5], [6] a geeral mathematical modelig approach for highly dimesioal systems of evolutio equatios correspodig to a large umber of players (particles or agets). They exteded the field of such mea-field approaches also to problems i ecoomics, fiace ad game theory. They studied -player stochastic differetial games ad the related problem of the existece of equilibrium poits, ad see also the populatio mass actio iterpretatio i Nash (95).

3 by lettig ted to ifiity they derived the mea-field limit equatio. However, the discrete time Markov games are ot examied i their aalysis. Decetralized mea field stochastic cotrol ad Nash Certaity Equivalece have bee studied i [7], [8] for large populatio stochastic dyamic systems. Ispired by mea field approximatios i statistical mechaics ad liear quadratic differetial games, the authors aalyzed a commo situatio where the dyamics ad rewards of ay give aget are iflueced by certai aggregate of the mass multi-aget behaviors ad established the existece of equilibria ad optimal cotrol strategies. I the ifiite populatio limit, the agets become statistically idepedet uder some assumptios, a pheomeo related to the propagatio of chaos i mathematical physics. The paradigm of mea field dyamics has bee to associate relative growth rate to actios accordig to the expected payoff they achieved, the study the asymptotic trajectories of the state of the system, i.e. the fractio of users that adopt the differet idividual ad actios. Classes of mea field optimal cotrols have bee studied i [7], [9], []. The aspects of heterogeeity of players which play a importat role i heterogeeous systems are ot cosidered i [7], [9], [], [8], []. The authors i [2] have itroduced evolvig stochastic games with a fiite umber players, i which each player i the populatio iteracts with other radomly selected players (the umber of players i iteractio evolves i time). The states ad actios of each player i a iteractio together determie the istataeous payoff for all ivolved players. They also determie the trasitio probabilities to move to the ext state. Each idividual wishes to maximize the total expected payoff over a ifiite horizo. Uder restricted class of Markovia strategies, the radom process cosistig of oe specific player ad the remaiig populatio coverges weakly to a jump process drive by the solutio of a system of differetial equatios. The authors proved that the large populatio asymptotic of the microscopic model is equivalet to a (macroscopic) stochastic populatio game i which a local iteractio is described by a sigle player agaist a evolvig populatio profile. Aother characterizatio of the log ru mea field game is described by usig the so-called differetial populatio games. Here we exted the work i [2] which is limited to determiistic limit uder strog coditios (bouded secod momet, statioary strategies). Our work exteds also [] i which the decisios of the players are ot examied. Our cotributio The cotributio i this paper is threefold. First, from the theoretical poit of view, we preset a ovel mea field approach, which tries to overcome oe of the limitatios of the classical mea field approach, that is, the approximatio of a iheretly stochastic system with a determiistic represetatio (ordiary differetial equatio). We propose a ew approach i which we preserve the mai advatage of the classical mea field, that is, the reductio of the parameters i the aalysis of large systems, but addig a radom or oisy compoet. This ew additio could lead to a more realistic mathematical model of the origial situatio ad may local ad simultaeous trasitios may occurs at the same time makig the secod momet ubouded. I this cotext the work i [], [2], [2] are ot applicable aymore because the secod momet of the umber of object trasitios per time slot may ot vaish whe the umber of objects goes to ifiity. A typical sceario is whe may players do parallel trasitios. The idea of the proposed aalysis is that, if the third order i the Taylor approximatio is bouded, the the oise is ot egligible ad a covergece to a stochastic mea field limit ca be established. Ispired from the work of [3] based o multidimesioal diffusio process, we were able to establish a mea field covergece to o-determiistic differetial equatios ad exted the previous works i mea field iteractio models (with ad without cotrols). This ew mea field limit which is stochastic is called oisy mea field limit ad applied i this work to malware propagatio i opportuistic etworks. Secod, from the malware propagatio modellig poit of view, we exted the model developed o [] i which the types are ot used ad the impact of the cotrol parameters are ot specially studied. This leads to a limitatio i the results obtaied, because differet types of systems could lead for example to slower rates of propagatio. They ca represet, for example, differet operatig systems, differet versios of the operatig systems or patched/upatched versio of the same operatig system. To the best of our kowledge, i most of the related work about malware spreadig i large etworks authors do ot model the heterogeeity of the etwork. The authors i [4] examied cotrolled dissemiatio usig Potryagi maximum priciple. However, the mea field covergece of their fluid model is ot provided. I [5], the authors aalyze spatial mea field model betwee differet locatios but the cotrol framework is ot examied there. We observe that cotrol parameters are importat

4 i the mea field limit sice they give ew isights to ucotrolled mea field framework which may be costraied (eergy limitatio). This helps i cotrollig the proportio of ifected odes. Third, we study the impact cotrol desig i IEEE 82. CSMA-based cogitive radio etworks. We show that the cotrol ca help i stabilizig cyclig cofiguratio uder which the decouplig assumptio may holds (ad the fixed poit approximatio eeds more justificatio). Orgaizatio The rest of the paper is structured as follows. I ext sectio we preset the stochastic game with radom set of iteractig players. The we preset a backgroud o existig mea field models i discrete time. We provide a geeral covergece to mea field which is characterized by a stochastic differetial equatio ad the payoff evolutio are solutio of partial differetial equatios. We characterize the mea field optimality by a coupled system of Bellma-Shapley optimality combied with a discrete time mea field (forward) equatio, Hamilto-Jacobi-Bellma equatio combied a mea field differetial equatio (Kolmogorov forward or Fokker- Plack equatio). We examie the existece of solutios of these systems ad derived the existece of mea field equilibrium. Coectio to differetial populatio games ad aoymous sequetial games are established. Fially, we apply mea field to opportuistic wireless large etworks ad illustrates importat of cotrol desig i IEEE 82. CSMA-based cogitive radio etworks. II. STOCHASTIC GAME WITH INDIVIDUAL STATES I this sectio, we preset the class of stochastic games with idividual states ad radom umber of iteractig players which is closely related to the mea field model that we study here. We cosider a stochastic game i discrete time i which each player has a fiite umber of states. For each idividual state, the player has fiite umber of actios. I cotrast to classical formulatio, here we describe explicitly the idividual state evolutio ad the variability of the set of iteractig players, leadig to a stochastic game with radom umber of iteractig players. A. The settig A stochastic game with idividual states ad radom set of iteractig players is a collectio Γ = (N, S, (X j, Ãj, A j, r j ) j N, q, B ) where N is the set of players. The cardiality of N is. S is a set of eviromet states. A state vector. For every player j, X j is its ow state space. A state has two compoets as follows: the type of the player ad the iteral state. The type is a costat durig the game. The state of player j at time t is deoted by Xj (t) = (θ j, Yj (t)) j is the type. The set of possible states X j = {, 2,..., Θ} Y j is fiite. Y j may iclude other parameters, such as, space locatio, curret directio ad so o. For every player j N, Ã j is the set of actios of that player. A j : S X j 2Ãj is a set-valued map (correspodece) that assigs to each state (s, x j ) S X j the set of actios A j (s, x) that are available to player j. We deote the state-actio space by SXA = { (s, x, a) : (s, x,..., x ) S j N X j, a = (a j ) j N, a j A j (s, x), j N } For every player j N, r j : SXA R is a istat payoff fuctio for player j. q : SXA (S j X j) is a trasitio fuctio where (S j X j) is the space of probability distributios over S j X j. The margial of q is deoted by q j, which represets the trasitio probabilities of state of player j.

5 B is a radom set of players: B N deotes the set of iteractig players at curret time. The payoff fuctio will be deoted by rj B. How the dyamic game evolves? The dyamic game starts at a iitial (s, x ) S j X j ad is played as follows. Time is discrete ad time space is N. A radom set Bt of players are radomly selected for a oe-shot iteractio. The curret state s t, x t, Bt is kow by all the selected players. At each time t N, each player j Bt, chooses a actio a j,t A j (s t, x t ), ad receives a istat payoff r j (s t, x t, a t ), where a t = (a j,t ) j N, ad the game moves to a ew state accordig to a probability distributio give by q(. s t, x t, a t ) (S j X j ). The payoff ad trasitios of o-selected players are zero ad the payoff fuctio r B t j of a selected player j does ot deped o the state-actios of the o-selected players. B. Policies ad Strategies We start by the case where the actios ad the value of the past states are observed, that is, at each time t, the collectio {s, x, B, a, s, x, B, a,..., s t, x t, B t } of states-actios that were visited i the past ad the actios that were chose by all the players is kow at time t. This assumptio is too strog for most of etwork applicatios, ad the assumptio eed to be relaxed for example, the set of active players is ot ecessarily kow by all the players ad the states of the other players are ot kow. Histories Uder complete iformatio of past play, a history of legth t + correspods to the sequece h t = (s, x, B, a, s, x, B, a,..., s t, x t, B t ). Strategies A pure strategy σ j of player j is a mappig that assigs to every fiite history h t a elemet of A j (s t, x t ). A mixed strategy is a probability distributio of the pure strategies. The set of mixed strategies is deoted by Σ j. A behavioral strategy of j is a collectio of mappigs (σ j,t ) t with σ j,t from the space of histories to a probability distributio over A j (s t, x t ) Ãj. Statioary strategies A simple class of strategies is the class of statioary strategies. A strategy profile (σ j ) j N is statioary if j, σ j (h t ) depeds oly o the curret pair (s t, x t ). A statioary strategy of player j ca be idetified with elemet of the product-space of (s,x) S X (A j(s, x)). Every profile σ = (σ j ) j N j Σ j of mixed strategies, together with the iitial state s, x iduces a probability distributio P s,x,σ over the space of ifiite plays SXA. We deote the correspodig expectatio operator by E s,x,σ. C. Differet types of payoffs We examie three classes of payoff fuctios to evaluate the sequeces of payoffs that the players receive i a stochastic game. Let σ be a mixed strategy profile. I the fiite-horizo payoff, a player cosiders the cumulative payoff durig the first T times. For every fiite T N, the fiite-horizo payoff is for player j is ] F j,t (s, x, σ) = E s,x,σ [ T t= r B t j (s t, x t, a t )l {j B t }. () We deote the associate stochastic game by Γ T. I the discouted payoff, a player cosiders the discouted sum of his istataeous payoffs: For differet discout factors β j <, the β j discouted payoff uder the strategy σ for player j is [ + ] F j,β (s, x, σ) = ( β j )E s,x,σ βjr t B t j (s t, x t, a t )l {j B t }. (2) The term ( β j ) is a ormalizatio factor: Sice + t= βt j = β j, by multiplyig by ( β j ) both sides oe gets ( β j ) + t= βt j =. We deote the associated stochastic game by Γ β. t=

6 The limif (or limsup) payoff, i which a player cosiders the limif of his log-ru average payoffs. For player j, the limif payoff is [ T ] t= F j, (s, x, σ) = E s,x,σ lim if rb t j (s t, x t, a t )l {j B t } T + T t = l. (3) {j B t } We will cosider the limif or the limsup whe the the limit of Cesaro-mea payoff does ot exists. We deote the associate stochastic game by Γ. Defiitio. Let ɛ. A strategy profile σ = (σ j ) j N is a T stage ɛ (Nash) equilibrium if (s, x ) S j X j, j N, σ j Σ j (4) F j,t (s, x, σ ) F j,t (s, x, (σ j, σ j)) ɛ, (5) where σ j meas (σ i ) i j ad (σ j, σ j ) deotes the profile (σ,..., σ j, σ j, σ j+,..., σ ). A strategy profile σ = (σ j ) j N is a β discouted ɛ equilibrium if (s, x ) S j X j, j N, σ j Σ j (6) F j,β (s, x, σ ) F j,β (s, x, (σ j, σ j)) ɛ, (7) A strategy profile σ = (σ j ) j N is a lim if ɛ equilibrium if (s, x ) S j X j, j N, σ j Σ j, (8) F j, (s, x, σ ) F j, (s, x, (σ j, σ j)) ɛ, (9) The payoff of a ɛ equilibrium is called ɛ equilibrium payoff. A equilibrium (simply called - Nash - equilibrium) of the stochastic game is a strategy profile such that o player ca profit by a uilateral deviatio. A refied otio of (Nash) equilibrium is subgame perfect equilibrium ad ɛ subgame perfect equilibrium. It ca be see as the requiremet for the credibility of a equilibrium strategy. A strategy profile σ is called a subgame perfect ɛ equilibrium if σ(h t ) is a ɛ equilibrium strategy profile for every fiite history h t. Note that the radom set Bt ca be see as a state compoet. Whe j Bt its actio space is A j (s t, x t ) otherwise his actio space is a set of dummy actios i term of payoffs i.e { B j (s t, x t, Bt Aj (s ) = t, x t ) if j Bt B j otherwise where the actios i B j does ot affect the payoffs. The, uder complete iformatio, the stochastic game with radom umber of iteractig players ca be trasformed i the classical formulatio with (s t, x t, B t ) as the augmeted state of dyamic game. Therefore, the followig result follows. Lemma. For every N, the stochastic games with radom umber of iteractig players Γ T, Γ β have at least oe equilibrium. The stochastic game Γ has a equilibrium uder ergodicity coditios [6] The discouted stochastic game Γ β has at least oe statioary equilibrium. Moreover, Γ β has at least oe subgame perfect ɛ-equilibrium. The first ad secod statemets are classical results i stochastic games with idividual states, see [2] ad the refereces therei. The proof of the last statemet follows from [6].

7 D. Computatio of equilibria Uder discout factors, complete iformatio, fixed set of players durig all the iteractio, we ca write the Belma-Shapley operator to compute the equilibria (the solutios obtaied by dyamic programmig are also subgame perfect equilibria ad the obtaied strategies are Markovia). The operator O = (O j ) j N is give by O j (V )[s, x] = max{( β j )r j (s, x, u) + β j q(s, x s, x, u)v j (s, x )} u j s,x where u = (u,..., u ) is a statioary strategy profile. O j (V ) = (O j (V )[s, x]) s,x. A fixed poit solutio of O gives equilibrium payoff as well as β discouted equilibrum obtaied by collectig the maximizers of ( β j )r j (s, x, a)+β j s,x q(s, x s, x, a)v j (s, x ). Uder the same assumptios, oe ca write similar dyamic programmig priciple (by omittig the factor β j ) for the fiite horizo case (backward iductio). Uder variable set of iteractig players, the dyamic programmig priciple becomes: O j (V )[s, x, B {, u j] = max ( β j )r B u j (s, x, u) j + β j L(s, x, B s, x, B, u)v j (s, x, B, u j) s,x, B where L(s, x, B s, x, B, u) = q(s, x s, x, u)q 2 ( B B ), q 2( B B ) = P(B t = B ) is the probability that B is the set of active players at the curret time. The set B describes all the possible subsets of N (except the empty set). So, oe has 2 elemets i additio to the set S j X j. Naturally, several questios arise o the complexity of computatio of this system. The first questio is how to reduce the complexity o solvig of the dyamic programmig equatios? The secod questio is the dyamic programmig priciple whe we have less iformatio ad relaxed observatio assumptios? These questios are crucial i term of computatio of the solutios ad the applicatio of stochastic games to experimetal iteractive scearios. I ext subsectio, we examie the case where the kowledge of the trasitio probabilities ca be relaxed via Q-learig ad stochastic approximatios. E. Q-values While a optimal strategy ca, i priciple, be obtaied by the methods of dyamic programmig, policy iteratio, ad value iteratio, such computatios are ofte prohibitively time-cosumig. I particular, the size of the state space grows expoetially with the umber of state variables, a pheomeo referred to by Bellma as the curse of dimesioality. Similarly, the size of the actio space ca also lead to computatioal itractability. I the sigle player case, Q-learig [7] perhaps the most well-kow example of reiforcemet learig, is a stochasticapproximatio-based solutio approach to solvig the fixed-poit equatio ( ) Q (s, x, a) = ( β)r(s, x, a) + β s,x q(s, x s, x, a) sup Q (s, x, b) b A(s,x ) Oe ice feature is to lear the Q-value without the kowledge of the trasitio probabilities. The iterative versio is give by Q t+ (s, x, a) = Q t (s, x, a) + λ t (s, x, a) [( β)r(s, x, a)+ ( ) ] β sup Q t (s, x, b) b A(s,x ) Q t (s, x, a) where λ t (s, x, a) is a learig rate fuctio satisfyig t λ t(s, x, a) = +, t λ t(s, x, a) 2 < +, which is a stadard assumptio i stochastic approximatio. I the sigle player case, the Q-learig algorithms give good properties. However, the covergece of the algorithm i the multi-player stochastic game case is still a ope ad challegig issue. I particular, i presece of multiple equilibria the covergece of the Q-values are uclear.

8 F. Dyamic team problems ad stochastic potetial games I the case of actio idepedet trasitios, we show the existece of pure equilibrium if the auxiliary games are potetial games. Below, we preset sufficiet coditios for the existece of pure equilibrium i the game Γ β. Propositio. If i the discouted stochastic game Γ β the payoff fuctio is the same for all the players i.e r j (.) = r(.) the the discouted stochastic game is a team problem or a coordiatio game ad has at least oe pure (Nash) equilibrium. If i the discouted stochastic game Γ β, the trasitios betwee the states are idepedet o the actios played ad for each state (s, x), the fiite game G(s, x) = (N, A j (s, x), r j (s, x,.)) is a potetial game i the sese of Moderer & Shapley (996, [8]), the the discouted stochastic game has at least oe pure (Nash) equilibrium, where r j (s, x,.) = ( β j )r j (s, x,.) + β j s,x q(s, x s, x)v j (s, x ). Proof: The first result follows from the fact that, the discouted stochastic game ca be see as a discouted team problem. By the use of Bellma optimality, we associate a dyamic programmig priciple to the commo objective fuctio r. The, we use the result of Blackwell (962, [9]) which gives the existece of optimal statioary pure strategies. V R S X, V (s, x) = max uj (A j(s,x)){r(s, x, u) + β j s,x q(s, x s, x, u)v (s, x )}. We ow prove the secod result. Sice the players have o ifluece i the trasitio probabilities, each player does its best i the visited states. Each state (s, x), oe has a pure (Nash) equilibrium a s,x of the auxiliary game with r j (s, x, a) + β j s,x q(s, x s, x)v j (s, x ) by usig Moderer & Shapley [8]. The collectio of the pure equilibria a = (a s,x) (s,x) S X is a statioary pure equilibrium of the discouted stochastic game. This completes the proof. For the remaiig we assume that the states of the other players is ot kow. Each player has the its ow private history h j,t = (s, x j,, O, a j,,..., s t, x j,t, O t, a j,t ) where O t is a radom variable idicatig if j is active or ot, O t = l {j B t }. I particular a player does ot observe the past actios players by others ad does ot kow with whom he is curretly iteractig (the subset Bt is ukow by the players). We assume that A j (s, x) = A j (s, x j ), x j := (x j ) j j j X j. III. THE CLASSIC MEAN FIELD APPROACH I this sectio we preset a summary of the previous work o mea field ad the relatio with stochastic games. A. Mea Field Iteractio model [] The mai iterests of [] are models of iteractig objects i discrete time, with fiite umber of states. The objects share local resources, which have a fiite umber of states. The objects are observable oly through their state. I the limit, whe the umber of objects goes to ifiity, it is foud that the system ca be approximated by a determiistic, usually o liear dyamical system called the mea field limit. The mea field is i discrete or cotiuous time, depedig o how the model scales with the umber of objects. If the expected umber of trasitios per object per time slot vaishes whe grows, the the limit is i cotiuous time. Else the limit is i discrete time. Oe of the mai advatages of the mea field approach is that it could drastically reduce the umber of parameters i the aalysis of large systems by usig a aggregate dyamical system descriptio. The first assumptio is that X (t) X is a homogeeous Markov chai. The secod oe is that the trasitio kerel L of X (t) is ivariat uder ay permutatio of the labelig of the objects. That is, L (x,.., x, s; x,.., x, s ) = L (x σ(),.., x σ(), s; x σ(),.., x σ(), s ) = P(X (t+) = x,.., X (t+) = x, S(t+) = s X (t) = x,.., X (t) = x, S(t) = s) for ay permutatio σ of the idex set {, 2,.., }. The, X (t) is called a mea field iteractio model with objects. The occupacy measure is defied as Mi (t) = j= l {Xj (t)=i}. By the ivariace assumptio, (S(t), M (t)) is a homogeeous Markov chai. Its state space is S, where is the simplex {m : i m i =

9 , m i i}. Sice we are iterested i the asymptotic results whe grows to ifiity, we eed to set the proper time scalig. Assume there exists a vaishig δ called itesity (or step-size), which ca be uderstood as the expected umber of objects that do a trasitio i oe time slot. Also defie the drift f (m, s) = E(M (t + ) M (t) M (t) = m, S(t) = s), which is the expected chage i the occupacy measure i oe time slot. The mai scalig assumptios i this model are: H: The resource does ot scale with, that is, exists lim L s;s (m) = L s;s (m), s, s S where L s;s (m) = P(S(t + ) = s M (t) = m, S(t) = s). Moreover, the trasitio matrix L(m) is ergodic. H2: Itesity vaishes at a rate δ, that is, exists δ, such that lim δ = ad f (m, s) lim = f(m, s), s S δ H3: Secod momet of umber of object trasitios per time slot is bouded, which is equivalet to s S, x, x X l { m m 2}(m x m x)(m x m x ) L (dm ; m, s) m δ where L (m ; m, s) := (x L (x ; x, s).,...,x ) j= δ x =m j H4: L s;s (m) is a smooth fuctio of δ ad m. H5: f (m, s) is a smooth fuctio of δ ad m. Later we will see that these assumptios are ideed relaxed i our approach. This theorem ca be used to approximate M (t) by the solutio m(δ t) of a ODE with the same iitial coditio, that is, Theorem. If M () m i probability [resp. i mea square] as the sup τ T M (τ) m(τ) i probability [resp. i mea square], τ [tδ, (t + )δ ], where m(τ) satisfies { ṁ = f(m) m() = m where f(m) = s w s(m)f(m, s) ad w(m) is the ivariat probability of the trasitio matrix L(m). B. Mea Field Asymptotic of Markov Decisio Evolutioary Games ad Teams model [2] The mai iterests of [2] are large populatios of players i which frequet iteractios occur betwee small umbers of chose idividuals. Each iteractio i which a player is ivolved ca be described as oe stage of a dyamic game. The state ad actios of the players at each stage determie a immediate payoff for each player as well as the trasitio probabilities of a cotrolled Markov chai. Each player wishes to maximize or miimize its expected payoff averaged over time. This model with a fiite umber of players, is i geeral difficult to aalyze because of the huge state space required to describe all of the players. The takig the asymptotics as the umber of players grows to ifiity, the whole behavior of the populatio is replaced by a determiistic limit that represets the systems state, which is fractio of the populatio at each idividual state that use a give actio. For large, uder assumptios aalogous to H-H5, the mea field coverges to a determiistic measure that satisfies a o-liear ordiary differetial equatio for uder ay statioary strategy. They show that the mea field iteractio is asymptotically equivalet to a Markov decisio evolutioary game. The mea field asymptotic calculatios for large for give choices of strategies allows to compute the equilibrium of the game i the asymptotic regime. Theorem 2. If M () m i probability as the, for ay statioary strategy u, ad ay time t, M (t) coverges i distributio to the solutio of { ṁ = f(u, m) u is formally defied i the ext sectio. m() = m

10 IV. CONTROLLED MEAN FIELD INTERACTION I this sectio, we itroduce a cotrolled mea field iteractio model. The fiite versio of this model is a particular case of stochastic games with idividual states. We restrict our attetio ito a particular class of behavioral strategies withi we are able to establish the mea field covergece. This restrictio is due to the fact that whe the umber goes to ifiity, the dimesio of the set of statioary strategies goes to ifiity as well as. The mea field approach offers a alterative to direct computatio of the ifiite systems kow as curse of dimesioality. By lettig the size of the system go to ifiity, the discrete stochastic game problem is replaced by a limit of system of Hamilto-Jacobi-Bellma equatio coupled with a mea field limit ODE or coupled system of Bellma-Shapley optimality ad discrete mea field evolutio, that are determiistic ad where the dimesioality of the origial system has bee trasformed i the mass-behavior of the system. Time t N is discrete. The global state of the system at time t is (S(t), X (t)) = (S(t), X (t),..., X (t)). Deote by A (t) = (A (t),..., A (t)) the actio profile at time t. The system (S(t), X (t)) is Markovia oce the actio profile A (t) are draw uder Markovia strategies. We deote the set of Markovia strategies by U. Defie M (t) to be the curret populatio profile i.e Mx (t) = j= l {Xj (t)=x}. At each time t, M (t) is i the fiite set {,, 2,..., } X, ad Mx (t) is the fractio of players who belog to populatio of idividual state x. For a subset X X, defie M [u](t)(x ) := δ {X j [u](t) X }. Similarly, we associate the process Ua (t) = j= l {A j (t)=a} to the fractio of actios. Strategies ad radom set of iteractig players: At time slot t, a ordered list Bt, of players i {, 2,..., }, without repetitio, is selected radomly as follows. First we draw a radom umber of players k t such that P( Bt = k M (t) = m) =: Jk (m) where the distributio J k (m) is give for ay, m {,, 2,..., } X. Secod, we set Bt to a ordered list of k t players draw uiformly at radom amog the ( )...( k t + ) possible oes. Each player such that j Bt takes part i a oe-shot iteractio at time t, as follows. First, each selected player j Bt chooses a actio a j,t A(s, x j ) with probability u(a j s, x j ) where (s, x j ) is the curret player state. The stochastic array u is the strategy profile of the populatio. Deotig the curret set of iteractig players Bt = {j,..., j k }. Give the actios a j,..., a jk draw by the k players, we draw a ew set of idividual states (x j,..., x j k ) ad resource state s with probability L s;s (k, m, a), where a is the vector of the selected actios by the iteractig players. We assume that for ay give Markovia strategy, the trasitio kerel L is ivariat by ay permutatio of the idex of the players withi the same type. This implies i particular that the players are oly distiguishable through their idividual state. Moreover, this meas that the process M (t) is also Markovia oce the sequece of strategy is give. Deote by ws,s (u, m) be the margial trasitio probability betwee the resource states. Give ay Markov strategy ad ay vector m of (X ), the resource state geerates a irreducible Markov decisio process with limitig ivariat measure w s (u, m). The, we ca symplify the aalysis by fixig the resource state S(t) = s without losig geerality. The model is etirely determied by the probability distributios J, the trasitio kerels L ad the strategy profile u. j= V. NOISY MEAN FIELD APPROACH We provide a geeral covergece result of the mea field to a stochastic differetial equatio ad a martigale problem is formulated for the the law of the process M t. We are able to establish a mea field covergece to o-determiistic differetial equatios, thus, extedig the previous works i mea field iteractio [], [5], i mea field Markov decisio teams [2], [2] or i mea field Markov games [2], [9]. We show that eve if the expected umber of players that do a trasitio i oe time slot is ot bouded, oe ca have a mea field limit, but a stochastic oe. This mea field limit is referred to oisy mea field. Before to preset the mai theoretical results of this paper, we first itroduce some prelimiary otios. Let F t = σ(x (t ), A (t ), t t) be the filtratio geerated by the sequece of states ad actios up to t. The evolutio of

11 the system depeds o the decisio of the iteractig players. Give a history h t = (S(), X (), A (),..., S(t) = s, X (t), A (t)). X (t + ) evolves accordig to the trasitio probability L (x ; x, u, s) = P ( X (t + ) = x h t ) The term L (x ; x, u, s) is the trasitio kerel o X uder the strategy U. Let x = (x,..., x ) such that j= δ x = m ad defie j L (m ; m, u, s) = L (x ; x, u, s). The system evolves accordig to the kerel (x,...,x ) j= δ x =m j L (m ; m, u, s) := P(M (t + ) = m M (t) = m, U (t) = u, S(t) = s) = P(M (t + ) = m h t ) where h t = (S(t ), X (t ), A (t ), t t, S(t) = s, X (t) = x ), such that j= δ x = m. The term j L (m ; m, u, s) correspods to the projected kerel of L. Below we make sufficiet coditios o the trasitio kerels to get a weak covergece of the process Mt uder the strategy U (t). Ispired from chapter i [3] based o multidimesioal diffusio process, we are able to establish a mea field covergece to o-determiistic differetial equatios, thus, extedig the previous works i mea field iteractio [], [2], [5], i mea field Markov decisio teams [2], [], [2] or i mea field Markov games [2], [2], [9]. We first preset the mai assumptios of this paper. A: w s (u, m) is cotiuous differetiable i m ad u. A: There exists δ ad cotiuous mappig a : R d U S R d d such that lim sup u U sup m a (m, u, s) δ a(m, u, s) =, s S where a x,x,s(m, u, s) = l m m 2(m x m x )(m x m x )L (dm ; m, u, s), (x, x, s) X 2 S m ad the third momet is fiite. A2: There exists a cotiuous mappig f : R d U S R d such that where A3: For all ɛ > ; A3 : lim sup u U sup m f (m, u, s) δ f(m, u, s) =, s S fx (m, u, s) = l m m R(m x m x )L (dm ; m, u, s), x X, s S m R d sup u U lim sup u U δ sup sup m R d m R d l m m >ɛl (dm ; m, u, s) =, s S [ a (m, u, s) + f ] (m, u, s) <, s S δ δ Give a test fuctio φ defied o R d U, let Lφ(m, u) = f x (m, u) φ(m, u) + ã x,x (m, 2 u) φ(m, u). m x x 2 m x,x x m x where f x (m, u) = s S w s (m, u)f x (m, u, s)

12 ad ã x,x (m, u) = s S w s (m, u)a x,x (m, u, s) Let D φ (t) = m t t L zφ(m z, u z ) dz. D φ (t) is a martigale if ( t2 ) E φ(m t2 ) φ(m t ) L z φ(m z, u z ) dz m t, u t, t t = t With the otio of martigale, the questio of characterizatio of the mea field trajectory ca be formulate as a martigale problem. Oe ca ask if the property that D φ (.) is a martigale for all test fuctios φ uiquely characterizes the mea field m(.) apart from specifyig a iitial m(). The martigale problem for the L z is : The existece of probability measure π defied o R d such that π(m = m) =, ad D φ (.) is a martigale for ay test fuctio. Is there at most oe such π for each m? We ow defie the martigale problem for the iterpolated process from M t deoted by M (.). ( π,m,u ( M () = m) =, ) π,m,u M (t) = M (δ t ) + t δt δ (M (δ (t + )) M (δ t )), t [δ t, δ (t + )] =, t. [ ] π,m,u M (δ (t + )) E Ft = m E L (dm ; M (δ t ), u, s), t. where E is a measurable subset i R d. Theorem 3. Assume A A3. The, for ay test fuctio φ, geerator δ L φ(m, u, s) Lφ(m, u) for ay m, u. Moreover, if the fuctio ã(.,.) ad f(.,.) have the property that for each (m, u) R d U, the martigale problem for ã ad f has exactly oe solutio π m,u startig from m. The π,m,u π m,u as δ uiformly i m for ay strategy u U where π,m,u is the law of iterpolated process from M (t). I additio, if A3 holds, the the martigale problem has a uique solutio. The cosequece of this result is threefold: First, it gives a weak covergece of the mea field M t to a solutio of the stochastic differetial equatio dm t = f(m t, u t )dt + σ(m t, u t )dw t where W t is a Browia motio (the also called Wieer process) oise ad σ σ t = ã. Secod, this result geeralizes the determiistic mea field limit coditios established i [] for U equal to a sigleto. It geeralizes also the determiistic cotrolled mea field dyamics obtaied i [2]. Third, the coditios A A3, A3 are weaker tha those give i []. Uder the coditios i [], the oise term a whe goes the ifiity. Moreover, the cotiuity assumptio o the drift limit is ot eeded. If f admits a uique itegral curve, ad a is bouded ad cotiuous the, the result applies as well as. This allows us to apply it i wide rage of etworkig scearios with discotiuous drift limit but lower semi-cotiuity properties. Note that the uiformity i u may ot be satisfied. I that case, a local mea field solutio is derived. A example of such discotiuity is provided i [2], [2]. Our result exteds also the covergece theorem i [2] which was restricted to statioary strategies. Here u t is a admissible strategy at time t. Usig Itô s formula (see also [22]), the payoff evolutio for a fixed horizo T is the give by v(t, m) = g(m T ), t v(t, m) = r(m, u) + x X By combiig the system, oe gets the mea field optimality: Theorem 4. The mea field optimality for horizo T is give by f x (m, u) v(t, m) + ã x,x (m, 2 u) v(t, m) m x 2 m x m x (x,x ) X 2

13 v(t, m) = g(m T ) t v(t, m) = sup u t U t {r(m t, u t ) + f } x X x (m t, u t ) m x v(t, m t ) + 2 (x,x ) X ã 2 x,x (m t, u t ) 2 m v(t, m x m x t) dm t = f(m t, u t )dt + σ(m t, u t )dw t, t > m = m A. Big step-size VI. CONVERGENCE TO DISCRETE TIME MEAN FIELD I this subsectio, we aalyze the case where the step-size is ot vaishig whe goes to ifiity. I that case, the determiistic mea field limit is i discrete time ad drive by the probability trasitio L t (u, m). Give a iitial populatio profile m ad a termial payoff, the sequece of populatio profile {m t } t is drive by the trasitio probabilities {L t,s,x,s,x (u, m t)} t. m t+ (x) = x X m t (x )L t,x,x (u, m t ). () where L t,x,x (u, m) = k s w s(u, m)l t,s,x,x (u, m; k)j k (m), L t,s,x,x (u, m; k) is the limitig probability trasitio from x to x whe the resource state is s ad the umber of iteractig players is k. Combiig with the Bellma-Shapley optimality criterio, oe gets the followig system i the fiite horizo case: v j,t (s, x, m) = max u j r(s, x, u, m t) + L(s, x s, x, u, m t )v j,t+ (s, x () ) s,x m t+ (x) = x X m t (x )L t,x,x(u, m t ) For the discouted case, oe gets { vj,β (s, x, m) = max uj {( β j )r(s, x, u, m t ) + β j s,x L(s, x s, x, u, m t )v j,β (s, x )} m t+ (x) = x X m t(x )L t,x,x(u, m t ) ad for the average case { hj + v j (s, x, m) = max uj {r(s, x, u, m) + s,x L(s, x s, x, u, m)v j (s, x )} where h j is a costat. m t+ (x) = x X m t(x )L t,x,x(u, m t ) Defiitio 2 (Aoymous sequetial populatio game). To a game as defied i subsectio IV, we associate a macroscopic populatio game, defied as follows. Each member j of the populatio, with state X j (t) ad a populatio profile m t [u]. The iitial coditios of the game are S() = s, X j () = x, m[u]() = m. The populatio profile is solutio to the discrete time evolutio ad S(t), X j (t) evolves as a jump process give by the margial of L which depeds o m t ad u. Further, let F T (s, x, u, u, m) ( be the T stage payoff of player j i this game, give that X j () = T ) x ad m() = m, i.e. F T (s, x, u, u, m) = E r(s(t), X j(t), u, u, m t ) S() = s, X j () = x, m() = m, where m t evolves as i (). The discouted F β ad limitig average payoff F are defied i the same way. Equivalece to aoymous sequetial game of [23] Uder the above assumptios, the discrete time mea field stochastic game are equivalet to a large populatio aoymous sequetial game. The discouted case is a aoymous sequetial populatio game i the sese of [23]: The istataeous payoff of a idividual with strategy v, state x whe the populatio profile (mass-behavior) is m t drive by u is defied by r(s, x, v, u, m t ). The trasitio betwee the states is give by L(s, x s, x, v, u, m t ).

14 If (u, m) satisfies the followig system: m t+ (x ) = x m t(x)l x;x (u t, m t ) t, x, a such that m x u t (a s, x) > = a arg max b ( β)r(s, x, b, m t ) + β s,x v t+(s, x, m t )L(s, x s, x, b, u t, m t ) the we say that (u, m) is a mea field equilibrium. Propositio 2. Uder the above assumptios, the fiite horizo (resp. the discouted) aoymous sequetial populatio game has at least oe mea field equilibrium. Proof: The proof of this result follows from Jovaovic & Rosethal (988). ) Mea field coordiatio games: I this subsectio, we provide sufficiet coditios for mea field covergece i team problems ad global optimal problems. Cosider oe team. A payoff of a player i the team is r(x, a) whe the team actio is a A ad x is the state x X. The payoff fuctio is the same of each member of the team. For a measure m (X ), we defie the expected payoff of the team as (r a (m)) a A. r a (m) = x X r(x, a)m x A history of legth t is ow a sequece of (A (t ), M (t )) t t. Let σ be a behavioral strategy. The payoff defied above becomes [ T ] FT (m, σ) = E m,σ r at (Mt ). (2) t= [ + ] Fβ (m, σ) = ( β)e m,σ β t r at (Mt ). (3) F (m, σ) = E m,σ t= [ lim if T + T ] T r at (Mt ). (4) We keep the idex i the payoff fuctios because the strategy σ takes a sequece of occupacy measure or populatio profile M (t) ad gives A (t), these radom process depeds o. Our aim is to prove the covergece of fiite coordiatio game to a mea field coordiatio game for (i) strategies, (ii) populatio profile, ad (iii) payoffs. Propositio 3. Assume that lim t= sup L (. x, u, m ) L(. x, u, m ) = m (x )> There exists c > such that for all u, x, m, m, oe gets M () m, The, the followig holds L (. x, u, m) L(. x, u, m ) c m m Covergece of payoff fuctios: lim F β (M ()) = F β (m ). Characterizatio of limitig optimal strategies: For all N, there exists > such that BR β (θ, M ()) BR β (θ, m ) where v β (θ, m) = sup{( β)r θ,a (m) + β L (m m, θ, a)vβ (θ, m )} a m

15 BR β (θ, m) = {a, v β (θ, m) = ( β)r θ,a(m) + β m L (m m, θ, a)v β (θ, m )} is the set of actios reachig the maximum ad BR β (θ, m ) = {a, v β (θ, m ) = ( β)r θ,a (m ) + βv β (θ, m, L(. m, θ,.) )} A mea field limit solves m = m, L(. u, m, θ,.) ad u BR(m ) Proof: I appedix. B. Vaishig step-size We defie the re-scaled processes with step-size δ. X j (δ t) = X j (t), j N, M j (δ t) = M (t), t N. M (δ t + t ) = M (t) + t δ (M (t + ) M (t)), t (, δ ) ) Defie the drift as f (u, m, s) = E ( M (δ (t + )) M (δ t) M (δ t) = m, Ã (δ t) u, S(δ t) = s We assume (i) δ f (u, m, s) f(u, m, s) where f is locally Lipschitz, (ii) The first ad secod momets of B are fiite whe +. Uder these assumptios, if the iitial process M () coverges i distributio to m (S j X j) the, the process M coverges i distributio to a determiistic measure (mea field limit) m(t) which is solutio of the ordiary differetial equatio ṁ = f(m). Propositio 4 ([2]). Uder the same assumptios, the process M [u] (which the M uder the strategy u) coverges i distributio to a determiistic measure m[u](t) which is solutio of the ordiary differetial equatio ṁ = f(u, m). Defiitio 3 (Idividual optimizatio framework i populatio game). To a game as defied i subsectio IV, we associate a macroscopic populatio game, defied as follows. Each member j of the populatio, with state X j (t) ad a populatio profile m[u](t). The iitial coditio of the game is S(t) = s, X j () = x, m[u]() = m. The populatio profile is solutio to the discrete time evolutio ad S(t), X j (t) evolves as a jump process give by the margial of q which depeds o m(t), u ad the strategy u of j. Further, let F T (s, x, u, u, m) be the T stage payoff of player j i this game, give that X j () = x ad m() = m, i.e. ( T ) F T (s, x, u, u, m) = E r(s(t), X j (t), u, u, m(t)) S() = s, X j () = x, m() = m where m is solutio of the ODE ṁ(t) = f(u, m(t)). The discouted F β ad limitig average payoff F are defied i the same way. Equivalece to differetial populatio game of [2] Uder the above assumptios, the cotiuous time mea field stochastic game are equivalet to a differetial populatio game. The istataeous payoff of a idividual with strategy v, state x whe the populatio profile (mass-behavior) is m(t) drive by u is defied by r(s, x, v, u, m). The trasitio betwee the state is give by L(s, x s, x, v, u, m). ) Cetralized mea field cotrol: We ow provide the feedback optimality priciple for the global expected mea field payoff r(u, m). The T stage mea field optimizatio problem write subjected to ṁ = f(u, m), m() = m. sup u T r(u(t), m(t)) dt + g(m T )

16 A set of strategy u (t) = φ(t, m) costitutes a optimal mea field solutio if there exist cotiuously differetiable fuctios v(t, m) defied o [, T ] R d R satisfyig the followig Hamilto-Jacobi-Bellma equatio combied with the mea field ODE. = v(t, m) + max t { r(u, m) + mv(t, m), f(u, m) } (5) u = t v(t, m) + r(φ(t, m), m) + mv(t, m), f(φ, m) (6) v(t, m) = g(m T ) (7) ṁ = f(u, m), m() (8) I may situatios, the horizo of the game is ot kow (for example the lifetime of a user i the system is ukow). Oe of the techiques to solve this problem is to cosider the ifiite horizo game. The optimality equatio for discouted payoff sup u + subjected to ṁ = f(u, m) where β > writes: t e β (t t ) r(u(t), m(t)) dt = β v(t, m) + max { r(u, m) + mv(t, m), f(u, m) } u = β v(t, m) + r(φ(t, m), m) + m v(t, m), f(φ, m), (9) v(t, m) = g(m T ) (2) ṁ = f(u, m), m(t ). (2) VII. APPLICATION TO MALWARE PROPAGATION I this sectio, we apply the mea field approach to a cotrolled malware propagatio i opportuistic etworks. The malware propagatio model is based o [] i which the impact of the cotrol parameters is ot examied ad the player types are ot used. The types ca represet, for example, differet operatig systems, differet versios of the operatig systems, or patched/upatched versio of the same operatig system. I most of the related work about malware spreadig i large etworks authors do ot model the heterogeeity of systems which forms the etwork, as far as we kow. This leads to a limitatio i the results obtaied, because differet types of systems could lead for example to slower rates of propagatio. I this example we have mobile odes that ca be ifected by a malicious code. There are two ifected states: passive ad active. No ifected odes are susceptible. The, the set of possible state of a ode is {P, A, S} (for passive, active ad susceptible) ad the set of possible types is {θ, θ 2 }. The state of the system at time t is X (t) = (P (t), P 2 (t), A (t), A 2 (t), S(t)), where P (t) + A(t) + S(t) =, j P j(t) = P (t), j A j(t) = A(t), t ad is the total umber of mobiles i the system. I this example, there is o resource. The occupacy measure is M θ (t) = (P θ(t)/, A θ (t)/, S(t)/) = (P θ (t), A θ (t), S (t)). At every time step we wat to cotrol the proportio of ifected odes, which is I (t) := A (t)+p (t). There are two fudametal ways to get ifected: ) Caused by a system flaw. (e.g: a exploit that could allow arbitrary code executio). 2) Caused by huma flaw. (e.g: the user is deceived ad executes a dagerous piece of code). We ca model this system as a cotrolled mea field iteractio model. The iteractio is simulated usig the followig rules: ) A passive ode may become susceptible (ioculatio) with probability δ P. 2) A passive ode with type θ may opportuistically ecouter aother passive ode of type θ, ad both become active. This occurs with probability proportioal to the frequecy of other passive odes at time t. For type θ, the probability is λ(p θ (t) l {θ=θ }). Note that the passive ode ca decide to cotact the other passive ode or ot, so there are two possible actios: {m, m} (for meet ad ot meet). Those evets will be modeled

17 as a Beroulli radom variable with success (meetig) probability δ m, which represets u(m P, θ). Here we model the possibility of gettig ifected by a system flaw. 3) A active ode may become susceptible (ioculatio) with probability δ A. 4) A active ode of type θ may become passive with probability β P θ (t) h θ+pθ (t) at time t. Here is assumed that, at high cocetratios of passive odes, each active ode ifects some maximum umber of passive oes per time step. This reflects fiite total badwidth. The parameter β has the iterpretatio of the maximum ifectio rate. The parameter h θ is the passive ode desity at which the ifectio proceeds at half of its maximum rate. Here we model the possibility of gettig ifected by a system flaw. 5) A susceptible ode may become active with probability δ S 6) A susceptible ode may become passive via two ways. First, δ Sm is the probability of gettig ifected by a huma flaw. I this case, the susceptible ode ca decide to get deceived or ot, so there are two possible actios: {o, ō}. The statioary strategy i this case will be modeled as a coi toss with probability δ e. Secod, η(pθ (t) + P θ (t)) models the probability of ecouterig a passive ode. I this case, the passive ode ca decide to cotact the susceptible ode or ot, ad it is modeled aalogously to the other two cases. At every time step, oe of the trasitios is radomly selected ad performed. The umber of odes that do a trasitio i oe time slot is always, or 2. I order to cotrol the ifected populatio, each trasitio has a certai payoff cotributio which could be if o ifected ode is ioculated, / if there is a ode which is ioculated ad / if oe ode is ifected. I Table I are the trasitio probabilities, the cotributio to M (t + ) M (t), the set of actios, ad the cotributio to the total payoff. Case Trasitio proba. (θ, θ {, 2}). Mθ (t + ) Mθ (t) Actios Payoff cotrib. Pθ (t)δ P (,, )/ sigleto set / 2 Pθ (t)δmλ(p 2 θ (t) ) ( 2, 2, )/ {m, m} 3 A θ (t)δ A (,, )/ sigleto set / 4 A θ (t)β P θ (t) h θ +P θ (t) (,, )/ sigleto set 5 S (t)δ S (,, )/ sigleto set / 6 S (t)(δ eδ Sm + δ mηp (t)) (,, )/ {o, ō, m, m} / TABLE I PROBABILITIES, EFFECTS, ACTIONS AND PAYOFFS. The itesity, that is, the probability that oe arbitrary object does a trasitio i oe time slot is of the order of /. The drift, that is, the expected chage of M i oe time step, give the curret state of the system is: fθ (m) = E(M θ (t + ) M θ (t) M (t) = m) = p θ δ P 2p θ δmλ 2 pθ a θ β pθ h θ+p θ +s(δ e δ Sm +δ m η(p θ +p θ )) 2p θ δmλ 2 pθ a θ δ A + a θ β pθ h θ+p θ + sδ S p θ δ P + a θ δ A sδ S s(δ e δ Sm +δ m η(p θ +p θ )) where m = (p θ, p θ, a θ, a θ, s). The the limit f(m) is p θ δ P 2λp 2 θ δ2 m a θ β pθ h θ+p θ +s(δ e δ Sm +δ m η(p θ +p θ )) p θ δ P 2λp 2 θ δ m 2 a θ β p θ h θ +p θ +s(δ e δ Sm +δ m η(p θ +p θ )) 2λp 2 θ δ2 m a θ δ A + a θ β pθ h θ+p θ + sδ S 2λp 2 θ δ 2 m a θ δ A + a θ β p θ h θ +p θ + sδ S (p θ +p θ )δ P +(a θ +a θ )δ A 2sδ S 2s(δ e δ Sm +δ m η(p θ +p θ ))

18 A. Homogeeous system We briefly metio the homogeeous mea field. The drift is obtaied by computig the expected chages i oe time slot: f (m) = E(M (t + ) M (t) M (t) = m) = pδ P 2pδ mλ 2 p aβ p h+p + s(δ eδ Sm + δ m ηp) 2pδ mλ 2 p aδ A + aβ p h+p + sδ S pδ P + aδ A sδ S s(δ e δ Sm + δ m ηp) where m = (p, a, s). The the limit is f(m) = pδ P 2p 2 δ 2 mλ aβ p h+p + s(δ eδ Sm + δ m ηp) 2p 2 δ 2 mλ aδ A + aβ p h+p + sδ S pδ P + aδ A sδ S s(δ e δ Sm + δ m ηp) I all the simulatios, we kept this parameters uchaged: β = 2, δ A = 5 3 ad δ P = δ S = δ Sm = 4. O parameter h (the passive ode desity at which the ifectio of active odes proceeds at half of it maximum value) depeds the stability of the system. Here we set h = 2 i order to obtai a ustable behaviour. Regardig the cotrol parameters, δ m =, δ e = meas o cotrol. We ivestigate the evolutio of the system i the followig scearios: Trajectory of oe ru of the simulatio (figure ), Mea trajectory of multiple simulatios (figure 2), Mea field limit trajectory (figure 3), Trajectory of the payoff fuctio, Cotrolled mea field limit (figure 4), Trajectory of the heterogeeous malware propagatio (figure 6), Optimal cotrol uder the mea field limit (figure 7), Noisy mea field (stochastic path), (figure 8). These cofiguratios are aalyzed with cotrol ad without cotrol parameters. We observe that the time mea T T m(s)ds is covergig to the statioary poit iside the limit cycle. ) Ucotrolled behaviour: I figures,2,3 we ca see the simulatio results, usig the well-kow algorithm for exact simulatio of a discrete time Markov chai. The iitial cofiguratio is (.2,,.8). A oscillatig behaviour of the total reward ca be see Simulatio path, Particles = Trajectory Startig poit Time mea of trajectory Active Total reward Total Reward Time mea Passive Fig.. Left: Path of oe simulatio without cotrol. Right: Total reward. 2) Cotrolled behaviour: I order to illustrate the cotrol parameters, suppose we wat to keep the proportio of ifected odes below.9 for all times. Oe simple way to achieve this is to reduce the cotact tedecy of a passive ode. The, we set δ m =.75, which is the more relevat cotrol parameter. The results ca be see i figures 4 ad 5.

19 Mea simulatio trajectories, Replicatios =, Particles = Mea of trajectories Startig poit Time mea of trajectories.9.8 Total reward, Replicatios =, Particles = Active Total reward Total Reward Time mea Passive Fig. 2. Leftmost: Mea of multiple simulatio trajectories without cotrol. Rightmost: Total reward Mea field Mea Field Startig poit Time mea of trajectory.9.8 Active Total reward Total Reward Time mea Passive Fig. 3. Left: Mea field with o cotrol. Right: Total reward. B. Heterogeeous system We ivestigate umerically the behavior of the mea field limit for two types θ ad θ. I figure 6 we ca see that it is possible to stabilize the homogeeous system usig classes. C. Optimal strategy for the homogeeous system Sice the payoff fuctio is the same for all the players i.e. r j ( ) = r( ) the discouted stochastic game with commo payoff ca be trasformed i a team problem. Moreover, the set of actios is the same for all the players. I figure 7, we plot the optimal strategy obtaied by solvig the system i the fiite horizo case (equatio ()). The existece of a domiat strategy ca be observed i the plot. D. Noisy mea field I order to give some feelig o how the oisy mea field evolves with time, we show i figure 8 two differet realizatios, for = ad = 2. The variace of the oise is obtaied from σ = δ f f which its orm sup is bouded by 2Dλ, the we have d m = f( m)dt + σ dw t Note that the smooth versio of the last equatio ca be ṁ = f (m ) δ. It is worth to metio that, i this case, the simulatio algorithm is oly exact whe the umerical time step vaishes. I figure 9 we compare the classic mea field versus the mea trajectory of the oisy mea field. E. Cyclig behavior i IEEE 82. CSMA based Cogitive Networks The backoff process i IEEE 82. is govered by a Markovia decisio process if the duratio of per-stage backoff is take ito accout: every ode i backoff state k attempts trasmissio with probability u k for every time-slot;

20 .9.8 Mea field Mea Field Startig poit Time mea of trajectory Total Reward Time mea Active Susceptible Active 6 4 Total reward Passive Passive Fig. 4. Leftmost: Mea field with cotrolled passive ode cotact tedecy. Rightmost: Total reward Susceptible Active prob.meet = Passive Fig. 5. Evolutio of the limit poit of the time mea, for δ m =,.25,..,.925,. if it succeeds, k chages to ; otherwise, k chages to (k + ) mod (K + ) where K is the idex of the maximum backoff state. The stadard Markov chai models, which have bee widely used i IEEE 82., very ofte lead to excessive complicatios. Recetly a mea field approach have bee proposed i []. The authors i [] have poit out that the validity of the Decouplig assumptio (odes idepedece at the asymptotic) should be justified, ot just by a simple fixed poit method but a deep study of the stability of the ordiary differetial equatio is eeded. This is because the existece ad uiqueess of a rest poit does ot implies that the dyamics coverges to this fixed poit. I this sectio we illustrate this statemet via a cycle limit behavior i a heterogeeous CSMA system uder cotrol parameters. To costruct a cyclig behavior of the mea field limit we follow the work of [2]. We.9.8 h =.5, h 2 = h =.9, h 2 =. Mea Field Startig poit Active Active Passive Passive Fig. 6. Left: Ustable mea field, for h =.5, h 2 =.5. Right: Stable mea field h =.9, h 2 =..

21 Fig. 7. Optimal strategy usig backward iductio for the homogeeous system. Noisy mea field, Particles = Noisy mea field, Particles = Active.5 Active Passive Passive Fig. 8. Left: Path of oe simulatio of the oisy mea field, for =, Right: = 2. show that the mea field limit depeds o the cotrol parameters ad the ucotrolled mea field limit system ca be suboptimal depedig o the performace metric. Hece, the cotrol parameters give ew isights ad help i uderstadig the behavior of the mea field limit. We cosider cogitive etwork with two classes of users: primary users ad secodary users (the classes are based o the differetiatio of cotetio widow or arbitratio iterframe space as doe i [2]). The primary users have high priority i the sese that they have reserved time slots for them but also they ca attempt a trasmissio i the other time slots (the commo time slots). Both class of the users ca access to the chael durig commo slots. The backoff state of the secodary users are suspeded durig the reserved time slots. Deote by the total size of the populatio of users ad by θ the size of the subpopulatio θ. Hece, θ θ =. The set of users i Mea field Mea of oisy trajectories, Replicatios =, Particles = Active.5 Active Passive Passive Fig. 9. Left: Classic mea field, Right: Mea of oisy trajectories.

Modified Line Search Method for Global Optimization

Modified Line Search Method for Global Optimization Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

More information

Properties of MLE: consistency, asymptotic normality. Fisher information.

Properties of MLE: consistency, asymptotic normality. Fisher information. Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

Research Article Sign Data Derivative Recovery

Research Article Sign Data Derivative Recovery Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

Class Meeting # 16: The Fourier Transform on R n

Class Meeting # 16: The Fourier Transform on R n MATH 18.152 COUSE NOTES - CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,

More information

THE ABRACADABRA PROBLEM

THE ABRACADABRA PROBLEM THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

Overview of some probability distributions.

Overview of some probability distributions. Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

More information

Sequences and Series

Sequences and Series CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

More information

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13 EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may

More information

*The most important feature of MRP as compared with ordinary inventory control analysis is its time phasing feature.

*The most important feature of MRP as compared with ordinary inventory control analysis is its time phasing feature. Itegrated Productio ad Ivetory Cotrol System MRP ad MRP II Framework of Maufacturig System Ivetory cotrol, productio schedulig, capacity plaig ad fiacial ad busiess decisios i a productio system are iterrelated.

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process A example of o-queched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which

More information

Analyzing Longitudinal Data from Complex Surveys Using SUDAAN

Analyzing Longitudinal Data from Complex Surveys Using SUDAAN Aalyzig Logitudial Data from Complex Surveys Usig SUDAAN Darryl Creel Statistics ad Epidemiology, RTI Iteratioal, 312 Trotter Farm Drive, Rockville, MD, 20850 Abstract SUDAAN: Software for the Statistical

More information

The analysis of the Cournot oligopoly model considering the subjective motive in the strategy selection

The analysis of the Cournot oligopoly model considering the subjective motive in the strategy selection The aalysis of the Courot oligopoly model cosiderig the subjective motive i the strategy selectio Shigehito Furuyama Teruhisa Nakai Departmet of Systems Maagemet Egieerig Faculty of Egieerig Kasai Uiversity

More information

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

Department of Computer Science, University of Otago

Department of Computer Science, University of Otago Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

LECTURE 13: Cross-validation

LECTURE 13: Cross-validation LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M

More information

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee

More information

5 Boolean Decision Trees (February 11)

5 Boolean Decision Trees (February 11) 5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

More information

Entropy of bi-capacities

Entropy of bi-capacities Etropy of bi-capacities Iva Kojadiovic LINA CNRS FRE 2729 Site école polytechique de l uiv. de Nates Rue Christia Pauc 44306 Nates, Frace iva.kojadiovic@uiv-ates.fr Jea-Luc Marichal Applied Mathematics

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

Asymptotic Growth of Functions

Asymptotic Growth of Functions CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

More information

Convexity, Inequalities, and Norms

Convexity, Inequalities, and Norms Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

More information

MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this

More information

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad

More information

Normal Distribution.

Normal Distribution. Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

More information

Hypergeometric Distributions

Hypergeometric Distributions 7.4 Hypergeometric Distributios Whe choosig the startig lie-up for a game, a coach obviously has to choose a differet player for each positio. Similarly, whe a uio elects delegates for a covetio or you

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

Confidence Intervals for One Mean

Confidence Intervals for One Mean Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

Soving Recurrence Relations

Soving Recurrence Relations Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

More information

Universal coding for classes of sources

Universal coding for classes of sources Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric

More information

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009) 18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

More information

Determining the sample size

Determining the sample size Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors

More information

Groups of diverse problem solvers can outperform groups of high-ability problem solvers

Groups of diverse problem solvers can outperform groups of high-ability problem solvers Groups of diverse problem solvers ca outperform groups of high-ability problem solvers Lu Hog ad Scott E. Page Michiga Busiess School ad Complex Systems, Uiversity of Michiga, A Arbor, MI 48109-1234; ad

More information

Institute of Actuaries of India Subject CT1 Financial Mathematics

Institute of Actuaries of India Subject CT1 Financial Mathematics Istitute of Actuaries of Idia Subject CT1 Fiacial Mathematics For 2014 Examiatios Subject CT1 Fiacial Mathematics Core Techical Aim The aim of the Fiacial Mathematics subject is to provide a groudig i

More information

Sequences II. Chapter 3. 3.1 Convergent Sequences

Sequences II. Chapter 3. 3.1 Convergent Sequences Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,

More information

where: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return

where: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return EVALUATING ALTERNATIVE CAPITAL INVESTMENT PROGRAMS By Ke D. Duft, Extesio Ecoomist I the March 98 issue of this publicatio we reviewed the procedure by which a capital ivestmet project was assessed. The

More information

3 Basic Definitions of Probability Theory

3 Basic Definitions of Probability Theory 3 Basic Defiitios of Probability Theory 3defprob.tex: Feb 10, 2003 Classical probability Frequecy probability axiomatic probability Historical developemet: Classical Frequecy Axiomatic The Axiomatic defiitio

More information

Plug-in martingales for testing exchangeability on-line

Plug-in martingales for testing exchangeability on-line Plug-i martigales for testig exchageability o-lie Valetia Fedorova, Alex Gammerma, Ilia Nouretdiov, ad Vladimir Vovk Computer Learig Research Cetre Royal Holloway, Uiversity of Lodo, UK {valetia,ilia,alex,vovk}@cs.rhul.ac.uk

More information

Irreducible polynomials with consecutive zero coefficients

Irreducible polynomials with consecutive zero coefficients Irreducible polyomials with cosecutive zero coefficiets Theodoulos Garefalakis Departmet of Mathematics, Uiversity of Crete, 71409 Heraklio, Greece Abstract Let q be a prime power. We cosider the problem

More information

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

Maximum Likelihood Estimators.

Maximum Likelihood Estimators. Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the

More information

Theorems About Power Series

Theorems About Power Series Physics 6A Witer 20 Theorems About Power Series Cosider a power series, f(x) = a x, () where the a are real coefficiets ad x is a real variable. There exists a real o-egative umber R, called the radius

More information

THE HEIGHT OF q-binary SEARCH TREES

THE HEIGHT OF q-binary SEARCH TREES THE HEIGHT OF q-binary SEARCH TREES MICHAEL DRMOTA AND HELMUT PRODINGER Abstract. q biary search trees are obtaied from words, equipped with the geometric distributio istead of permutatios. The average

More information

Incremental calculation of weighted mean and variance

Incremental calculation of weighted mean and variance Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

More information

Estimating Probability Distributions by Observing Betting Practices

Estimating Probability Distributions by Observing Betting Practices 5th Iteratioal Symposium o Imprecise Probability: Theories ad Applicatios, Prague, Czech Republic, 007 Estimatig Probability Distributios by Observig Bettig Practices Dr C Lych Natioal Uiversity of Irelad,

More information

A Combined Continuous/Binary Genetic Algorithm for Microstrip Antenna Design

A Combined Continuous/Binary Genetic Algorithm for Microstrip Antenna Design A Combied Cotiuous/Biary Geetic Algorithm for Microstrip Atea Desig Rady L. Haupt The Pesylvaia State Uiversity Applied Research Laboratory P. O. Box 30 State College, PA 16804-0030 haupt@ieee.org Abstract:

More information

hp calculators HP 12C Statistics - average and standard deviation Average and standard deviation concepts HP12C average and standard deviation

hp calculators HP 12C Statistics - average and standard deviation Average and standard deviation concepts HP12C average and standard deviation HP 1C Statistics - average ad stadard deviatio Average ad stadard deviatio cocepts HP1C average ad stadard deviatio Practice calculatig averages ad stadard deviatios with oe or two variables HP 1C Statistics

More information

Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread and Boxplots Discrete Math, Section 9.4 Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

Designing Incentives for Online Question and Answer Forums

Designing Incentives for Online Question and Answer Forums Desigig Icetives for Olie Questio ad Aswer Forums Shaili Jai School of Egieerig ad Applied Scieces Harvard Uiversity Cambridge, MA 0238 USA shailij@eecs.harvard.edu Yilig Che School of Egieerig ad Applied

More information

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

More information

Trading the randomness - Designing an optimal trading strategy under a drifted random walk price model

Trading the randomness - Designing an optimal trading strategy under a drifted random walk price model Tradig the radomess - Desigig a optimal tradig strategy uder a drifted radom walk price model Yuao Wu Math 20 Project Paper Professor Zachary Hamaker Abstract: I this paper the author iteds to explore

More information

CHAPTER 3 THE TIME VALUE OF MONEY

CHAPTER 3 THE TIME VALUE OF MONEY CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

More information

Project Deliverables. CS 361, Lecture 28. Outline. Project Deliverables. Administrative. Project Comments

Project Deliverables. CS 361, Lecture 28. Outline. Project Deliverables. Administrative. Project Comments Project Deliverables CS 361, Lecture 28 Jared Saia Uiversity of New Mexico Each Group should tur i oe group project cosistig of: About 6-12 pages of text (ca be loger with appedix) 6-12 figures (please

More information

Multi-server Optimal Bandwidth Monitoring for QoS based Multimedia Delivery Anup Basu, Irene Cheng and Yinzhe Yu

Multi-server Optimal Bandwidth Monitoring for QoS based Multimedia Delivery Anup Basu, Irene Cheng and Yinzhe Yu Multi-server Optimal Badwidth Moitorig for QoS based Multimedia Delivery Aup Basu, Iree Cheg ad Yizhe Yu Departmet of Computig Sciece U. of Alberta Architecture Applicatio Layer Request receptio -coectio

More information

Matrix Model of Trust Management in P2P Networks

Matrix Model of Trust Management in P2P Networks Matrix Model of Trust Maagemet i P2P Networks Miroslav Novotý, Filip Zavoral Faculty of Mathematics ad Physics Charles Uiversity Prague, Czech Republic miroslav.ovoty@mff.cui.cz Abstract The trust maagemet

More information

Heavy Traffic Analysis of a Simple Closed Loop Supply Chain

Heavy Traffic Analysis of a Simple Closed Loop Supply Chain Heavy Traffic Aalysis of a Simple Closed Loop Supply Chai Arka Ghosh, Sarah M. Rya, Lizhi Wag, ad Aada Weerasighe April 8, 2 Abstract We cosider a closed loop supply chai where ew products are produced

More information

Subject CT5 Contingencies Core Technical Syllabus

Subject CT5 Contingencies Core Technical Syllabus Subject CT5 Cotigecies Core Techical Syllabus for the 2015 exams 1 Jue 2014 Aim The aim of the Cotigecies subject is to provide a groudig i the mathematical techiques which ca be used to model ad value

More information

arxiv:1506.03481v1 [stat.me] 10 Jun 2015

arxiv:1506.03481v1 [stat.me] 10 Jun 2015 BEHAVIOUR OF ABC FOR BIG DATA By Wetao Li ad Paul Fearhead Lacaster Uiversity arxiv:1506.03481v1 [stat.me] 10 Ju 2015 May statistical applicatios ivolve models that it is difficult to evaluate the likelihood,

More information

A Constant-Factor Approximation Algorithm for the Link Building Problem

A Constant-Factor Approximation Algorithm for the Link Building Problem A Costat-Factor Approximatio Algorithm for the Lik Buildig Problem Marti Olse 1, Aastasios Viglas 2, ad Ilia Zvedeiouk 2 1 Ceter for Iovatio ad Busiess Developmet, Istitute of Busiess ad Techology, Aarhus

More information

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006 Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

More information

CHAPTER 3 DIGITAL CODING OF SIGNALS

CHAPTER 3 DIGITAL CODING OF SIGNALS CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity

More information

Section 11.3: The Integral Test

Section 11.3: The Integral Test Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

More information

Infinite Sequences and Series

Infinite Sequences and Series CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...

More information

Capacity of Wireless Networks with Heterogeneous Traffic

Capacity of Wireless Networks with Heterogeneous Traffic Capacity of Wireless Networks with Heterogeeous Traffic Migyue Ji, Zheg Wag, Hamid R. Sadjadpour, J.J. Garcia-Lua-Aceves Departmet of Electrical Egieerig ad Computer Egieerig Uiversity of Califoria, Sata

More information

Chair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics

Chair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics Chair for Network Architectures ad Services Istitute of Iformatics TU Müche Prof. Carle Network Security Chapter 2 Basics 2.4 Radom Number Geeratio for Cryptographic Protocols Motivatio It is crucial to

More information

Chatpun Khamyat Department of Industrial Engineering, Kasetsart University, Bangkok, Thailand ocpky@hotmail.com

Chatpun Khamyat Department of Industrial Engineering, Kasetsart University, Bangkok, Thailand ocpky@hotmail.com SOLVING THE OIL DELIVERY TRUCKS ROUTING PROBLEM WITH MODIFY MULTI-TRAVELING SALESMAN PROBLEM APPROACH CASE STUDY: THE SME'S OIL LOGISTIC COMPANY IN BANGKOK THAILAND Chatpu Khamyat Departmet of Idustrial

More information

TO: Users of the ACTEX Review Seminar on DVD for SOA Exam MLC

TO: Users of the ACTEX Review Seminar on DVD for SOA Exam MLC TO: Users of the ACTEX Review Semiar o DVD for SOA Eam MLC FROM: Richard L. (Dick) Lodo, FSA Dear Studets, Thak you for purchasig the DVD recordig of the ACTEX Review Semiar for SOA Eam M, Life Cotigecies

More information

Notes on exponential generating functions and structures.

Notes on exponential generating functions and structures. Notes o expoetial geeratig fuctios ad structures. 1. The cocept of a structure. Cosider the followig coutig problems: (1) to fid for each the umber of partitios of a -elemet set, (2) to fid for each the

More information

Distributions of Order Statistics

Distributions of Order Statistics Chapter 2 Distributios of Order Statistics We give some importat formulae for distributios of order statistics. For example, where F k: (x)=p{x k, x} = I F(x) (k, k + 1), I x (a,b)= 1 x t a 1 (1 t) b 1

More information

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

More information

Gibbs Distribution in Quantum Statistics

Gibbs Distribution in Quantum Statistics Gibbs Distributio i Quatum Statistics Quatum Mechaics is much more complicated tha the Classical oe. To fully characterize a state of oe particle i Classical Mechaics we just eed to specify its radius

More information

TAYLOR SERIES, POWER SERIES

TAYLOR SERIES, POWER SERIES TAYLOR SERIES, POWER SERIES The followig represets a (icomplete) collectio of thigs that we covered o the subject of Taylor series ad power series. Warig. Be prepared to prove ay of these thigs durig the

More information

Transient Behavior of Two-Machine Geometric Production Lines

Transient Behavior of Two-Machine Geometric Production Lines Trasiet Behavior of Two-Machie Geometric Productio Lies Semyo M. Meerkov Nahum Shimki Liag Zhag Departmet of Electrical Egieerig ad Computer Sciece Uiversity of Michiga, A Arbor, MI 489-222, USA (e-mail:

More information

Systems Design Project: Indoor Location of Wireless Devices

Systems Design Project: Indoor Location of Wireless Devices Systems Desig Project: Idoor Locatio of Wireless Devices Prepared By: Bria Murphy Seior Systems Sciece ad Egieerig Washigto Uiversity i St. Louis Phoe: (805) 698-5295 Email: bcm1@cec.wustl.edu Supervised

More information

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample

More information

WHEN IS THE (CO)SINE OF A RATIONAL ANGLE EQUAL TO A RATIONAL NUMBER?

WHEN IS THE (CO)SINE OF A RATIONAL ANGLE EQUAL TO A RATIONAL NUMBER? WHEN IS THE (CO)SINE OF A RATIONAL ANGLE EQUAL TO A RATIONAL NUMBER? JÖRG JAHNEL 1. My Motivatio Some Sort of a Itroductio Last term I tought Topological Groups at the Göttige Georg August Uiversity. This

More information

The Stable Marriage Problem

The Stable Marriage Problem The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

More information

Lesson 17 Pearson s Correlation Coefficient

Lesson 17 Pearson s Correlation Coefficient Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) -types of data -scatter plots -measure of directio -measure of stregth Computatio -covariatio of X ad Y -uique variatio i X ad Y -measurig

More information

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed. This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio

More information

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

More information

Queuing Systems: Lecture 1. Amedeo R. Odoni October 10, 2001

Queuing Systems: Lecture 1. Amedeo R. Odoni October 10, 2001 Queuig Systems: Lecture Amedeo R. Odoi October, 2 Topics i Queuig Theory 9. Itroductio to Queues; Little s Law; M/M/. Markovia Birth-ad-Death Queues. The M/G/ Queue ad Extesios 2. riority Queues; State

More information

INVESTMENT PERFORMANCE COUNCIL (IPC)

INVESTMENT PERFORMANCE COUNCIL (IPC) INVESTMENT PEFOMANCE COUNCIL (IPC) INVITATION TO COMMENT: Global Ivestmet Performace Stadards (GIPS ) Guidace Statemet o Calculatio Methodology The Associatio for Ivestmet Maagemet ad esearch (AIM) seeks

More information