STOCHASTIC approximation algorithms have several

Transcription

1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER Trackg a Markov-Modulated Statoary Degree Dstrbuto of a Dyamc Radom Graph Mazyar Hamd, Vkram Krshamurthy, Fellow, IEEE, ad George Y, Fellow, IEEE Abstract Ths paper cosders a Markov-modulated duplcato-deleto radom graph where at each tme stat, oe ode ca ether jo or leave the etwork; the probabltes of jog or leavg evolve accordg to the realzato of a fte state Markov cha Two results are preseted Frst, motvated by socal etwork applcatos, the asymptotc behavor of the degree dstrbuto s aalyzed Secod, a stochastc approxmato algorthm s preseted to track emprcal degree dstrbuto as t evolves over tme The trackg performace of the algorthm s aalyzed terms of mea square error ad a fuctoal cetral lmt theorem s preseted for the asymptotc trackg error Also, a Hlbert-space-valued stochastc approxmato algorthm that tracks a Markov-modulated probablty mass fucto wth support o the set of oegatve tegers s aalyzed Idex Terms Adaptve algorthms, complex etworks, degree dstrbuto, Markov-modulated radom graphs, power law, socal etworks, stochastc approxmato algorthms I INTRODUCTION STOCHASTIC approxmato algorthms have several applcatos dverse areas such as target trackg, chage detecto, commucato systems, ad ecoomcs [1] [7] The ubqutous use of stochastc approxmato algorthms s maly due to ther ablty to track a tme-varyg ukow parameter of a system; ths s called trackg capablty, see [5] I ths paper, motvated by socal etwork applcatos, we cosder a class of stochastc approxmato algorthms to track a tme-varyg probablty mass fucto that evolves accordg to a fte-state Markov cha whose trasto matrx s close to detty I the cotext of socal etwork aalyss, the tme-varyg probablty mass fucto whch we am to track s the expected degree dstrbuto of a dyamc radom graph Dyamc radom graphs have bee wdely used to model socal etworks, bologcal etworks [8] ad Iteret graphs [9] Such dyamc models ca be vewed as a sequece of graphs where the radom graph at each tme may deped o all the earler graphs sapshots of the evolvg graph at earler tmes [9] Motvated by aalyzg socal etworks, we troduce Markov-modulated duplcato-deleto radom graphs 1 where at each tme stat, odes ca ether be added to or elmated from the graph wth probabltes that chage accordg to a fte-state Markov cha Such graphs mmc socal etworks where the teractos betwee odes evolve over tme accordg to a Markov process that udergoes frequet jumps A example of such a socal etwork s the fredshp etwork amog resdets of a cty, where the dyamcs of the etwork chage the evet of a large festval A class of stochastc approxmato algorthms are used to track the expected degree dstrbuto of such Markov-modulated dyamc graphs A Why Aalyze the Degree Dstrbuto? The degree of a ode a etwork also kow as the coectvty s the umber of coectos the ode has that etwork The most mportat measure that characterzes the structure of a etwork specally whe the sze of the etwork s large ad the coectos adjacecy matrx of the uderlyg graph are ot gve s the degree dstrbuto of the etwork The degree dstrbuto ca further be used to vestgate the dffuso of formato or dsease through socal etworks [10] [12] The exstece of a gat compoet 2 complex etworks ca be studed usg the degree dstrbuto The sze ad exstece of a gat compoet has mportat mplcatos socal etworks terms of modelg formato propagato ad spread of huma dsease [13] [15] Mauscrpt receved February 27, 2013; revsed Jue 6, 2014; accepted July 12, 2014 Date of publcato August 7, 2014; date of curret verso September 11, 2014 Ths work was supported part by the Natural Sceces ad Egeerg Research Coucl of Caada ad part by the Socal Sceces ad Humates Research Coucl of Caada G Y was supported by the Ar Force Offce of Scetfc Research, Arlgto, VA, USA Ths paper was preseted at the 2012 IEEE Iteratoal Coferece o Acoustcs, Speech ad Sgal Processg M Hamd ad V Krshamurthy are wth the Departmet of Electrcal ad Computer Egeerg, Uversty of Brtsh Columba, Vacouver, BC V6T 1Z4, Caada e-mal: mazyarh@eceubcca; vkramk@eceubcca G Y s wth the Departmet of Mathematcs, Waye State Uversty, Detrot, MI USA e-mal: gy@mathwayeedu Commucated by G Moustakdes, Assocate Edtor for Detecto ad Estmato Color versos of oe or more of the fgures ths paper are avalable ole at Dgtal Object Idetfer /TIT B Ma Results ad Paper Orgazato SecII descrbes the costructo of Markov-modulated duplcato-deleto radom graphs SecIII provdes a asymptotc degree dstrbuto aalyss for the o-markov modulated case of two dfferet scearos: fxed sze duplcato-deleto radom graph, ad fte 1 The duplcato-deleto procedure for Markov-modulated radom graphs s descrbed SecII 2 A gat compoet s a coected compoet of sze Oη, whereη s the umber of vertces the graph If the average degree of a radom graph s strctly greater tha oe, the there exsts a uque gat compoet wth probablty oe [9], ad the sze of ths compoet ca be computed from the expected degree dstrbuto IEEE Persoal use s permtted, but republcato/redstrbuto requres IEEE permsso See for more formato

2 6610 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 duplcato-deleto radom graph Theorem 31 SecIII-A asserts that the expected degree dstrbuto of the fxed sze Markov-modulated radom graph at each tme ca be computed terms of the expected degree dstrbuto of the graph at the prevous tme ad the dyamcs of the graph va recursve equato 8 SecIII-B exteds the results of SecIII-A to fte radom graphs Theorem 32 parameterzes the degree dstrbuto of such a graph by the power law expoet whch depeds o the dyamcs of the graph SecIV cosders the problem of adaptvely estmatg the degree dstrbuto of a fxed sze Markov-modulated duplcato-deleto radom graph gve observatos of the degree dstrbuto A stochastc approxmato algorthm s preseted for trackg the degree dstrbuto as t evolves over tme I partcular, SecIV presets three results regardg the trackg performace of the stochastc approxmato algorthm: Mea square error aalyss: Theorem 41 aalyzes the asymptotc mea square error betwee the expected degree dstrbuto ad the estmate obtaed va the stochastc approxmato algorthm Dervg ths result uses error bouds o two-tme scale Markov chas ad perturbed Lyapuov fucto methods Weak covergece aalyss: Theorem 42 shows that the asymptotc behavor of the stochastc approxmato algorthm coverges weakly to the soluto of a swtched Markova ordary dfferetal equato Fuctoal cetral lmt theorem for scaled trackg error: Fally, Theorem 43 vestgates the asymptotc behavor of the scaled trackg error Smlar to [16], t s show that the terpolated scaled trackg error coverges weakly to the soluto of a swtchg dffuso process SecV exteds the results of SecIV to fte deumerable duplcato-deleto radom graphs where the umber of odes the graph ad so the support of degree dstrbuto s o loger fxed ad creases over tme A Hlbert-spacevalued stochastc approxmato algorthm s proposed to track the degree dstrbuto of the fte graph wth support o the set of o-egatve tegers To study the trackg performace of such a Hlbert-space-valued stochastc approxmato algorthm, lmt system characterzato ad asymptotc aalyss of scaled trackg error are provded Numercal examples are preseted SecVI C Related Works For a comprehesve developmet of stochastc approxmato algorthms, see [1], [5], [17] Trackg capablty of regme swtchg stochastc approxmato algorthms s further vestgated [18] For the applcatos of stochastc approxmato algorthms teractve sesg ad decso makg see [19], [20] Also, see [17], [21] [23] for the performace aalyss of adaptve algorthms dscrete-tme A detaled exposto of radom graphs s provded [24] The dyamcs of radom graphs are vestgated the mathematcs lterature, for example, see [9], [25], [26] ad the referece there I [27], a duplcato model s proposed where at each tme step a ew ode jos the etwork However, the probabltes of jog ths model do ot evolve over tme I [9], t s show that the degree dstrbuto of such etworks satsfes a power law II MARKOV-MODULATED DYNAMIC RANDOM GRAPH OF DUPLICATION-DELETION TYPE Ths secto outles the costructo of Markov-modulated dyamc radom graphs of duplcato-deleto type Let = 0, 1, 2, deote dscrete tme Deote by θ a dscretetme Markov cha wth state space M ={1, 2,,m}, 1 ad tal probablty dstrbuto π 0 Assumpto 21: The Markov cha θ evolves accordg to the trasto matrx A ρ = I + ρ Q 2 Here, I s a m m detty matrx, ρ s a small postve real umber, ad Q =[q j ] s a rreducble 3 geerator of a cotues-tme Markov cha satsfyg q j > 0 for = j, ad Q1 = 0, 3 where 1 ad 0 represet colum vectors of oes ad zeros, respectvely The trasto probablty matrx A ρ s therefore close to detty matrx Here ad heceforth, we refer to such a Markov cha θ as a slow Markov cha The tal dstrbuto π 0 s assumed depedet of ρ A Markov-modulated duplcato-deleto radom graph s parameterzed by the 7-tuple m, A ρ,π 0, r, p, q, G 0 Here, p ad q are m-dmesoal vectors wth elemets p ad q [0, 1], = 1,,m,wherep deotes the coecto probablty, ad q deotes the deleto probablty Also, r [0, 1] deotes the probablty of the duplcato step, ad G 0 deotes the tal graph at tme 0 I geeral, G 0 ca be ay fte smple coected graph For smplcty, assume that G 0 s a smple coected graph wth sze η 0 The duplcatodeleto radom graph s costructed va the duplcatodeleto Procedure 1 4 The Markov-modulated radom graph geerated by the duplcato-deleto Procedure 1 mmcs socal etworks where the teractos betwee odes evolve over tme due to the uderlyg dyamcs state of ature such as seasoal varatos eg, the hgh school fredshp socal etwork evolvg over tme wth dfferet wter/summer dyamcs I such cases, the coecto/deleto probabltes p, q evolve wth tme Procedure 1 models these tme varatos as a fte state Markov cha θ wth trasto matrx A ρ 3 The rreducblty assumpto mples that there exsts a uque statoary dstrbuto π R m 1 for ths Markov cha such that π = π A ρ 4 I Procedure 1, Step 1 s executed wth probablty r The, regardless of executo of Step 1, Step 2 s mplemeted For coveece the aalyss, assume that a ode geerated the duplcato step caot be elmated the deleto step mmedately after ts geerato Also, odes whose degrees chage the edge-deleto part of Step 2, rema uchaged the duplcato part of Step 2 at that tme stat Fally, to prevet formato of solated odes, assume that the eghbor of a ode wth degree oe caot be elmated the deleto step Note also that the duplcato step Step 2 esures that the graph sze does ot decrease

3 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH 6611 Procedure 1 Markov-Modulated Graph Parameterzed by m, A ρ,π 0, r, p, q, G 0 At tme, gvethegraphg ad Markov cha state θ, smulate the followg evets: Step 1: Duplcato step: Wth probablty r mplemet the followg steps: Choose ode u from graph G radomly wth uform dstrbuto Vertex-duplcato: Geerate a ew ode v Edge-duplcato: Coect ode u to ode v A ew edge betwee u ad v s added to the graph Coect each eghbor of ode u wth probablty pθ to ode v These coecto evets are statstcally depedet Step 2: Deleto Step: Wth probablty qθ mplemet the followg steps: Edge-deleto: Choose ode w radomly from G wth uform dstrbuto Delete ode w alog wth the coected edges graph G Duplcato Step: Choose a ode from graph x from G radomly ad mplemet Vertex-duplcato ad Edgeduplcato processes as descrbed Step 1 Step 3: Deote the resultg graph by G +1 Geerate θ +1 Markov cha usg trasto matrx A ρ Set + 1 adgotostep1 III ASYMPTOTIC DEGREE DISTRIBUTION ANALYSIS FOR NON-MARKOV MODULATED CASE Ths secto presets degree dstrbuto aalyss for duplcato-deleto radom graphs geerated accordg Procedure 1 for the o-markov modulated case, e, m = 1 The statoary degree dstrbuto obtaed SecIII-A below wll be used the Markov modulated case The results ths secto costtute a mor exteso of [9] to the duplcatodeleto radom graphs Notato: At each tme, let deote the umber of odes of graph G Also, let F be a dmesoal vector such that ts -th elemet, F, deotes the umber of vertces of graph G wth degree Clearly F 1 = where 1 deotes the vector of oes Here, s used to deote the traspose of a vector or matrx Defe the emprcal vertex degree dstrbuto as G = G, = 1, 2,, where G = F 4 Note that G ca be vewed as a probablty mass fucto sce all of ts elemets are o-egatve ad G 1 = 1 A Fxed Sze Radom Graph Ths subsecto aalyzes the evoluto of the expected degree dstrbuto for a fxed sze duplcato-deleto radom graph geerated accordg to Procedure 1 wth r = 0, m = 1 Recall r deotes the probablty of Step 1 Procedure 1 Therefore, the umber of vertces the graph remas fxed, e, = η 0 for = 0, 1, 2, Theorem 31 below gves a recurso for the expected degree dstrbuto of the fxed sze Markov-modulated duplcato-deleto radom graph Theorem 31: Cosder the fxed sze duplcato-deleto radom graph geerated accordg to Procedure 1, where r = 0, m = 1 Let G deote the expected degree dstrbuto of odes at tme The, G satsfes the recurso G +1 = I + 1 η 0 L G, 5 where L s a geerator matrx 5 wth elemets for 1, j η 0 : 0, j < 1, qp 1 + q 1 + p 1, j = 1, l j = qp 1 1 p q p, j =, q p 1 1 p 2 + q + 1, j = + 1, q j 1 p 1 1 p j +1, j > + 1 Proof: The proof s preseted Appedx A Theorem 31 shows that evoluto of the expected degree dstrbuto a fxed sze Markov-modulated duplcatodeleto radom graph satsfes 5 Oe ca rewrte 5 as G +1 = B η 0 G, where B η0 = I + 1 η 0 L 7 Sce L s a geerator matrx, B η0 ca be cosdered as the trasto matrx of a slow Markov cha It s also straghtforward to show that B η0 s rreducble ad aperodc 6 Hece, there exsts a uque statoary dstrbuto G = G, = 1, 2, such that G = B η 0 G 8 The statoary dstrbuto G s the statoary expected degree dstrbuto of a fxed sze duplcato-deleto radom graph geerated accordg to Procedure 1 where r = 0 B Power Law Expoet for Ifte Duplcato-Deleto Radom Graph The degree dstrbuto aalyss provded the prevous subsecto was for a fxed sze radom graph geerated accordg to the duplcato-deleto Procedure 1 wth r = 0 Ths secto exteds ths aalyss to fte duplcatodeleto radom graphs obtaed by choosg r = 1 Assume that G 0 s a empty set Sce r = 1, at tme, the graph G has odes By employg the same approach as the proof of Theorem 31, t wll be show that the fte duplcatodeleto radom graph wthout Markova dyamcs satsfes a power law A expresso s further derved for the power law expoet Defto 31 Power Law Dstrbuto: The degree dstrbuto G = G, = 1, 2,,ofagraphG has a power 5 That s, each row adds to zero ad each o-dagoal elemet of L s postve 6 It s straghtforward to show that all elemets of B η0 η 0 are strctly greater tha zero Therefore, B η0 s rreducble ad aperodc

4 6612 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 law dstrbuto 7 f there exsts a teger such that for all, log G = α β log where α s a costat 8 ad β>1 Parameter β s called the power law expoet The power law s satsfed may etworks such as WWW-graphs, peer-to-peer etworks, phoe call graphs, co-authorshp graph ad varous massve ole socal etworks eg Yahoo, MSN, Facebook [29] [33] The followg theorem states that the graph geerated accordg to Procedure 1 wth r = 1adm = 1 satsfes a power law Theorem 32: Cosder the fte radom graph wth Markova dyamcs G obtaed by Procedure 1 wth 7-tuple 1, 1, 1, 1, p, q, G 0 wth the expected degree dstrbuto G q 1+q The, f log p + p < < p, the expected degree of odes G has a power law dstrbuto wth expoet β>1 The power law expoet s computed from 1 + qp β 1 + pβ p = 1 + βq 9 Here, p ad q are the probabltes defed duplcato ad deleto steps, respectvely Proof: The proof s smlar to that of Theorem 31 wth some modfcatos, see [34] Here, we oly preset a outle of the proof whch s comprsed of two steps: fdg the power law expoet, ad showg that the degree dstrbuto coverges to a power law wth the computed expoet as To fd the power law expoet, we derve a recursve equato for the umber of odes wth degree + 1 at tme + 1, deoted by F+1 +1,termsofthe degrees of odes graph G The, rearragg ths recursve equato yelds a equato for the power law expoet To prove that the degree dstrbuto satsfes a power law, we show that lm k=1 E{F k}= k=1 ck β,whereβ>1 s the power law expoet computed the frst step ad F k s the k-th elemet of F Theorem 32 asserts that the fte duplcato-deleto radom graph wthout Markova dyamcs geerated by Procedure 1 satsfes a power law ad provdes a expresso for the power law expoet The sgfcace of ths theorem s that t esures, wth use of oe sgle parameter the power law expoet, we ca descrbe the degree dstrbuto of graphs wth relatvely large umber of odes The above result slghtly exteds [8], [27], where oly a duplcato model was cosdered Theorem 32 allows us to explore characterstcs such as searchablty, dffuso, ad exstece/sze of the gat compoet of large etworks whch ca be modeled wth the fte duplcato-deleto radom graphs 7 There s a dfferece betwee power law ad power law dstrbuto Power law s a fuctoal relatoshp betwee two parameters where oe parameter s proportoal to the power of aother, e, x y β, where β ca be ay real umber I comparso, the expoet of a power law dstrbuto s strctly greater that oe [28] Otherwse, the probablty dstrbuto does ot add up to oe 8 The ormalzato costat α s computed from α = log[ζβ, ], where ζβ, = k= k β deotes the complete Rema ζ -fucto IV ESTIMATING TRACKING THE DEGREE DISTRIBUTION OF THE FIXED SIZE MARKOV-MODULATED DUPLICATION-DELETION RANDOM GRAPH I SecII, a expresso was gve for the uque statoary degree dstrbuto G for the o-markov modulated case, see 8 I ths secto, we cosder fxed sze Markov modulated duplcato deleto radom graphs Cosder Procedure 1 ad assume that there are m possble statoary degree dstrbutos, amely G = {G1, G2,,Gm} correspodg to the m states of a Markov cha Here each G s computed usg 8 where the correspodg parameters p, q are used At each tme, a statoary dstrbuto Gθ G s chose where θ evolves accordg to a m-state Markov cha as descrbed SecII We assume that the statoary degree dstrbuto of the graph s sampled by a etwork admstrator How ca the etwork admstrator track the expected degree dstrbuto of the fxed sze Markovmodulated duplcato deleto radom graph wthout kowg the dyamcs of the graph? The motvato for trackg the statoary degree dstrbuto stems from socal etworks where the dyamcs of the degree dstrbuto evolve o a faster tme scale tha the Markov cha θ Therefore, t suffces to track Gθ gve observatos At each tme, the etwork admstrator samples a ode from the graph based o degree dstrbuto Gθ ad records ts degree y θ LetY θ = e y θ deote the observato vector where e R η0 1 s the -th stadard ut vector Such a samplg procedure ca be tme correlated Therefore, we allow Y θ to be a mxg process wth thee followg assumpto: Assumpto 41: For each θ M, the sequece {y θ} s statoary φ-mxg wth suffcetly fast mxg rate such that the sequece {y 1,,y m} s depedet of {θ } ad that for each θ M, {Y θ} s a statoary φ-mxg sequece wth mxg rate ψ satsfyg j=0 ψ 1/2 j < Remark 41: Because {y θ} s a statoary φ-mxg sequece for each θ M, {Y θ} s a bouded sequece of φ-mxg process for each θ M [35, p 82] see also [36, p170] The statoarty mples that EY θ=ey 1 θ= e Py 1 θ== G θe = Gθ =0 =0 10 The mxg rate gve requres that for ay postve tegers ad j, E k I {y θ=} G θ ψ k for k, E[I {yl θ= j} G j θ][i {y =} G θ] ψ 1/2 l ψ1/2 l k for ay k < l <, 11 where E k deotes the codtoal expectato o the past data up to tme k e, codto o the σ -algebra F k geerated by {Y j θ : j k} adi{ } deotes the dcator fucto Here, s used to deote the Eucldea orm The aalyss ths paper ca be geeralzed to clude certa o-statoary cases for the observato process {y θ}

5 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH 6613 For example, for each θ M, suppose {ζ θ} s a ergodc fte state Markov cha 9 Let y θ = f ζ θ The-step trasto probablty matrx of the Markov cha coverges to a matrx wth detcal rows cosstg of ts statoary dstrbuto wth expoetal rate The t ca be verfed smlar to [36, p 178] that y θ s mxg Although 10 does ot hold, the aalyss usg mxg equaltes ca stll be obtaed Gve the observato sequece Y θ, = 0, 1, 2,,the am s to adaptvely estmate Gθ va the followg stochastc approxmato algorthm wth small postve costat stepsze : Ĝ +1 = Ĝ + Y θ Ĝ, Ĝ0 = e 1 12 To summarze, the evoluto of the slow Markov cha θ ad stochastc approxmato algorthm 12 form a two-tme-scale Markova system as follows whe ρ, = o η 1 0 True system: Gθ {G1,,Gm}, where θ evolves accordg to A ρ = I + ρ Q, Algorthm: Ĝ +1 = Ĝ + 13 Y θ Ĝ, Y θ = e y θ, where y θ Gθ Note that the stochastc approxmato algorthm 12 does ot assume ay kowledge of the Markov-modulated dyamcs of the graph The Markov cha assumpto for the radom graph dyamcs s oly used our covergece ad trackg aalyss By meas of the stochastc approxmato 12, the etwork admstrator ca track the statoary expected degree dstrbuto Gθ A Trackg Error of the Stochastc Approxmato Algorthm The goal here s to aalyze how well algorthm 12 tracks the emprcal degree dstrbuto of the fxed sze Markovmodulated duplcato-deleto graph Defe the trackg error as G = Ĝ Gθ Theorem 41 below shows that the dfferece betwee the sample path ad the statoary degree dstrbuto s small mplyg that the stochastc approxmato algorthm ca successfully track the Markov-modulated ode dstrbuto gve the osy measuremets We aga emphasze that o kowledge of the Markov cha parameters are requred the algorthm It also fds the order of ths dfferece terms of ad ρ Theorem 41: Cosder the radom graph m, A ρ,π 0, p, q, r, G 0 Suppose ρ 2 ad Assumptos 21 ad 41 hold 10 The, for suffcetly large, the trackg error of the stochastc approxmato algorthm 12 s E G 2 = O + ρ + ρ2 14 Proof: The proof uses the perturbed Lyapuov fucto method ad s provded Appedx B 9 Respodet drve samplg RDS was troduced [37] as a approach for samplg from hdde populatos socal etworks RDS has bee selected by the US Ceters for Dsease Cotrol ad Preveto as part of the HIV behavoral survellace system RDS ca be vewed as a form of Markov Cha Mote Carlo samplg [38] 10 I ths paper, we assume that ρ = O Therefore, ρ 2 Remark 42: Most exstg lterature aalyzes stochastc approxmato algorthms for trackg a parameter that evolves accordg to a slowly tme-varyg sample path of a cotuous-valued process so that the parameter chages by small amouts over small tervals of tme Whe the rate of chage of the uderlyg parameter s slower tha the adaptato rate of the stochastc approxmato algorthm eg, a slow radom walk, the mea square trackg error ca be aalyzed as [1], [17], [21] [23], ad [39] I comparso, our aalyss covers the case where the uderlyg parameter evolves wth dscrete jumps that ca be arbtrarly large magtude o short tervals of tme Also, the jumps occur o the same tme scale as the speed of adaptato of the stochastc approxmato algorthm We explctly cosder ths Markova tme-varyg parameter our mea square error ad weak covergece aalyss As a corollary of Theorem 41, we obta the followg mea square error covergece result Corollary 41: Uder the codtos of Theorem 41, f ρ = O, Therefore, lm E G 2 = O lm 0 lm E G 2 = 0 B Lmt System of Regme-Swtchg Ordary Dfferetal Equatos The followg theorem asserts that the sequece of estmates geerated by the stochastc approxmato algorthm 12 follows the dyamcs of a Markov-modulated ordary dfferetal equato ODE Before proceedg wth the ma theorem below, let us recall a defto Defto 41 Weak Covergece: Let Z k ad Z be R r -valued radom vectors We say Z k coverges weakly to Z Z k Z f for ay bouded ad cotuous fucto f, EfZ k EfZ as k Weak covergece s a geeralzato of covergece dstrbuto to a fucto space 11 Theorem 42: Cosder the Markov-modulated radom graph geerated accordg to Procedure 1, ad the sequece of estmates {Ĝ }, geerated by the stochastc approxmato algorthm 12 Suppose Assumptos 21 ad 41 hold ad ρ = O Defe the cotuous-tme terpolated process Ĝ t = Ĝ, θ t = θ for t [, The, as 0, Ĝ, θ coverges weakly to Ĝ, θ, whereθ s a cotuous-tme Markov cha wth geerator Q, Ĝ satsfes the Markov-modulated ODE dĝt = Ĝt + Gθt, dt Ĝ0 = Ĝ 0 16 ad Gθ G 11 We refer the terested reader to [1, Ch 7] for further detals o weak covergece ad related matters

6 6614 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 The above theorem asserts that the lmt system assocated wth the stochastc approxmato algorthm 12 s a Markova swtched ODE 16 As metoed SecI, ths s uusual sce typcally averagg of stochastc approxmato algorthms, covergece occurs to a determstc ODE The tuto behd ths s that the Markov cha evolves o the same tme-scale as the stochastc approxmato algorthm If the Markov cha evolved o a faster tme-scale, the the lmtg dyamcs would be a determstc ODE weghed by the statoary dstrbuto of the Markov cha If the Markov cha evolved slower tha the dyamcs of the stochastc approxmato algorthm, the the asymptotc behavor would also be a determstc ODE wth the Markov cha beg a costat C Scaled Trackg Error Next, we study the behavor of the scaled trackg error betwee the estmates geerated by the stochastc approxmato algorthm 12 ad the expected degree dstrbuto The followg theorem states that the trackg error should also satsfy a swtchg dffuso equato ad provdes a fuctoal cetral lmt theorem for ths scaled trackg error Let ν k = Ĝk Gθ k deote the scaled trackg error Theorem 43: Suppose Assumptos 21 ad 41 hold Defe ν t = ν k for t [k, k + 1 The, ν, θ coverges weakly to ν, θ such that ν s the soluto of the followg Markova swtched dffuso process νt = t 0 νsds + t θτdωτ 17 Here, ω s a R η 0-dmesoal stadard Browa moto The covarace matrx θ 17 ca be explctly computed as θ = Zθ Dθ + DθZθ Dθ GθG θ 18 Here, Dθ = daggθ ad Zθ = I B η0 θ + 1G θ 1, where Gθ G For each θ M, Bη0 θ s computed usg 7 where the correspodg parameters p, q are used For geeral swtchg processes, we refer to [40] I fact, more complex cotuous-state depedet swtchg rather tha Markova swtchg are cosdered there Equato 18 reveals that the covarace matrx of the trackg error depeds o B η0 θ ad Gθ ad, cosequetly, o the parameters p ad q of the radom graph Recall from SecII that B η0 θ s the trasto matrx of the Markov cha whch models the evoluto of the expected degree dstrbuto duplcato-deleto radom graphs ad ca be computed from Theorem 31 V ESTIMATING THE DEGREE DISTRIBUTION OF INFINITE DUPLICATION-DELETION RANDOM GRAPHS Ths secto has two results: Frst, the results of Sec IV are exteded to fte radom graphs wthout Markova dyamcs geerated accordg to Procedure 1 Secod, we show how ths aalyss ca be exteded to Markov-modulated probablty mass fuctos wth deumerable support The aalyss s o-stadard, sce t s formulated o a Hlbert space A Ifte Radom Graphs Wthout Markova Dyamcs Cosder the fte duplcato-deleto radom graph wthout Markova dyamcs geerated accordg to Procedure 1 wth 7-tuple 1, 1, 1, 1, p, q, G 0 I ths secto, let G represet the degree dstrbuto of the fte graph wth support o the set of o-egatve tegers; ts elemets are deoted by G, = 0, 1, 2, Recall from SecIII-B that, the sze of such a graph cremets at each tme by oe ad thus the sze of the graph at tme s equal to ; that s = Therefore, the maxmum degree of the graph at tme caot exceed 1 ad G j = 0 for j Smlar to the proof of Theorem 31, the followg theorem asserts that the expected degree dstrbuto of the fte duplcato-deleto radom graph satsfes a recursve equato Theorem 51: Cosder the fte duplcato-deleto radom graph wthout Markova dyamcs geerated accordg to Procedure 1 wth 7-tuple 1, 1, 1, 1, p, q, G 0, Let G = E{G } deote the expected degree dstrbuto of odes wth support o the set of o-egatve tegers The, G satsfes the followg recurso G +1 = G + 1 L G, 19 where L s a geerator matrx of fte sze wth elemets: 1 + q p p 1, j = 1, 1, j 1 + q p 1 1 p p q + 1, j =, 1, j l j = 1 + q +1 1 p 1 1 p 2 + q + 1, j = + 1, 1, j 1 + q j 1 p 1 1 p j +1, j > + 1, 1, j 0, otherwse 20 Proof: The proof s smlar to the proof of Theorem 31 ad s omtted due to the lack of space Remark 51: Theorem 32 SecIII-B asserts that the expected degree dstrbuto coverges to a power law probablty dstrbuto G wth expoet β > 1, f log p + p < q 1+q < p; that s lm G = ζβ β We assume that the dyamcs of the degree dstrbuto evolve o a faster tme scale tha the stochastc approxmato algorthm Therefore, t suffces to track the statoary degree dstrbuto G gve observatos At each tme, the etwork admstrator samples from the graph ad records the degree of a radomly chose vertex of the graph whch s deoted by y Let Y = e y deote the observato vector Here, e s the -th stadard ut vector wth support o the set of o-egatve tegers e, e = 0,,1, R The followg

7 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH 6615 stochastc approxmato algorthm s used to estmate the expected degree dstrbuto of the graph from such samples Ĝ +1 = Ĝ + Y Ĝ 21 Here, > 0 deote a small postve step sze ad Ĝ 0 = e 1 Therefore, 21 s a Hlbert-space-valued stochastc approxmato algorthms By meas of the stochastc approxmato 21, the etwork admstrator ca track the expected degree dstrbuto of the fte graph whose sze creases over tme Defe Ĝ t = Ĝ for t [, + The Ĝ D[0, : l 2 the space of fuctos defed o [0, takg values l 2 ={z R : =0 z 2 < } such that the fuctos are rght cotuous ad have left lmts edowed wth the Skorohod topology Here, we obta a weak covergece result of the terpolated sequece of terates Theorem 52 below asserts that the mea square trackg error s bouded ad shows that the sequce of estmates obtaed by 21 coverge to the soluto of a ODE Before proceedg to the ma theorem, we shall use the followg codtos Theorem 52: Suppose Assumpto 41 holds wth the modfcato that m = 1, e, there s o Markova dyamcs Defe G = G Ĝ The, lm E G 2 = O Also, Ĝ s tght D[0, : l 2 Ay coverget subsequece has a lmt Ĝ that s the soluto of the dfferetal equato dĝt = G Ĝt, Ĝ0 = e 1 22 dt Proof: The proof s preseted Appedx E The proof of the theorem s dvded to several steps, whch uses techques stochastc approxmato [1] but wth the modfcato that l 2 s a Hlbert space see [41], [42] The above result cocers ad 0, remas to be bouded We ext obta a result wth 0,, Corollary 51: Cosder Ĝ +t, where t as 0 Uder the codto of Theorem 52, Ĝ +t G probablty as 0 Proof: Note that {Ĝ k } s tght Defe Ĝ,large = Ĝ +t Usg the same approach, we ca show that {Ĝ,large } s tght We extract a weakly coverget subsequece of Ĝ,large, Ĝ,large T wth lmt deoted by Ĝ, Ĝ T We ote that Ĝ0 = Ĝ T T ad that Ĝ T 0 belogs to a set that s bouded probablty Wrtg t varatoal form, we obta Ĝ T T = e T Ĝ T 0 + T = e T Ĝ T 0 + G G 0 as T e T t Gdt T e t dt The desred result the follows To study the rate of varato of estmato error, we defe the sequece of scaled estmato error ν = Ĝ G/ Theorem 53 asserts that the scaled estmato error satsfy a dfferetal equato ad provdes a weak covergece results for t Theorem 53: Suppose assumptos of Theorem 52 hold The, for suffcetly small there s a N such that E{ ν,ν }=O1 for all > N Defe the sequece of cotuous-tme terpolato of estmato error as ν t = ν for t [ N, N + 1 Uder the assumptos of Theorem 52, {ν } s tght D[0, ; l 2 Moreover, suppose that ν 0 coverges weakly to ν0, ν coverges weakly to ν such that ν s the soluto of the followg stochastc dfferetal equato dνt = νtdt + dwt 23 Here, Wt = =0 W te ad the covarace operator s gve by E Wt, v Wt, z = t z,ɣv = t =0 σ 2 e,v e, z for v,z l 2, where W s a real-valued Weer process wth covarace ad tσ 2 σ 2 = E[ Y 0 G, e ] E Y 0 G, e Y j G, e j=1 Proof: The proof s preseted Appedx F Note that the covarace σ 2 depeds o the statoary expected degree dstrbuto G ad thus s a fucto of the power law expoet β B Markov-Modulated Probablty Mass Fuctos Wth Deumerable Support Here, we exted the above results to the problem of trackg a tme-varyg probablty mass fucto wth fte support The am s to track a probablty mass fucto wth support o the set of o-egatve tegers that evolves accordg to a slow Markov cha θ wth m states ad tal probablty dstrbuto π 0 The state space M, ad the trasto probablty matrx A ρ of the uderlyg Markov cha θ are defed 1 ad 2, respectvely For each θ M, let Gθ =[G 1 θ, G 2 θ, ], 24 be a probablty mass fucto wth support o the set of o-egatve tegers such that =1 G θ = 1 ad G θ β θ,whereβ θ > 1 Whe the uderlyg Markov cha θ jumps from oe state to aother wth M, Gθ swtches accordgly At each tme, wesampley θ from PMF Gθ ;thats y θ Gθ LetY θ = e y θ deote the observato vector To estmate Gθ, the followg costat step sze stochastc approxmato algorthm s deployed Ĝ +1 = Ĝ + Y θ Ĝ 25 Here >0 deotes a small postve step sze ad Ĝ 0 = e 1 We further assume that the Markov cha s slowly chagg

8 6616 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 that the rate of chages s a order slower tha that of adaptato 25; that s ρ = 2 To aalyze the asymptotc propertes of the stochastc approxmato algorthm, we defe the sequece of cotuous-tme terpolato Ĝ t = Ĝ for t [, + Smlar to what have bee obtaed thus far for the o-markova case, wth the detals omtted, we obta the followg weak covergece results Theorem 54 states that the sequece of estmates obtaed va Hlbert-spacevalued stochastc approxmato algorthm 25 coverges weakly to the soluto of a ODE whch depeds o the tal dstrbuto of the uderlyg Markov cha Theorem 54: Suppose Assumptos 21 ad 41 hold The Ĝ s tght D[0, : l 2 Ay coverget subsequece has a lmt Ĝ that s the soluto of the dfferetal equato where dĝt dt Gθ = = m Gθp θ Ĝt, Ĝ0 = e 1, 26 θ=1 G θe, ad p θ : θ m = π 0 =0 s the tal probablty dstrbuto of Markov cha Proof: The proof s preseted Appedx G Furthermore, we ca obta the followg corollary The proof s smlar to that of Corollary 51 ad thus omtted Corollary 52: Cosder Ĝ +t, where t as 0 Uder the codto of Theorem 54, Ĝ +t G = m θ=1 p θ Gθ probablty as 0 Redefe ν = Ĝ G/ It ca be show that there exsts N such that the sequece {ν : N } s tght Next, redefe ν t = ν for t [ N, N + Wth a lttle more effort, we ca also obta the assocated rates of covergece result, whch s stated the ext theorem Theorem 55: Suppose Assumptos 21 ad 41 hold The, {ν } s tght D[0, ; l 2 Moreover, suppose that ν 0 coverges weakly to ν0, theν coverges weakly to ν such that ν s the soluto of the followg stochastc dfferetal equato SDE m dut = νtdt + p θ dwθ, t, 27 θ=1 where for each θ M, Wθ, s a Weer process as gve Theorem 53 Proof: The proof s smlar to the proof of Theorem 53 wth modfcatos smlar to those of the proof of Theorem 54 VI NUMERICAL EXAMPLES I ths secto, umercal examples are gve to llustrate the results from SecII, ad SecIV The ma coclusos are: The fte duplcato-deleto radom graph wthout Markova dyamcs geerated by the duplcatodeleto Procedure 1 satsfes a power law as stated Theorem 32; see Example 1 Fg 1 The degree dstrbuto of the duplcato-deleto radom graph satsfes a power law The parameters are specfed Example 1 of SecVI Fg 2 The degree dstrbuto of the fxed sze duplcato-deleto radom graph The parameters are specfed Example 2 of SecVI The degree dstrbuto of the fxed sze duplcatodeleto radom graph geerated by the duplcatodeleto Procedure 1 ca be computed from Theorem 31 Whe η 0 the sze of the radom graph s suffcetly large, umercal results show that the degree dstrbuto satsfes a power law as well; see Example 2 The estmates obtaed by stochastc approxmato algorthm 12 follow the expected probablty dstrbuto precsely wthout formato about the Markova dyamcs; see Example 3 Example 1: Cosder a fte duplcato-deleto radom graph wthout Markova dyamcs so m = 1 geerated by Procedure 1 wth p = 05 adq = 01 Theorem 32 mples that the degree sequece of the resultg graph satsfes a power law wth expoet computed usg 9 Fg1 dsplays the u-ormalzed degree dstrbuto o a logarthmc scale The learty Fg1 excludg the odes wth very small degree, mples that the resultg graph from duplcatodeleto process satsfes a power law As ca be see Fg1, the power law s a better approxmato for the mddle pots compared to both eds Example 2: Cosder the fxed sze duplcato-deleto radom graph geerated by Procedure 1 wth r = 0, η 0 = 10, p = 04, ad q = 01 We cosder m = 1 o Markova dyamcs to llustrate Theorem 31 Fg 2 depcts the

9 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH 6617 sold le The fgure shows that the estmates usg by the stochastc approxmato algorthm 12 follow the expected degree dstrbuto 8 satsfactorly eve though the algorthm has o formato about the uderlyg Markova dyamcs Fg 3 Degree dstrbuto of the fxed sze duplcato-deleto radom graph satsfes a power law whe η 0 s suffcetly large The parameters are specfed Example 2 of SecVI Fg 4 The estmates obtaed by SA algorthm 12 follows the expected PMF precsely wth o kowledge of the Markova dyamcs The parameters are specfed Example 3 ormalzed degree dstrbuto of the fxed sze duplcatodeleto radom graph obtaed by Theorem 31 As ca be see Fg 2, the computed degree dstrbuto s close to that obtaed by smulato The umercal results show that the degree dstrbuto of the fxed sze radom graph also satsfes a power law for some values of p whe the sze of radom graph s suffcetly large Fg 3 shows the umber of odes wth specfc degree for the fxed sze radom graph obtaed by Procedure 1 wth r = 0, η 0 = 1000, p = 04, ad q = 01 o a logarthmc scale for both horzotal ad vertcal axes Example 3: Cosder the fxed sze Markov-modulated duplcato-deleto radom graph geerated by Procedure 1 wth r = 0adη 0 = 500 Assume that the uderlyg Markov cha has three states, m = 3 We choose the followg values for probabltes of coecto ad deleto: state 1: p = q = 005, state 2: p = 02 adq = 01, ad state 3: p = 04, q = 015 The sample path of the Markov cha jumps at tmes = 3000 from state 1 to state 2 ad = 6000 from state 2 to state 3 As the state of the Markov cha evolves, the expected degree dstrbuto, Gθ, obtaed by 8 evolves over tme The correspodg values for the expected degree dstrbuto for odes of degree = 3 are dsplayed Fg 4 usg a dotted le The estmated probablty mass fucto, Ĝ, obtaed by the stochastc approxmato algorthm 12 s plotted Fg 4 usg a VII CONCLUSION Markov-modulated duplcato-deleto radom graphs are aalyzed terms of ther degree dstrbuto Whe the sze of graph s fxed r = 0 ad ρ s small, the expected degree dstrbuto of the Markov-modulated duplcatodeleto radom graph ca be computed from 5 for each state of the uderlyg Markov cha Ths result allows us to express the structure of etwork degree dstrbuto terms of the dyamcs of the model We also showed that, the fte duplcato-deleto radom graph wthout Markova dyamcs geerated accordg to Procedure 1 r = 1, m = 1 satsfes a power law wth compoet computed from 9 The mportace of ths result s that a sgle parameter power law compoet characterzes the structure of a possbly very large dyamc etwork Also a stochastc approxmato algorthm was preseted to adaptvely estmate the degree dstrbuto of radom graphs The stochastc approxmato algorthm 12 does ot assume kowledge of the Markov-modulated dyamcs of the graph Theorem 41 showed that the trackg error of the stochastc approxmato algorthm s small ad s order of O As a result of ths boud, we showed that the scaled trackg error weakly coverges to a dffuso process Motvated by the aalyss of socal etworks, we preseted a Hlbert-spacevalued stochastc approxmato algorthm to estmate the expected degree dstrbuto of the fte duplcato-deleto radom graph wthout Markova dyamcs The asymptotc behavour of such a algorthm s aalyzed terms of the power law degree dstrbuto Fally, we exteded the aalyss to a Hlbert-space-valued stochastc approxmato algorthm that ams to track a Markov-modulated probablty mass fucto wth deumerable support Usg weak covergece methods, t was show that the estmates obtaed va such a algorthm coverge weakly to the soluto of a ordary dfferetal equato It was also show that the terpolated sequece of scaled trackg error coverges weakly to the soluto of a stochastc dfferetal equato APPENDIX A Proof of Theorem 31 The proof s based o the proof of [9, Lemma 41, Ch 4, p 79] To compute the expected degree dstrbuto of the Markov-modulated radom graph, we fd a relato betwee the umber of odes wth specfc degree at tme ad the degree dstrbuto of the graph at tme 1 Recall that the -th elemet of F, F, deotes the umber of vertces wth degree at tme k Gve the resultg graph at tme, the am s to fd the expected umber of odes wth degree + 1 at tme + 1 The followg evets ca occur that result a ode wth degree + 1 at tme + 1: Degree of a ode wth degree cremets by oe the duplcato step Step 1 of the duplcato-deleto

10 6618 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 Procedure 1 ad remas uchaged the deleto step Step 2: A ode wth degree s chose at the duplcato step as a paret ode ad remas uchaged the deleto step The probablty of occurrece of such a evet s r 1 q +1 + q1+ p q1+ p +1/η F ; the probablty of choosg a ode wth degree s F ad the probablty of the evet that ths ode remas uchaged the deleto step s 12 1 q q1 + p q1 + p + 1/ Oe eghbor of a ode wth degree s selected as a paret ode; the paret ode coects to ts eghbors cludg the ode wth degree wth probablty p the edge-duplcato part of Step 1 The probablty of such a evet s r F p η 1 q +2+q1+ p +1 q1+ p +1 +2/η Note that the ode whose degree s cremeted by oe ths evet should rema uaffected Step 2; the probablty of beg uchaged Step 2 for such a ode s 1 q +2+q1+ p +1 q1+ p +1 +2/ A ode wth degree + 1 remas uchaged both Step 1 ad Step 2 of Procedure 1: Usg the same argumet as above, the probablty of such a evet s F +1 1 q 1 r p p+1+2 p A ew ode wth degree + 1 s geerated Step 1: The degree of the most recetly geerated ode the vertex- duplcato part of Step 1 cremets 12 The deleto step Step 2 of Procedure 1 comprses a edge-deleto step ad a duplcato step The probablty that the degree of ode wth degree chages the edge-deleto step s q+1 ; ether ths ode or oe of ts eghbors should be selected the edge-deleto step Also gve that the degree of ths ode dose ot chage the edge-deleto step, f ether ths ode or oe of ts eghbor s selected the duplcato step wth Step 2 the the degree of ths ode cremets by oe wth probablty 1+p Therefore, the probablty that the degree of a ode of degree remas uchaged Step 2 s 1 q q1 + p q1 + p + 1/ Note that for smplcty our aalyss, t s assumed that the odes whose degrees chage the edge-deleto part of Step 2, rema uchaged the duplcato part of Step 2 at that tme stat Also, the ew ode, whch s geerated the vertex-duplcato step of Step 1, remas uchaged Step 2 to + 1; the ew ode coects to " eghbors of the paret ode ad remas uchaged Step 2 The probablty of ths scearo s 1+p p + 1 η r 1 q j F j j p 1 p j Degree of a ode wth degree + 2 decremets by oe Step 2: A ode wth degree + 2 remas uchaged the duplcato step ad oe of ts eghbors s elmated the deleto step The probablty of ths evet s + 2 q 1 p A ode wth degree + 1 s geerated Step 2: The degree of the ode geerated the vertexduplcato part of duplcato step wth Step 2 cremets to + 1 The probablty of ths evet s q j 1 F j j p 1 p j Degree of a ode wth degree cremets by oe Step 2: A ode wth degree remas uchaged Step 1 ad ts degree cremets by oe the duplcato part of Step 2 The correspodg probablty s q1 + p p Let deote the set of all arbtrary graphs ad F deote the sgma algebra geerated by graphs G τ,τ Cosderg the above evets that result a ode wth degree + 1 at tme + 1, the followg recurrece formula ca be derved for the codtoal expectato of F : E{F F } 1+p p + 1 η = 1 q 1 r + r p F +1 1 q q1 + p q1 + p + 1/ 1+ p j + q j F j 1+p+1+2 F 1 q +r +3+ p +1 j p 1 p j F j j p 1 p j + q + 2

11 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH p +2+1 F +2 + q1+ p 1+ p 1 F η 28 Let F = E{F } By takg expectato of both sdes of 28 wth respect to trval sgma algebra {, }, the smoothg property of codtoal expectatos yelds: F = 1 q 1 r + r p +3+ p +1 1+p+1+2 p +1+1 F +1 q q1 + p q1+p+1 1+p+1+2 F 1 q + r +3+ p +1 F j j η j + 2 p q 1 η q1 + p p p 1 p j + q j 1 F j F +2 j p 1 p j F 29 Assumg that sze of the graph s suffcetly large, each term ca be eglected Eq 29 ca be wrtte as F = 1 q r + q p lke F η p r + q F + q + 2 F +1 F +2 j p 1 p j 30 + q 1 F j j Usg 29, we ca wrte the followg recurso for the + 1-th elemet of G +1 : G +1 η +1 = q r + q p G p r + q + +1 G + q G +2 j p 1 p j 31 + q 1 G j +1 j Sce the probablty of duplcato step r = 0, the umber of vertces does ot crease Thus, = η 0 ad 31 ca be wrtte as G = 1 1 q q p G +1 η pqg η + 1 q + 2G +2 0 η η 0 q j G j j p 1 p j 32 It s clear 32 that the vector G +1 depeds o elemets of G Usg matrx otato, 32 ca be expressed as G +1 = I + 1η0 L G 33 where L s defed as 6 To prove that L s a geerator, we eed to show that l < 0 ad η 0 =1 l k = 0 Accordgly, η 0 l k = qk q1 + pk pkq =1 + qk + q k p 1 1 p k +1 1 k 1 = q + q k p 1 1 p k k 1 Let m = 1 The, 34 ca be rewrtte as η 0 l k = q + q =1 k m=0 = q + q1 p k k p m 1 p k m m k k p m 35 m 1 p m=0 Kowg that k m=0 k m a m = 1 + a k, 35 ca be wrtte as η 0 =1 1 k l k = q + q1 p k = p Also, t ca be show that l < 0 Sce p 1 1, p 1 < p+ p Cosequetly, qp 1 1 p q +2+p<0 Therefore, l < 0 ad the desred result follows B Proof of Theorem 41 Defe the Lyapuov fucto V x = x x/2 forx R N 0 Use E to deote the codtoal expectato wth respect to the σ -algebra H geerated by {Y j θ j, θ j, j } The, E {V G +1 V G } = E { G [ G + Y θ Gθ } +Gθ Gθ +1 ] { + E G + Y θ Gθ +Gθ Gθ +1 2} 37 where Y θ ad Gθ are vectors R N 0 wth elemets Y θ ad Gθ, 1 N 0, respectvely Due to the Markova assumpto ad the structure of the trasto

12 6620 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 matrx of θ, defed 2, E {Gθ Gθ +1 } = E{Gθ Gθ +1 θ } m = E{G Gθ +1 θ = }I {θ = } = =1 m m G G ja ρ j I {θ =} =1 = ρ m =1 j=1 j=1 m G jq j I {θ = } = Oρ 38 where I { } deotes the dcator fucto Smlarly, t s easly see that E { Gθ Gθ +1 2 }=Oρ 39 Usg K to deote a geerc postve value wth the otato KK = K ad K + K = K, a famlar equalty ab a2 +b 2 2 yelds Oρ = O 2 + ρ 2 40 Moreover, we have G = G 1 G 2 + 1/2 Thus, Oρ G Oρ V G The, detaled estmates lead to { E G + Y θ Gθ + Gθ Gθ +1 2 } { K E 2 G Y θ Gθ G Y θ Gθ +1 + G Gθ Gθ +1 + Y θ Gθ Gθ Gθ +1 } + E { Gθ Gθ +1 } 2 42 It follows that { E G + Y θ Gθ + Gθ Gθ +1 2} = O 2 + ρ 2 V G Furthermore, {V G +1 V G } = 2V G + E { G [Y θ Gθ ]} + E { G [Gθ +1 Gθ ]} + O 2 + ρ 2 V G To obta the desred boud, defe V ρ 1 ad V ρ 2 as follows: V ρ 1 G, = j= G E {Y j θ j Gθ j }, V ρ 2 G, = j= G E {Gθ j Gθ j+1 } 45 It ca be show that V ρ 1 G, =OV G + 1, V ρ 2 G, =OρV G Defe W G, as Ths leads to W G, = V G + V ρ 1 G, + V ρ 2 G, 47 E {W G +1, + 1 W G, } = E {V ρ 1 G +1, + 1 V ρ 1 G, } + E {V G +1 V G } + E {V ρ 2 G +1, + 1 V ρ 2 G, } 48 Moreover, E {W G +1, +1 W G, }= 2V G +O 2 +ρ 2 V G Equato 49 ca be rewrtte as E {W G +1, + 1 W G, } 2W G, + O 2 + ρ 2 W G, If ad ρ are chose small eough, the there exsts a small λ such that 2 + Oρ 2 + O 2 λ Therefore, 50 ca be rearraged as E {W G +1, + 1} 1 λw G, +O 2 + ρ 2 51 Takg expectato of both sdes yelds E{W G +1, + 1} 1 λe{w G, } +O 2 + ρ 2 52 Iteratg o 52 the results E{W G +1, + 1} 1 λ N ρ E{W G Nρ, N ρ } + O 2 +ρ 2 1 λ j Nρ 53 j=n ρ As a result, E{W G +1, + 1} 1 λ N ρ E{W G Nρ, N ρ } +O + ρ 2 / 54 If s large eough, oe ca approxmate 1 λ N ρ = O Therefore, E{W G +1, + 1} O + ρ2 55 Fally, usg 46 ad replacg W G +1, + 1 wth V G +1, we obta E{V G +1 } O ρ + + ρ2 56

13 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH 6621 C Sketch of the Proof of Theorem 42 Sce the proof s smlar to [18, Th 45], we oly dcate the ma steps what follows ad omt most of the verbatm detals Step 1: Frst, we show that the two compoet process Ĝ, θ s tght D[0, T ]:R η 0 m Usg techques smlar to [43, Th 43], t ca be show that θ coverges weakly to a cotuous-tme Markov cha geerated by Q Thus, we maly eed to cosder Ĝ We show that [ Ĝ t + s Ĝ t ] 2 = 0 57 lm 0 lm sup E 0 sup E t 0 s where E t deotes the codtog o the past formato up to t The, the tghtess follows from the crtero [35, p 47] Step 2: Sce Ĝ, θ s tght, we ca extract weakly coverget subsequece accordg to the Prohorov theorem; see [1] To fgure out the lmt, we show that Ĝ, θ s a soluto of the martgale problem wth operator L 0 For each M ad cotuously dfferetal fucto wth compact support f,, the operator s gve by L 0 f Ĝ, = f Ĝ, [ Ĝ + G] + q j f Ĝ, j, M 58 j M We ca further demostrate the martgale problem wth operator L 0 has a uque soluto the sese of dstrbuto Thus, the desred covergece property follows D Sketch of the Proof of Theorem 43 The proof comprses of four steps as descrbed below: Step 1: Frst, ote ν +1 = ν ν + y +1 EGθ + E[Gθ Gθ +1 ] 59 The approach s smlar to that of [18, Th 56] Therefore, we wll be bref Step 2: Defe a operator L f ν, = f ν, ν tr[ 2 f ν, ] + q j f ν, j, M, 60 j M for fucto f, wth compact support that has cotuous partal dervatves wth respect to ν up to the secod order It ca be show that the assocated martgale problem has a uque soluto the sese of dstrbuto Step 3: It s atural ow to work wth a trucated process For a fxed, but otherwse arbtrary r 1 > 0, defe a trucato fucto { q r 1 1, f x S r 1, x = 0, f x R η 0 S r 1, where S r 1 = {x R η 0 : x r 1 } The, we obta the trucated terates ν r 1 +1 = νr 1 νr 1 qr 1 ν r 1 + y +1 EGθ + E[Gθ Gθ +1 ] q r 1 ν r 1 61 Defe ν,r 1t = ν r 1 for t [, + The, ν,r 1 s a r-trucato of ν ; see [1, p 284] for a defto We the show the trucated process ν,r 1, θ s tght Moreover, by Prohorov s theorem, we ca extract a coverget subsequece wth lmt ν r 1, θ such that the lmt ν r 1, θ s the soluto of the martgale problem wth operator L r 1 defed by L r 1 f r 1 ν, = f r 1 ν, ν tr[ 2 f r 1 ν, ] + q j f r 1 ν, j 62 j M for M, where f r 1 ν, = f ν, q r 1 ν Step 4: Lettg r 1, we show that the u-trucated process also coverges ad the lmt, deoted by ν, θ, s precsely the martgale problem wth operator L defed 62 The lmt covarace ca further be evaluated as [18, Lemma 52] E Proof of Theorem 52 The proof of the theorem s dvded to several steps ad uses techques stochastc approxmato [1] but wth the modfcato that l 2 s a Hlbert space see [41], [42] Wheever possble, we oly dcate the ma dea ad refer to the lterature of stochastc approxmato Step 0: Note that 22 has a uque soluto for each tal codto sce t s lear Ĝ Step 1: Prelmary estmates From 21, we obta that for 0 <<1, the elemets of Ĝ are o-egatve ad add up to oe Thus, Ĝ s bouded I addto, defe V Ĝ = 1 Ĝ 2 G, Ĝ G, whch ca be thought of as a Lyapuov fucto The usg perturbed Lyapuov fucto argumet [1], t ca be show EV Ĝ = O 63 Step 2: Tghtess of {Ĝ } Heceforth, we ofte use t/ ad t +s/ to deote t/ ad t +s/, the teger parts of t/ ad t + s/, respectvely By usg the boudedess of {Ĝ } establshed the frst step together wth the Hölder equalty, we have for each 0 < T <, ayt 0, ay 0 <δ,ay0< s δ, ad>0, t+s/ 1 2 E t Ĝ t + s Ĝ t 2 E t [Y j Ĝ j ] j=t/ t + s K t Ks, where K > 0 s depedet of ad E t deotes the codtoal expectato wth respect to Ft Thus lm δ 0 E t Ĝ t + s Ĝ t 2 Ks, lm sup E[ sup E t Ĝ t + s Ĝ t 2 ]= <s δ The tghtess crtero see [35, Th 3, p 47] wth R r replaced by l 2 ; see also [41] eables us to coclude that {Ĝ } s tght D[0, : l 2

14 6622 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 Step 3: Characterzato of the lmt process Sce {Ĝ } s tght, by Prohorov s theorem, we ca extract a coverget subsequece Select such a sequece ad stll deote t by Ĝ wth lmt deoted by Ĝ By usg the Skorohod represetato, wth a slght abuse of otato, we may assume that Ĝ coverges to Ĝ wp1 ad the covergece s uform o ay bouded tme terval We shall show that Ĝ s a soluto of the martgale problem wth operator L f Ĝ = f Ĝ, [G Ĝ] for ay f C0 1l 2 : R collecto of real-valued C 1 fuctos defed o l 2 wth compact support We eed to show that t f Ĝt f Ĝ0 L f Ĝτdτ s a margale 0 To prove the martgale property, we pck out ay bouded ad cotuous fucto h defed o l 2,ayT <, ay 0 < t, s T, ay postve teger κ, adt l1 t for ay l κ To derve the desred property, we eed oly show that EhĜt l1 : l 1 κ f Ĝt +s f Ĝt t+s t L f Ĝτdτ = 0 65 To prove 65, we work wth the process dexed by Frst, by the weak covergece ad the Skorohod represetato, lm 0 EhĜ t l1 : l 1 κ[ f Ĝ t + s f Ĝ t] = EhĜt l1 : l 1 κ[ f Ĝt + s f Ĝt] 66 Choose a sequece of tegers {m } such that m as 0but = m 0 Next, we ote f Ĝ t + s f Ĝ t = f Ĝ t+s/ f Ĝ t/ = = = t+s/ 1 lm =t/ t+s/ 1 lm =t/ t+s/ 1 lm =t/ [ f Ĝ lm +m f Ĝ lm ] lm +m 1 f Ĝlm, [Y j Ĝ j ] + o1 f Ĝlm, 1 lm +m 1 m [Y j Ĝ j ] + o1, where o1 0 probablty as 0 The statoarty ad the mxg codto mply that 1 m lm +m 1 E lm Y j EY 0 = e Py 0 = =0 = G probablty Therefore, EhĜ t l1 : l 1 κ [ t+s/ 1 f Ĝlm, 1 lm +m 1 ] Y j m lm =t/ = EhĜ t l1 : l 1 κ [ t+s/ 1 f Ĝlm, 1 lm +m 1 ] E lm Y j m lm =t/ EhĜt l1 : l 1 κ t+s f Ĝτ, G dτ t Lkewse, EhĜ t l1 : l 1 κ [ t+s/ 1 f Ĝlm, lm =t/ [ EhĜt l1 : l 1 κ Combg 66 68, 65 follows as 0 67 lm +m 1 ] Ĝ j m t+s f Ĝτ, Ĝτ dτ t ] 68 F Proof of Theorem 53 I the proof of Theorem 53, we use several lemmas ad propostos descrbed below From 21, ν +1 = ν ν + Y G 69 Lemma 71: Uder assumpto Theorem 52, for suffcetly small, theresan such that EV ν = O1 for all N Proof: The proof uses a perturbed Lyapuov fucto argumet To proceed, recall the defto of covarace operator ad Weer process [42], [44] o l 2 A covarace Ɣ of a l 2 -valued radom varable y s a operator from l 2 to l 2 defed by Ɣv = EY v, y for ay v l 2 A process W s a zero mea statoary cremet l 2 -valued Weer process f there are mutually depedet real-valued, zero mea, Weer processes {W } wth covaraces tρ satsfyg =0 ρ < ad there s a orthoormal sequece {β } wth β l 2 such that Wt = =0 W tβ Forv,z l 2,the covarace operator of Wt s defed by E Wt, v Wt, z = t z,ɣv = t ρ β,v β, z 70 Lemma 72: Assume the codtos of Theorem 52 For ay atural umber N, defe W t = t/ 1 j=0 =0 Y j G, e The W coverges weakly to a real-valued Weer process W wth covarace tσ 2,where σ 2 = E[ Y 0 G, e ] E Y 0 G, e Y j G, e 71 j=1

15 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH 6623 Proof: Note that wth the use of er product l 2, { Y G, e } s a real-valued mxg sequece wth mea 0 The desred covergece follows from the fuctoal varace prcple for mxg process; see [1, Ch 7] see also [36], [41] Lemma 73: Uder the codtos of Lemma 72, for = l, EW tw l t = 0 As a result, the lmt Weer processes W ad W l are depedet Proof: It s straghtforward that t/ 1 EW tw l t = E = E k=0 t/ 1 k=0 t/ 1 j=0 t/ 1 j=0 Y j G, e Yk G, e l Y j G, e e l Y k G = 0 sce e e l = 0 R Sce EW t = 0, we coclude that W l t = 0 Cosequetly, W t, W l t = 0, ad as a result W t ad W l t are depedet Weer processes Proposto 74: Uder the codtos of Lemma 72, defe W t = t/ 1 [Y j G] 72 j=0 The W coverges weakly to W such that Wt = W te, 73 =0 ad the covarace operator s gve by E Wt, v Wt, z = t z,ɣv = t =0 σ 2 e,v e, z for v,z l 2, 74 where σ 2 s defed 71 Proof: I vew of the defto of 72, for ay δ>0, t > 0, 0 < s δ, wth E t deotes the codtoal expectato wth respect to Ft, usg the mxg propertes, we ca show that lm δ 0 [ lm sup 0 sup 0 δ s E t W t + s W t, W t + s W t ] = 0 Thus W s tght D[0, ; l 2 We ca extract ay weakly coverget subsequece ad deote the lmt by W We ext characterze ts lmt Aga, usg 72 W t = =0 W te = Therefore, for each l N, =0 t/ 1 j=0 Y j G, e e E[ W t, e l ] 2 = E W l t 2 = tσ 2 l By vrtue of expoetal decay property of G β, l=0 σl 2 < By Lemma 72, W coverges weakly to W By vrtue of Lemma 73, W are depedet Weer processes I vew of the defto of Weer process o l 2, we coclude that W coverges weakly to W such that 73 holds I addto, the structure of the covarace operator 74 s obtaed We proceed to obta the desred weak covergece of ν Sce the stochastc dfferetal equato 23 s lear, there s a uque soluto for each tal codto The rest of the proof s smlar to the fte dmesoal couter part wth ecessary modfcatos smlar to that of the proof of Theorem 52 G Proof of Theorem 54 Before proceedg to the ma proof, we frst state a prelmary result The proofs of the assertos below ca be foud [43, Ths 36 ad 43] ad are thus omtted Lemma 75: Uder Assumpto 21, the followg clams hold: a Deote p ρ = [Pθ ρ = 1,,Pθ ρ = m] ad the -step trasto probablty by A ρ wth A ρ gve 2 wth ρ = 2 The p ρ = pρ + Oρ + ρ k0t/ρ, A ρ 0 = ρ,ρ 0 + Oρ + e k 0t t 0 /ρ, 75 where pt R 1 m ad t, t 0 R m m are the cotuous-tme probablty vector ad trasto matrx satsfyg dpt = ptq, p0 = p 0, dt d t, t 0 = t, t 0 Q, t 0, t 0 = I, 76 dt wth t 0 = ρ 0 ad t = ρ b θ ρ coverges weakly to θ, a cotuous-tme Markov cha geerated by Q To aalyze the algorthm, the techques developed the proof of Theorem 52 are used alog wth the deas ad methods developed [45] The developmets are smlar the approach ad the results, but are more complex due to the added swtchg process For example, wth modfcatos, Step 1 the proof of Theorem 52 ca stll be carred out Also Step 2 ca be proved So the sequece {Ĝ } s tght To characterze the lmt, we stll use martgale averagg techques We shall oly hghlght the ma dfferece here I carryg out the aalyss smlar to that of Step 3 the proof of Theorem 52, we wll ecouter the followg term EhĜ t l1 : l 1 κ [ t+s/ 1 f Ĝlm, 1 lm +m 1 ] Y j θ j m lm =t/ = EhĜ t l1 : l 1 κ [ t+s/ 1 f Ĝlm, 1 lm =t/ m lm +m 1 ] E lm Y j θ j

16 6624 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 60, NO 10, OCTOBER 2014 = EhĜ t l1 : l 1 κ [ t+s/ 1 f Ĝlm, lm =t/ 1 m m θ=1 lm +m 1 Sce Y j θ ad θ j are depedet, we have 1 m m θ=1 lm +m 1 ] E lm Y j θi {θ j =θ} 77 E lm Y j θi {θ j =θ} 78 = 1 m m lm +m 1 E lm Y j θpθ j =θ θ lm =ι 0 I {θlm =ι m 0 } ι 0 =1 θ=1 79 For each θ M, the averagg of Y j θ ca be carred out as Case 1 We cocetrate o the term volvg Markov cha By vrtue of Lemma 75, otg ρ = 2 ad usg 75, we have [A ρ ] j lm = 2 j, 2 lm + O 2 + e k 0 2 j 2 lm / 2 Because we are workg wth 25 ad the stepsze s I the terval [l, l + wth = m, t s readly see that 2 j, 2 lm 0, 0 = I as 0 As a result, Pθ j = θ θ lm = ι 0 + o 1 = δ ι0,θ + o 1 { 1, f ι0 = θ = 0, otherwse + o 1, where o1 0aso 1 0as 0 Puttg the above estmates 78, we obta the lmt probablty of 1 m lm +m 1 E lm Y j θpθ j = θ θ lm = ι 0 I {θlm =ι 0 } s the same as that of e θ θδ ι0 θ I {θ 2 2 lm =θ 0 } =1 Ths further leads to that as 0, EhĜ t l1 : l 1 κ [ t+s/ 1 f Ĝlm, 1 lm +m 1 ] Y j θ j m lm =t/ EhĜt l1 : l 1 κ [ t+s f Ĝτ, θ θpθ0 = θ ] dτ t [ t+s ] = EhĜt l1 : l 1 κ f Ĝτ, θ θe p θ dτ t REFERENCES [1] H J Kusher ad G Y, Stochastc Approxmato ad Recursve Algorthms ad Applcatos Stochastc Modelg ad Appled Probablty, vol 37, 2d ed New York, NY, USA: Sprger-Verlag, 2003 [2] X G Doukopoulos ad G V Moustakdes, Bld adaptve chael estmato OFDM systems, IEEE Tras Wreless Commu, vol 5, o 7, pp , Jul 2006 [3] X G Doukopoulos ad G V Moustakdes, Fast ad stable subspace trackg, IEEE Tras Sgal Process, vol 56, o 4, pp , Apr 2008 [4] X Xe ad R J Evas, Multple target trackg ad multple frequecy le trackg usg hdde Markov models, IEEE Tras Sgal Process, vol 39, o 12, pp , Dec 1991 [5] A Beveste, M Métver, ad P Prouret, Adaptve Algorthms ad Stochastc Approxmatos New York, NY, USA: Sprger-Verlag, 2012 [6] G G Y ad V Krshamurthy, LMS algorthms for trackg slow Markov chas wth applcatos to hdde Markov estmato ad adaptve multuser detecto, IEEE Tras If Theory, vol 51, o 7, pp , Jul 2005 [7] V Krshamurthy ad T Ryde, Cosstet estmato of lear ad o-lear autoregressve models wth Markov regme, J Tme Ser Aal, vol 19, o 3, pp , May 1998 [8] F Chug, L Lu, T G Dewey, ad D J Galas, Duplcato models for bologcal etworks, J Comput Bol, vol 10, o 5, pp , Oct 2003 [9] F Chug ad L Lu, Complex Graphs ad Networks Arlgto, VA, USA: NSF, 2006 [10] D López-Ptado, Dffuso complex socal etworks, Games Eco Behavor, vol 62, o 2, pp , Mar 2008 [11] F Vega-Rededo, Complex Socal Networks Ecoomc Socety Moographs Cambrdge, UK: Cambrdge Uv Press, 2007 [12] M O Jackso, Socal ad Ecoomc Networks Prceto, NJ, USA: Prceto Uv Press, 2008 [13] S Eubak et al, Modellg dsease outbreaks realstc urba socal etworks, Nature, vol 429, o 6988, pp , May 2004 [14] M E J Newma, Assortatve mxg etworks, Phys Rev Lett, vol 89, p , Oct 2002 [15] M E J Newma, D J Watts, ad S H Strogatz, Radom graph models of socal etworks, Proc Nat Acad Sc USA, vol 99, o S1, pp , 2002 [16] V Krshamurthy, K Topely, ad G Y, Cosesus formato a two-tme-scale Markova system, Multscale Model Smul, vol 7, o 4, pp , 2009 [17] V Solo ad X Kog, Adaptve Sgal Processg Algorthms: Stablty ad Performace Iformato ad System Sceces Eglewood Clffs, NJ, USA: Pretce-Hall, 1995 [18] G Y, V Krshamurthy, ad C Io, Regme swtchg stochastc approxmato algorthms wth applcato to adaptve dscrete stochastc optmzato, SIAM J Optm, vol 14, o 4, pp , 2004 [19] V Krshamurthy, O N Gharehshra, ad M Hamd, Iteractve sesg ad decso makg socal etworks, Foud Treds Sgal Process, vol 7, os 1 2, pp 1 196, 2014 [20] O N Gharehshra, V Krshamurthy, ad G Y, Dstrbuted trackg of correlated equlbra regme swtchg ocooperatve games, IEEE Tras Autom Cotrol, vol 58, o 10, pp , Oct 2013 [21] G V Moustakdes, Expoetal covergece of products of radom matrces: Applcato to adaptve algorthms, It J Adapt Cotrol Sgal Process, vol 12, o 7, pp , 1998 [22] L Guo, L Ljug, ad G-J Wag, Necessary ad suffcet codtos for stablty of LMS, IEEE Tras Autom Cotrol, vol 42, o 6, pp , Jul 1997 [23] J Reaso ad W Re, Estmatg the optmal adaptato ga for the LMS algorthm, Proc 32d IEEE Cof Decso Cotrol, Dec 1993, pp [24] R Durrett, Radom Graph Dyamcs Statstcal ad Probablstc Mathematcs Cambrdge, UK: Cambrdge Uv Press, 2007 [25] P Erdös ad A Réy, O the evoluto of radom graphs, Publcato Math Ist Hugara Acad Sc, vol 5, pp 17 61, 1960 [26] E Leberma, C Hauert, ad M A Nowak, Evolutoary dyamcs o graphs, Nature, vol 433, o 7023, pp , Ja 2005

17 HAMDI et al: TRACKING A MARKOV-MODULATED STATIONARY DEGREE DISTRIBUTION OF A DYNAMIC RANDOM GRAPH 6625 [27] R Pastor-Satorras, E Smth, ad R V Solé, Evolvg prote teracto etworks through gee duplcato, J Theoretcal Bol, vol 222, o 2, pp , May 2003 [28] M E J Newma, Power laws, Pareto dstrbutos ad Zpf s law, Cotemporary Phys, vol 46, o 5, pp , 2005 [29] A-L Barabás ad A Reka, Emergece of scalg radom etworks, Scece, vol 286, o 5439, pp , 1999 [30] B Bollobás, O Rorda, J Specer, ad G Tusády, The degree sequece of a scale-free radom graph process, Radom Struct Algorthms, vol 18, o 3, pp , 2001 [31] C Cooper ad A Freze, A geeral model of web graphs, Radom Struct Algorthms, vol 22, o 3, pp , 2003 [32] J Shrager, T Hogg, ad B A Huberma, A graph-dyamc model of the power law of practce ad the problem-solvg fa-effect, Scece, vol 242, o 4877, pp , 1988 [33] S H Strogatz, Explorg complex etworks, Nature, vol 410, o 6825, pp , 2001 [34] M Hamd, V Krshamurthy, ad G Y Feb 2013 Trackg the emprcal dstrbuto of a Markov-modulated duplcato-deleto radom graph [Ole] Avalable: [35] H J Kusher, Approxmato ad Weak Covergece Methods for Radom Processes, Wth Applcatos to Stochastc Systems Theory Cambrdge, MA, USA: MIT Press, 1984 [36] P Bllgsley, Covergece of Probablty Measures, vol 493 New York, NY, USA: Wley, 2009 [37] D D Heckathor, Respodet-drve samplg: A ew approach to the study of hdde populatos, Soc Problems, vol 44, o 2, pp , May 1997 [38] S Goel ad M J Salgak, Respodet-drve samplg as Markov cha Mote Carlo, Statst Med, vol 28, o 17, pp , Jul 2009 [39] A Beveste, M Métver, ad P P Prouret, Adaptve Algorthms ad Stochastc Approxmatos New York, NY, USA: Sprger-Verlag, 1990 [40] G Y ad C Zhu, Hybrd Swtchg Dffusos: Propertes ad Applcatos Stochastc Modelg ad Appled Probablty, vol 63 New York, NY, USA: Sprger-Verlag, 2010 [41] S N Ether ad T G Kurtz, Markov Processes: Characterzato ad Covergece, vol 282 New York, NY, USA: Wley, 2009 [42] H J Kusher ad A Shwartz, Stochastc approxmato Hlbert space: Idetfcato ad optmzato of lear cotuous parameter systems, SIAM J Cotrol Optm, vol 23, o 5, pp , 1985 [43] G Y ad Q Zhag, Dscrete-Tme Markov Chas: Two-Tme-Scale Methods ad Applcatos Applcatos of Mathematcs, vol 55 New York, NY, USA: Srpger-Verlag, 2005 [44] R F Curta ad A J Prtchard, Ifte Dmesoal Lear Systems Theory Lecture Notes Cotrol ad Iformato Sceces, vol 8 New York, NY, USA: Sprger-Verlag, 1978 [45] G Y, Y Su, ad L Y Wag, Asymptotc propertes of cosesustype algorthms for etworked systems wth regme-swtchg topologes, Automatca, vol 47, o 7, pp , 2011 Mazyar Hamd was bor 1986 He receved hs BSc from Sharf Uversty of Techology, Tehra, Ira 2007 ad MSc from the Uversty of Brtsh Columba, Vacouver, Caada 2010, both Electrcal ad Computer Egeerg He s curretly workg toward hs PhD degree at Statstcal Sgal Processg Lab, Uversty of Brtsh Columba Hs curret research terests clude stochastc optmzato, socal learg, statstcal sgal processg o radom graphs, ad teractve sesg ad decso makg socal etworks Vkram Krshamurthy S 90 M 91 SM 99 F 05 was bor 1966 He receved hs PhD from the Australa Natoal Uversty, Caberra, 1992 He s curretly a professor ad holds the Caada Research Char at the Departmet of Electrcal ad Computer Egeerg, Uversty of Brtsh Columba, Caada Hs curret research terests clude socal etworks, computatoal game theory, stochastc optmzato ad schedulg I 2009 ad 2010 he served as Dstgushed lecturer for the IEEE sgal processg socety He has served as Edtor--Chef of IEEE JOURNAL SELECTED TOPICS IN SIGNAL PROCESSING I 2013 he was awarded a hoorary doctorate from KTH Royal Isttute of Techology, Swede He s coauthor of the moograph Iteractve Sesg ad Decso Makg Socal Networks publshed Foudatos ad Treds Sgal Processg 2014 George Y F 02 receved the PhD Appled Mathematcs from Brow Uversty ad joed Waye State Uversty 1987 Hs research terests clude stochastc systems ad applcatos He was Co-Char of SlAM Coferece o Cotrol ad Its Applcato 2011, Co-orgazer of 2006 IMA PI coferece ad 2005 IMA Workshop o Wreless Commucatos, ad Co-Char of 1996 AMS-SIAM Summer Semar Appled Mathematcs ad 2003 AMS-IMS-SIAM Summer Research Coferece He has bee servg as Char, Vce Char, ad Program Drector of SlAM Actvty Group o Cotrol ad Systems Theory the past fve years I addto to charg a umber of award commttees, he s a assocate edtor of SlAM Joural o Cotrol ad Optmzato, was a assocate edtor of Automatca ad IEEE TRANSACTIONS ON AUTOMATIC CONTROL, ad s o the edtoral board of a umber of other jourals He was presdet of Waye State Uversty s Academy of scholars