Preprins, 1s IFAC Conference on Modelling, Idenificaion and Conrol of Nonlinear Sysems June 24-26, 2015. Sain Peersburg, Russia Simulaneous Perurbaion Sochasic Approximaion in Decenralized Load Balancing Problem Naalia Amelina, Vicoria Erofeeva, Oleg Granichin, Nikolai Malkovskii Sain Peersburg Sae Universiy (Faculy of Mahemaics and Mechanics), S. Peersburg, Russia e-mail: ngranichina@gmail.com, vicki.ulramarine@gmail.com, malkovskynv@gmail.com Sain Peersburg Sae Universiy (Faculy of Mahemaics and Mechanics, and Research Laboraory for Analysis and Modeling of Social Processes), Insiue of Problems in Mechanical Engineering, Russian Academy of Sciences, and ITMO Universiy, S. Peersburg, Russia e-mail: oleg granichin@mail.ru Absrac: In his work he load balancing problem is sudied for decenralized sochasic nework wih unknown bu bounded noise in measuremens and varying produciviies of agens. The load balancing problem is formulaed as a consensus problem in a sochasic nework. Consideraion of Laplasian poenial funcion corresponded o he nework graph allows o inroduce a new randomized local voing proocol wih consan sep-size which is based on simulaneous perurbaion sochasic approximaion algorihm. The condiions are formulaed for he approximae consensus achievemen which corresponds o achieving of a subopimal level of agens load. The new algorihm is illusraed by simulaions. Keywords: Simulaneous perurbaion sochasic approximaion, randomized algorihms, muliagen sysems, consensus problem. 1. INTRODUCTION In recen years he consensus approach has been widely used for solving differen pracical problems Olfai-Saber and Murray (2004); Olfai-Saber e al. (2007); Ren e al. (2007); Ren and Beard (2008); Cheboarev and Agaev (2009); Kar and Moura (2009); Granichin e al. (2012); Amelin e al. (2013); Lewis e al. (2014), including he load balancing problem Amelina e al. (2015). For he problem of achieving consensus a lo of heoreical resuls were obained. In Tsisiklis e al. (1986); Huang and Manon (2009); Li and Zhang (2009) he sochasic approximaion ype algorihms were used for achieving he consensus, and heir applicabiliy under some saisical uncerainies was analyzed in Amelina and Fradkov (2012); Amelina e al. (2015), where i was assumed ha measuremen noise and delays have a saisical naure wih sandard properies of zeromean and bounded covariance. Emphasize, when he undireced opology graph has a spanning ree, he load balancing problem can be reformulaed as a minimizaion problem of a Laplacian poenial associaed wih a graph (see Olfai-Saber and Murray (2004)). In his paper we sugges o use a simulaneous perurbaion sochasic approximaion (SPSA) for solving his problem. SPSA algorihm recursively generaes esimaes along a random direcions and uses The auhors acknowledge he Russian Minisry of Educaion and Science (agreemen 14.604.21.0035, unique no. RFMEFI60414X0035), RFBR (projecs 13-07-00250, 14-08-01015, and 15-08-02640), and SPbSU (projec 6.37.181.2014). only wo observaions of minimized funcion a each ieraion. SPSA and similar procedures wih one (or wo) measuremens per ieraion were inroduced in Granichin (1989, 1992) Polyak and Tsybakov (1990). and Spall (1992). They are similar o random search mehods Rasrigin (1963). The general overview of SPSA ype algorihms and heir applicaions in differen fields are done in Granichin e al. (2015). Generally, a cenralized algorihm for load balancing which is based on SPSA was considered in Granichin and Amelina (2015); Granichin (2015). The paper is organized as follows. In Secion II, he problem saemen is described, and basic conceps of a graph heory ha are used hereinafer are inroduced. In Secion III, he load balancing conrol sraegy is considered. Secion IV presens a new resul abou a mean-risk opimizaion problem under linear consrains. In Secion V we inroduce he new randomized local voing proocol and Secion VI gives condiions of an asympoic mean square ε-consensus. Simulaion resuls are given in Secion VII. Secion VIII conains conclusions. 2. PROBLEM FORMULATION Le he nework sysem be composed by m agens (processors, machines, ec.) which are numbered by naurals i, i = 1,..., m, and N = {1,..., m} be a se of agens in he sysem. This sysem execues a se of asks of he same ype. Tasks feed o he sysem in differen discree ime insans = 0, 1,... hrough differen agens. Agens perform incoming asks in Copyrigh IFAC 2015 946
June 24-26, 2015. Sain Peersburg, Russia parallel. Tasks can be redisribued among agens based on feedbacks. We assume, ha he ask can no be inerruped afer i has been assigned o he agen. In his paper we use he following noaion and erms from he marix and graph heories. A communicaion graph (N, E) is defined by a se of nodes N and a se of edges E. A dynamic nework of d agens is deermined by a se of dynamic sysems (agens) ha inerac according o he communicaion graph. We associae a weigh a i,j > 0 wih each edge (j, i) E. A graph can be represened by an adjacency marix A = [a i,j ] wih weighs a i,j > 0 if (j, i) E, and a i,j = 0 oherwise. Assume, ha a i,i = 0. We use he noaion G A for a graph which is represened by an adjacency marix A. Define a weighed in-degree of node i as a sum of i-h row of marix A: d i (A) = n j=1 ai,j, and D(A) = diag{d i (A)} as a corresponding diagonal marix. Le L(A) = D(A) A denoes he Laplacian of he graph G A. Noe, ha he sum of rows of he Laplacian equals o zero. The symbol d max (A) sands for a maximum in-degree of he graph G A, Re(λ 2 (A)) is he real par of he second eigenvalue of marix A ordered by absolue magniude, A T is he ranspose marix. Le N i = {j : a i,j > 0} be a neighbors se of agen i N, N i is a corresponding number of neighbors. The graph G A is called undireced if a i,j = a j,i for all i, j N. A each ime insan he behavior of each agen i N is described by wo characerisics: q i is he queue lengh of aomic elemenary asks of agen i a ime insan ; θ i is he produciviy of agen i a ime insan. Here and below, an upper index of agen i is used as a corresponding number of an agen (no as an exponen). The execuion ime of a ask varies from one agen o anoher and depends on a produciviy of an agen. Consider he case when he dynamic model of he sysem is described by he following equaions q i +1 = q i θ i + z i + u i, i N, = 0, 1,..., (1) where z i are amouns of new sysem asks received hrough agen i a ime insan ; u i R are conrol acions (redisribued asks o agen i a ime insan pars of sysem asks previously received hrough oher agens), which could (and should) be chosen. We assume, ha o form he conrol sraegy u i each agen i N has knowledge abou is own produciviy, produciviies of is neighbors and noisy daa abou is own queue lengh: y i,i = q i + ξ i,i, (2) and, if he neighbors se N i is no empy, he knowledge abou produciviies of is neighbors and noisy observaions abou is neighbors queue lenghs: y i,j where {w i,j } is an observaion noise. = q j + ξ i,j, j N i, (3) Denoe T i as a ime momen when agen i complees currenly assigned asks (a ime momen ). T i can be formally described as: T i = min τ τ θk i q. i k= Consider he problem of minimizaion of implemenaion ime of all asks: max T i (q0, i u i 1, z1, i u i 2, z2, i...) min. (4) i {1,...,m} u 1 1,...,um 1,u1 2,... For he saionary case when z i = 0 (i.e. here are no new receiving asks for > 0), such value does no vary over ime and so he problem becomes a wors-case opimizaion problem (moreover, i is easy o show ha he problem can be furher reduced o minimizaion of some good convex funcional). For he nonsaionary case he problem is more difficul as we should race drifing minimum poin. 3. LOAD BALANCING An ideal scheduling algorihm is he one which keeps all he nodes busy execuing essenial asks, and minimizes he inernode communicaion required o deermine he schedule and pass daa beween asks. The scheduling problem is paricularly challenging when he asks are generaed dynamically and unpredicably in he course of execuing he algorihm. This is he case when many recursive divide-and-conquer algorihms have o be used, including backrack search, game ree search and branch-and-bound compuaion. When all queue lenghs and produciviies (performance) of nodes are known, hen he bes conrol sraegy is a proporional disribuion of asks such ha q 1 /θ 1 = q 2 /θ 2 = = q m /θ m. The proof of his resul is no difficul and could be found, for example, in Amelina e al. (2015). This conrol sraegy is called load balancing. The reasons menioned above allow us o reformulae he considering problem: he goal is o mainain he balanced (equal) load across he nework. Assume, ha he following condiions are saisfied A1: Graph G A is undireced, and i has a spanning ree. A2: θ i θ min > 0, i N, = 0, 1,.... (Noe, if Assumpion A1 is saisfied hen 0 < Re(λ 2 (A)) (see Lewis e al. (2014))). If we ake x i = q/θ i i as a sae of agen i of considered dynamic nework a ime insans = 0, 1..., hen he conrol goal of achieving consensus in nework will correspond o he opimal redisribuion of asks among agens (see Amelina e al. (2015)). Under his noaion, he dynamics of each agen can be rewrien as x i +1 = x i + f i + ũ i, (5) where f i = z i /θ i 1, and ũ i = ū i /θ i, i N are normalized conrol acions. We can rewrie Equaion (5) in he vecor form x +1 = x + f + u, (6) where m-vecors x, f, and u consis of corresponding elemens x 1,..., x m, f 1,..., f m, and ũ 1,..., ũ m. If undireced graph G A has a spanning ree, he load balancing problem can be reformulaed as a minimizaion problem of a Laplacian poenial associaed wih graph G A (see Olfai-Saber and Murray (2004)) 947
June 24-26, 2015. Sain Peersburg, Russia Φ (x ) = 1 2 n j=1 subjec o m a i,j (x j x i ) 2 min x, (7) m m x i θ i = q 1, i (8) since Φ (x ) = 0 for he case x 1 = x 2 =... = x m and Φ (x ) > 0 for all oher cases. Is is also menioned in Olfai- Saber and Murray (2004) ha local voing proocol (see, e.g., Amelina e al. (2015)) is equivalen o gradien descen for Laplacian poenial. Linear consrain (8) is naural for problems of asks redisribuion because we canno loss he asks during a redisribuion process. To solve he problem (7),(8) we could use he algorihm and resul from Granichin (2015). 4. MEAN-RISK OPTIMIZATION PROBLEM UNDER LINEAR CONSTRAINS Consider a se of differeniable funcions {f w (θ)} w W, f w (θ) : R m R, le x 1, x 2,... be he se of observaion poins chosen by experimener. For each = 1, 2,... we ge measuremens y 1, y 2,... of f w ( ) wih addiive exernal noise v y = f w (x ) + v, (9) where {w } is an unconrollable sequence, w W. Le (Ω, F, P ) be he underlying probabiliy space, and le F 1 be he σ-algebra of all probabilisic evens occurred before = 1, 2,.... The problem is o find opimal θ ha minimizes mean-risk funcional F (θ) = E F 1 f w (θ) min (10) θ subjec o linear consrains H θ = q 1 (11) wih marices H of dimension k m and vecors q 1 R k, 0 k < m (wih k = 0 i is assumed ha here is no consrains). Hereandafer E is a symbol for mean value and E F 1 is a symbol for condiional mahemaical expecaion wih respec o F 1,, is a scalar produc of wo vecors, is an Euclidean norm of a vecor. If rankh = k hen here exiss linear funcion h : R m R m k and is reverse funcion g : R m k R m such as x = g (h (x)), x M = {H x = q 1 }. We assume ha h ( ) could always be chosen. Le n, n = 1, 2,... be an observed sequence of independen random variables in R m k, called he simulaneous es perurbaion, wih Bernoulli disribuion which elemens equal ±1 wih probabiliies 1 2. Le us ake a fixed iniial vecor θ 0 R m and choose posiive numbers α and β. Consider he algorihm x ± n = g 2n 1± 1 (h 2 2n 1± 1 ( θ 2 2n 2 ) ± β n ), θ 2n 1 = g 2n 1 (h 2n 1 ( θ 2n 2 )), (12) y n θ + yn 2n = g 2n (h 2n ( θ 2n 1 ) α n ), 2β which is similar o one proposed in Granichin (2015) when H ( ) does no depend on. Nex, we assume he following abou f w (x), F (x) and uncerainies in he model: A3: Funcion F ( ) has unique minimum poin θ and z R m k z h (θ ), E F 1 z f w (g (z)) µ z h (θ ) 2 wih a consan µ > 0. A4: w W gradien z f w (g (z)) saisfies he Lipschiz condiion: z, z R d k z f w (z ) z f w (z ) M z z wih a consan M µ. A5: Vecor-gradien f ( ) is uniformly bounded in poin h (θ ): E f (h(θ )) c 1, E f (h(θ )) 2 c 2, E f (h(θ )), f 1 (h(θ 1)) c 2 (c 1 = c 2 = 0 if w is nonrandom, i.e. f w (x) = F (x)). A6: For n = 1, 2,..., a) n and w 2n 1, w 2n (if hey are random) do no depend on σ-algebra F 2n 2. b) If w 2n 1, w 2n are random hen random vecors n and elemens w 2n 1, w 2n are independen. c) he successive differences v n = v 2n v 2n 1 of observaion noises are bounded: v n c v <, or E v n 2 c 2 v, if a sequence {v } is random. d) If v n is random hen v n and vecor n are independen. A7: Marices H 2n 1 and H 2n (if hey are random) do no depend on σ-algebra F 2n 2. A8: The drif is bounded: h (θ θ 1)) δ θ <, or E h (θ θ 1) 2 δθ 2 and E h (θ θ 1) h(θ 1 θ 2) δθ 2, if a sequence {w } is random. The rae of drif is bounded in a such way ha z R d k : E F2n 2 φ n (z) 2 c 3 z h (θ2n 2) 2 + c 4, where φ n (x) = f w2n (x) f w2n 1 (x). Denoe κ = 2(µ αγ), b = 2βMc 3 (1 + 6αMc2 ) + δ θ(m + 2µ + 6αM 2 c 4 ), l = 2αc 2 (c 2 c v + 3(max 4 n 2β + c2 (c 2 + M 2 (δ θ +2βc ) 2 )))+2δ θ (4βMc 3 +Mδ θ +c 1 +3µδθ 2 ), where γ = 3c 2 (M 2 c 2 + c 3 2β ). The following Theorem shows he asympoically efficien mean-squared weak upper bound of esimaion residuals by algorihm (12). Theorem 1. If rankh = k, assumpions A3-A8 hold, and α is sufficienly small: α (0; µ/γ) if µ 2 > 2γ, or α (0; µ µ 2 2γ 2γ ) ( µ+ µ 2 2γ 2γ ; µ/γ) oherwise, hen he sequence of esimaes provided by he algorihm (12) has asympoically efficien mean-squared weak upper bound of esimaion residuals L = (b + b 2 + κ l)/κ, (13) i.e. ε > 0 N such ha n > N E θ 2n θ 2n 2 L + ε. Proof of Theorem 1 is slighly differen from he correspondence proof in Granichin (2015) since we consider more complicaed problem seing and addiional Assumpion A5. 5. TASK REDISTRIBUTION PROTOCOL Generally, o ensure load balancing across a nework (in order o increase he overall hroughpu of a sysem and o reduce 948
June 24-26, 2015. Sain Peersburg, Russia execuion ime) i is naurally o use he redisribuion proocol over ime. Minimum poin x of (7) vary over ime due o he sysem dynamics (6). Consider SPSA algorihm (12) wih nonvanishing sep-sizes for racking he changes x using h (x) = h(x) = col(x 1,..., x m 1 ) m g (z) = col(z 1,..., z m 1, qi 1 d 1 j=1 zj θ i ). We have iniial guess x 0 which is formed by q i 0/θ i 0, i = 1,..., m. Le α > 0 and β > 0 be fairly small sep-sizes. The ieraion sep consiss of Compue wo values y ± n = Φ 2n 1± 1 2 (g 2n 1± 1 2 (h( x 1) ± β )); (14) Compue quasigradien vecor n = n y + n y n 2β Ge new esimae x 2n 1 = g 2n 1 (h( x 2n 2 ); θ m ; (15) x 2n = g 2n (h( x 2n 2 ) α n ). (16) We canno use (16) in decenralized load balancing problem since each agen is able o use informaion abou is neighbors only. Consider he ih componen of he quasigradien vecor from (15). By virue (7) and (14) we have i = i Φ ( x 1 + β ) Φ ( x 1 β ) = 2β i 1 n n a k,j 4β k=1 j=1 ( (x j + β j x k β k ) 2 (x j β j x k + β k ) 2). By using he difference of squares (formula: a 2 b 2 = (a b)(a + b)), we derive n n i = i a k,j ( j k )(x j x k ) = k=1 j=1 (a i,j + a j,i )(1 i j )(x j x i )+ j N i n n a k,j ( j k )(x j x k ), i k i j i since ( i ) 2 = 1. Denoing η i = n k i n j i ak,j ( j k )(x j x k ) we ge i = 2 j N i a i,j (1 i j )(x j x i ) + i η i. Following by he SPSA ieraion sep (16) we could consider decenralized conrol proocol u i = α ( ) a i,j (1 i j θ i ) θ j y i,j y i,i, i N, (17) j N i where α > 0 is a sep-size of conrol proocol (17). For each i N he dynamics of he closed loop sysem wih proocol (17) is as follows x i +1 = x i + f i +α ( ) a i,j y i,j (1 i j ) θ j yi,i θ i. (18) j N i If we denoe marix B = [b i,j ], where b i,j = a i,j (1 i j ), hen properies of a similar conrol algorihm, called a local voing proocol, for a load balancing problem were sudied in Amelina e al. (2015). The common feaure is ha he conrol value of he local voing proocol for each agen was deermined by he weighed sum of differences beween he informaion abou he sae of he agen and he informaion abou is neighbors saes. However, he analysis in Amelina e al. (2015) was done only for he case of saisical noise (noise wih Gaussian disribuion) wih sandard zero-mean and bounded covariance properies. Here we consider he randomized modificaion (he special case) of he local voing proocol, which was inspired by SPSA mehods. Probably we could use weaker condiions abou observaion noise {v i,j } and disurbances f i if we assume he independence of simulaneous es perurbaion on noise and disurbances (see Granichin e al. (2015)). 5.1 Connecion o gossip algorihms For considered SPSA algorihm we used Bernoulli disribued simulaneous perurbaion bu in fac we can use differen wihou losing core properies of he SPSA. In Granichin e al. (2015) required condiions are presened (chaper 3, condiions (3.8)). One possible disribuion is as follows: 0, k i, j k n = ±1, w.p. 1 2, k = i or k = j 1, Assuring i n = j n k = i or k = j If we apply such o (18) here will be only wo nonzero coordinaes and whole sum conains single nonzero erm. Muliplier ( j n k n) is ±2 for nonzero erms and he sign doesn affec summary value. Wih ha, considered sochasic approximaion procedure can be described as ieraively pick random pair of indices and average values of corresponding coordinaes (value of β affecs how much he values are drawn o heir average). Such algorihms was previously sudied in Boyd e al. (2006) and are known as gossip algorihms. As we menioned in previous secions, SPSA algorihm works wih arbirary bounded noises. Wih ha, gossip algorihms should work wih arbirary noises as well if choice of a pair of indices does no correlae wih exernal noise. 6. ASYMPTOTIC MEAN SQUARE ε-consensus Definiion 1. n agens are said o achieve he asympoic mean square ε-consensus, if E x i 0 2 <, i N, and here exiss a sequence {x } such ha for all i N. lim E x i x 2 ε Assume ha he following assumpions are saisfied: A9: a) For each i, j = 1,..., m vecor and ξ i,j (if i is random) are independen. b) For each i = 1,..., m vecor and z i (if i is random) are independen. 949
June 24-26, 2015. Sain Peersburg, Russia c) For each i = 1,..., m vecor and θ i (if i is random) are independen. d) For all i, j N, = 1, 2,... observaion noise ξ i,j is bounded: ξ i,j c ξ <, or E(ξ i,j ) 2 c 2 ξ if ξi,j is random. e) For each i = 1,..., m z i is bounded: z i c z <, or E(z) i 2 c 2 z if z i is random. Le x 0 be he average of he iniial daa x 0 = 1 n x i 0 n and {x } is he rajecory of he averaged sysem x +1 = x + 1 n f i. (19) n Theorem 1 allows o derive he level of upper bound of he asympoic mean square ε-consensus: lim E x x 1 m 2 ε wih some ε which can be calculaed using Theorem 1resul. Here 1 m is m-vecor of ones, For considered case µ = Re(λ 2 (A)), M = 4d max (A). To verify he applicabiliy of Theorem 1 we need o check ha: 1: The funcion Φ ( ) is srongly convex on subspace X = {x R m : x T 1 m = x T 1 m }, i.e. i has a unique minimum poin x 1 m and (x x 1 m ) T Φ (x) µ x x 1 m 2, x X wih a consan µ = Re(λ 2 (A)) > 0. Calculaing he derivaives, we ge Φ (x) n n = a i,j (x j x i ) + a j,i (x i x j ). i j=1 j=1 Hence, gradien-vecor Φ (x) equals o 2L(A)x. The vecor 1 m is he righ eigenvecor of Laplacian marix L(A) and corresponding o he zero eigenvalue: L(A)1 m = 0. Sums of all elemens in rows of marix L(A) is equal o zero and, moreover, all he diagonal elemens are posiive and equal o he absolue value of he sum of all oher elemens in he row. By virue Assumpion A1 marix A has a spanning ree. By Lemma 2.10 from Ren and Beard (2008) he rank of marix L(A) equals m 1. Hence we can derive (x x 1 m ) T Φ (x) = 2(x x 1 m ) T L(A)x = 2(x x 1 m ) T L(A)(x x 1 m ) Re(λ 2 (A)) x x 1 m 2, 2: By using Gershgorin crieria (see Lewis e al. (2014)), we ge ha he gradien Φ (x) saisfies he Lipschiz condiion: x, x R m Φ (x ) Φ (x ) = 2 L(A)(x x ) 4d max (A) x x wih a consan M = 4d max (A) µ = Re(λ 2 (A)). 3: By virue Assumpion A9a he vecor does no depend on observaion noise and drif. 4: By virue Assumpion A9d he observaion noise saisfies: ( ) a i,j ξ i,j (1 i j ) θ j ξi,i θ i j N i 4d max (A)c v /θ min <. 5: By virue Assumpion A9e he drif is bounded: x 1 m x 11 m = 1 m f i m m max{1, c z /θ min 1} n = δ x <. 7. SIMULATION RESULTS To illusrae he heoreical resuls we consider he decenralized compuer nework of m = 50 compuing nodes. We will show ha he proposed randomized conrol algorihm (17) provides load balancing of he nework similar o he one presened in Fig. 1. Fig. 1. The nework opology. The nework opology is a ring wih chords which are randomly chosen by he following rule for every node: (1) simulae a number of added chords by a Poisson disribuion wih mean value m/2 (2) randomly selec nodes ha aach o he curren (he number of such unis is equal o he value obained in sep 1) We generae he iniial produciviies θ 1, θ 2,..., θ m randomly by he uniform disribuion over he inerval (10; 50). We assume ha produciviy measure in our case is he number of available jobs in ime insan = 0, 1,..., he produciviies do no change over ime and θ i 0 i. The asks are divided ino wo ses: regular and burs. The firs one is served on each ac o a randomly chosen node and he second one a any given ime. During sysem operaion we will be adding regular asks from he inerval (12; 100) and burs asks from (10000; 25000). Fig. 2 shows he dependence of algorihm convergence rae on choosing of coefficien α. In Fig. 3, we can see he sysem of m = 50 nodes operaing in nonsaionary case wih he conrol proocol (17). Each line indicaes how he load x i evolves over ime. For clariy, he char displays 3 maximum and 3 minimum values. These lines also show how he sysem evolves o reach load-balancing or consensus. We can see ha even when he new burs ask se is received during he sysem work, i does no affec he qualiy of load balancing. During he simulaion we have se he coefficien α = 0.007, which is he mos suiable value for he curren opology and chosen parameers (see Fig. 2). In addiion 950
June 24-26, 2015. Sain Peersburg, Russia Fig. 2. Rae of convergence based on α. o he obained resuls, i is planned o sudy he possibiliy of SPSA applicaion for racking he opimal value of α. Fig. 3. Perfomance of he sysem wih m = 50 nodes x i for he nonsaionary case. 8. CONCLUSION In his paper he problem of load balancing in a muli-agen sysem under unknown bu bounded disurbances was examined. To solve he load balancing problem he new randomized local voing proocol wih nonvanishing sep-size was proposed. Condiions for achieving an approximae consensus (balance of he nework load) were obained. To illusrae he heoreical resuls we presened he simulaions for he compuing nework. REFERENCES Amelin, K., Amelina, N., Granichin, O., Granichina, O., and Andrievsky, B. (2013). Randomized algorihm for uavs group fligh opimizaion. 11h IFAC Inernaional Workshop on Adapaion and Learning in Conrol and Signal Processing, 205 208. Amelina, N. and Fradkov, A. (2012). Approximae consensus in he dynamic sochasic nework wih incomplee informaion and measuremen delays. Auomaion and Remoe Conrol, 73(11), 1765 1783. Amelina, N., Fradkov, A., Jiang, Y., and Vergados, D. (2015). Approximae consensus in sochasic neworks wih applicaion o load balancing. IEEE Transacions on Informaion Theory, 61(4), 1739 1752. Boyd, S., Ghosh, A., Prabhakar, B., and Shah, D. (2006). Randomized gossip algorihms. IEEE Transacions on Informaion Theory, 52(6), 2508 2530. Cheboarev, P.Y. and Agaev, R.P. (2009). Coordinaion in muliagen sysems and laplacian specra of digraphs. Auomaion and Remoe Conrol, 70(3), 469 483. Granichin, O. and Amelina, N. (2015). Simulaneous perurbaion sochasic approximaion for racking under unknown bu bounded disurbances. IEEE Transacions on Auomaic Conrol, 60(5). Granichin, O., Skobelev, P., Lada, A., Mayorov, I., and Tsarev, A. (2012). Comparing adapive and non-adapive models of cargo ransporaion in muli-agen sysem for real ime ruck scheduling. Proceedings of he 4h Inernaional Join Conference on Compuaional Inelligence, 282 285. Granichin, O., Volkovich, Z.V., and Toledano-Kiai, D. (2015). Randomized Algorihms in Auomaic Conrol and Daa Mining. Springer. Granichin, O. (1989). A sochasic recursive procedure wih correlaed noise in he observaion, ha employs rial perurbaions a he inpu. Vesnik Leningrad Universiy: Mah, 22(1), 27 31. Granichin, O. (1992). Unknown funcion minimum poin esimaion under dependen noise. Problems of Informaion Transmission, 28(2), 16 20. Granichin, O. (2015). Sochasic approximaion search algorihms wih randomizaion a he inpu. Auomaion and Remoe Conrol, 76(5), 761 774. Huang, M. and Manon, J. (2009). Coordinaion and consensus of neworked agens wih noisy measuremens: sochasic algorihms and asympoic behavior. SIAM Journal on Conrol and Opimizaion, 48(1), 134 161. Kar, S. and Moura, J.M. (2009). Disribued consensus algorihms in sensor neworks wih imperfec communicaion: Link failures and channel noise. IEEE Transacions on Signal Processing, 57(1), 355 369. Lewis, F.L., Zhang, H., Hengser-Movric, K., and Das, A. (2014). Cooperaive conrol of muli-agen sysems: opimal and adapive design approaches. Springer Publishing Company, Incorporaed. Li, T. and Zhang, J. (2009). Mean square average-consensus under measuremen noises and fixed opologies: Necessary and sufficien condiions. Auomaica, 45(8), 1929 1936. Olfai-Saber, R., Fax, J., and Murray, R. (2007). Consensus and cooperaion in neworked muli-agen sysems. Proceedings of he IEEE, 95(1), 215 233. Olfai-Saber, R. and Murray, R. (2004). Consensus problems in neworks of agens wih swiching opology and ime-delays. IEEE Transacions on Auomaic Conrol, 49(9), 1520 1533. Polyak, B.T. and Tsybakov, A.B. (1990). Opimal order of accuracy of search algorihms in sochasic opimizaion. Problemy Peredachi Informasii, 26(2), 45 53. Rasrigin, L. (1963). The convergence of he random search mehod in he exremal conrol of a many-parameer sysem. Auomaion and Remoe Conrol, 24(10), 1337 1342. Ren, W. and Beard, R. (2008). Disribued Consensus in Mulivehicle Cooperaive Conrol. Communicaions and Conrol Engineering. Springer. Ren, W., Beard, R., and Akins, E. (2007). Informaion consensus in mulivehicle cooperaive conrol. Conrol Sysems, IEEE, 27(2), 71 82. Spall, J.C. (1992). Mulivariae sochasic approximaion using a simulaneous perurbaion gradien approximaion. IEEE Transacions on Auomaic Conrol, 37(3), 332 341. Tsisiklis, J., Bersekas, D., and Ahans, M. (1986). Disribued asynchronous deerminisic and sochasic gradien opimizaion algorihms. IEEE Transacions on Auomaic Conrol, 31(9), 803 812. 951